Replicating, stress-testing, and extending the experiments from Anthropic's blog post "Teaching Claude Why."
Replicating, stress-testing, and extending the experiments from Anthropic's blog post "Teaching Claude Why."
Project Details
Updated 07/17/26 · Edited by orgWe are verifying and extending the main experiments in Anthropic’s Teaching Claude Why (TCW) with open-source midsize models (~32B) as part of the Second Look research fellowship. In TCW, Anthropic introduces new mid-training alignment techniques with synthetic document fine-tuning (SDF) and supervised fine-tuning (SFT) to align and attach the model to the "Claude" character that behaves in accordance with Claude’s constitution. They found that this technique successfully mitigates agentic misalignment propensities, generalizes well out-of-distribution (OOD), and the effect persists through reinforcement learning (RL) post-training.
At a high level, our replication tests the following claims:
-
Does the SDF corpus's effectiveness hinge on an AI assistant persona or simply on generalizable principles illustrated in the data? How much does the mitigation rely on the Personal Selection Model?
-
How important is the reasoning aspect in the SDF for instilling moral decision-making?
-
Do improvements from SDF persist through long-run alignment RL and/or capabilities RL?
Second Look is committed to funding replications of core experiments in the post. However, this funding only covers a minimum viable product with single training seeds and limited evaluations. Additional funding would allow us to
-
Run each main experiment multiple times with extra evaluations to provide uncertainty measurements
-
Test SDF + SFT result persistency in much longer RL training steps (i.e., the depth of the persona attachment)
-
Derive a scaling law for the SDF corpus size to understand how much safety data is needed to efficiently mitigate misalignment
Details about each experiment and additional alternative hypothesis testing can be found in our full implementation details document.
The concrete output will be an open-source replication of TCW with additional experiments and ablation studies. We will publish a series of detailed write-ups of the research design and methods, all associated codes, model training checkpoints, and high-quality datasets. Inventing better alignment techniques is out of scope for this project, but we believe that our work could inspire them.
Second Look fellows will conduct the project with oversight from the core team:
-
Anastasia Wei (leading the project, will be first author)
-
Stewy Slocum (mentor, previously xAI)
-
Arav Dhoot, Finn Cairns, Jack Thompson, Brandon Qi (co-authors)
-
Yixiong Hao, Zephaniah Roe (research management)
-
Harshul Basava (operations/logistics)
Theory of Impact
Updated 07/17/26 · By grantmaking.aiAnthropic’s TCW blog describes some of the only alignment techniques shown to be practical and effective in production frontier models. It’s plausible that Anthropic's current alignment plan relies on implementing mid-training with constitutional documents (roughly as detailed in TCW), and then handing off alignment research to (hopefully) aligned human-level AIs. Since mid-training alignment is a central part of their strategy for aligning superintelligence, these techniques must be verified to hold up and scale. Otherwise, recursive self-improvement could easily amplify minor hidden alignment failures into catastrophic outcomes.
Additionally, whether these techniques are robust, generalize OOD, and persist through RL directly impacts whether other frontier labs should adopt similar practices. Notably, in OpenAI’s replication of self-fulfilling misalignment, they found that mid-training (mis)alignment largely washes out over longer post-training RL runs. While Anthropic partially demonstrated alignment persistence through harmlessness RL, they did not test it against capability RL, and it’s unclear whether they ran a sufficient number of RL optimization steps. Therefore, we believe that verifying the long-term persistence of this alignment technique is especially load-bearing.
People
Updated 07/17/26 · Edited by orgProject Leader
Team Member
Discussion
I'm excited about this! In addition to benefiting the broader research community, this replication would also benefit other labs, most of whom do not yet have great versions of the techniques in the Teaching Claude Why blogpost. In addition, this could be much more thorough than the original Anthropic post, which is quite light on details. As a result, it seems likely this ends up carrying a bunch of novel research instead of only being a replication.
Disclosure: I'm a co-founder of Second Look Research, the host org of this project. However, needless to say, I do endorse this project for the exact theory of change stated above!
More open science and stress testing research & evaluation results from frontier labs is important to prevent safety washing and has nice yet vague forward chaining impacts.
Hi @Anastasia Wei and team,
I respectfully endorse this project, and I read it slowly before that.
It is difficult not to notice that the field keeps building on results nobody has re-run, including the one you address. I also remember that publication where Anthropic showed that models trained on explanations of why a model's behaviour is right or wrong come out steadier than models trained on bare rules. If that holds, in my opinion, it is one of the cheapest alignment levers we have, because it sits upstream, in the training data. The upstream lever is the one I bet my own work on, so this replication matters to me personally.
What I like furthermore: you do not just repeat the result, you stress-test its load-bearing parts as well. It checks whether the lesson sticks to the persona it was taught with, whether the reasoning itself is doing the work, and whether it survives later RL. I believe those are exactly the three places where it would quietly break.
You have a big team, and I send my best wishes to you all.
I also didn't publish my detailed budget here, and honestly I have no idea whether it affects the reviewers' decisions..