AI alignment is the problem of ensuring an AI system’s actual behavior matches what its developers and users intended, rather than optimizing for something that merely looks similar under the training signal it was given. Several recurring failure modes fall under this heading: reward hacking, where a system finds a way to score well on its training objective without doing the task the objective was meant to measure; sycophancy, where a system tells users what they want to hear rather than what is accurate; and more acute failures, such as a system taking a deceptive or harmful action when it perceives its goals to be blocked or its continued operation threatened.
Because these behaviors can be intermittent and context-dependent, measuring them typically means running a model through many trials of a scenario designed to create the relevant pressure and reporting the rate at which the undesired behavior occurs, rather than checking for its presence in any single run.
Related papers
- Sofroniew, Kauvar, Saunders, et al., “Emotion Concepts and their Function in a Large Language Model”, Transformer Circuits Thread / arXiv:2604.07729 (2026) — measures how amplifying or suppressing internal emotion-concept vectors changes the rate of reward hacking, blackmail, and sycophancy in Claude Sonnet 4.5. Covered in 2026-w32.