Activation steering tests or controls a language model’s behavior by directly modifying its internal activations at inference time, rather than by changing its prompt or its weights. A “steering vector” for some concept is typically found by contrasting the model’s internal activations on inputs that do and do not involve that concept, then averaging the difference; adding a scaled copy of that vector back into the model’s residual stream during a later forward pass pushes the model’s internal state toward — or, with a negative scale, away from — the concept.
Because the intervention happens on the model’s internals rather than its input text, a steering experiment that changes a measurable output, such as how often a model exhibits some behavior, is stronger evidence that the underlying direction is causally implicated in that behavior than a purely correlational observation would be. The size of the effect can still depend heavily on how much the vector is amplified and where in the network it is injected.
Related papers
- Sofroniew, Kauvar, Saunders, et al., “Emotion Concepts and their Function in a Large Language Model”, Transformer Circuits Thread / arXiv:2604.07729 (2026) — amplifies and suppresses emotion-concept vectors to test their causal effect on rates of blackmail, reward hacking, and sycophancy. Covered in 2026-w32.
- Peiris, “Functional Emotions or Situational Contexts? A Discriminating Test from the Mythos Preview System Card” (2026) — proposes a test for whether such steering effects reflect a genuine emotion representation or a situational-context confound. Covered in 2026-w32.