Reward modeling supplies a reinforcement-learning system with a reward signal that is itself learned or measured, rather than hand-written from the task’s own formal rules — as a game’s score function might be. The best-known form is RLHF (reinforcement learning from human feedback), where a model is trained to predict which of two outputs human raters would prefer, and that learned preference model then supplies the reward an RL algorithm optimizes against. The standard way to turn a pairwise preference into that scalar reward is the Bradley–Terry model,
fitting a per-response reward so that this probability matches the observed preference data.
The same structure generalizes past human preference to any signal that is expensive or slow to obtain directly but can stand in as a reward — an experimental wet-lab measurement, for instance, in place of a person’s judgment. Whatever the source, the resulting policy is only as good as the reward signal is: it will learn to satisfy whatever the reward model actually measures, which is not automatically the same as the underlying objective the reward model was meant to approximate.
Related papers
- Blalock, Seshadri, Babber, Fahlberg, and Romero, “Functional Alignment of Protein Language Models via Reinforcement Learning with Experimental Feedback”, Machine Learning for Structural Biology Workshop, NeurIPS 2024; bioRxiv preprint (2025) — uses experimentally measured protein fluorescence, rather than a human preference, as the reward signal. Covered in 2026-w32.