Glossary entry

Preference data

Human or model judgments that compare outputs or score their quality.

Definition

What it means

Preference data records which responses are better, often as rankings, pairwise comparisons, or rubric-based scores. It can train reward models or optimize a model directly.

Why it matters

Role in the system

Many desirable qualities are easier to judge comparatively than to specify as exact rules.

Example

In practice

A physician ranks two proposed diagnostic explanations for accuracy and clarity.