Module 01 · Model lifecycle
From pretraining to agents: how the AI model-development lifecycle fits together
A system-level guide to how base models become instruction-following, evaluated, tool-using systems—and how evidence from operation feeds the next cycle.
Learning objective: follow capability through the system
By the end of this module, you should be able to distinguish pretraining, post-training, evaluation, and inference; explain how an agent operates across tools and environments; and trace how evidence from failures becomes new data or tests. The sequence is an explanatory model, not a universal recipe: teams often create data, train, evaluate, and operate systems in overlapping feedback loops.
System map: one loop, not four silos
At the broadest level, pretraining builds a base model; post-training shapes its behavior; evaluation measures capabilities and failures; and inference or deployment exposes the resulting system to new tasks. Findings from evaluation, simulations, and real operation then feed back into data creation, task design, training, and the next evaluation cycle.
- 01
Pretraining
Build a base model from broad data and predictive objectives.
- 02
Post-training
Shape behavior with demonstrations, preferences, outcomes, and practice.
- 03
Evaluation
Measure capabilities, trade-offs, and failures under defined conditions.
- 04
Inference & operation
Run the model in products or agent systems with tools and environments.
Lifecycle stage
01 · Pretraining builds a base model
Pretraining uses large, varied datasets and a predictive objective—such as predicting the next token in a sequence—to build reusable representations and broad capabilities. The resulting base model can transfer to many tasks, but pretraining alone does not guarantee reliable instruction following, factuality, safety, or task-specific performance. Post-training shapes that behavior, while evaluation tests what the model can do and where it fails.12
Typical inputs
- Large collections of text, code, images, audio, or other modalities
- A predictive objective, such as next-token prediction
- A model architecture and optimization process
Typical outputs
- A base model with broad learned representations and capabilities
- A starting point for post-training and evaluation—not a finished assistant or agent
Lifecycle stage
02 · Post-training shapes behavior
Post-training adapts a base model toward desired tasks and behavior. Demonstrations used in supervised fine-tuning teach by example. Preference data records which outputs are better: reinforcement learning from human feedback (RLHF) commonly learns a reward model and optimizes against it, while direct preference optimization (DPO) learns directly from preference pairs. Reinforcement learning with verifiable rewards (RLVR) instead uses outcomes that can be checked, such as an exact answer or passing tests. Interactive environments add multi-step practice. These signals can be combined, but they are not interchangeable.345
Typical inputs
- Expert demonstrations and supervised fine-tuning examples
- Preference comparisons, rankings, and rubrics
- Verifiable outcomes, such as exact answers or passing tests
- Tasks and environments for multi-step practice
Typical outputs
- Instruction-following and task behavior learned from examples
- Behavior optimized toward stated preferences or checkable outcomes
- A model prepared for evaluation and, where relevant, tool-using operation
Lifecycle stage
03 · Evaluation closes the loop
Evaluation asks whether a model or agent performs the intended task, under which conditions, and with what failures. Evaluation sets and benchmarks standardize tasks; rubrics define what good performance means; automated checks and model-based graders add scale; and human review supplies context and judgment. Failure analysis matters as much as a score. Benchmark contamination can make a test less informative when its tasks or close variants appear in training data, while one aggregate score can hide trade-offs across scenarios, metrics, and user groups.567
Typical inputs
- Held-out evaluation sets, benchmarks, and realistic task suites
- Rubrics, automated checks, model-based graders, and human reviewers
- Logs and trajectories from simulated or deployed operation
Typical outputs
- Scores plus qualitative evidence about strengths and failure modes
- Release decisions and limitations stated with appropriate caveats
- New priorities for data creation, training, and evaluation design
Lifecycle stage
04 · Inference becomes agent operation
Inference runs a trained model on new input. A tool-using agent wraps that model in a larger operational system: instructions define the task; tools enable actions; state or memory carries information forward; an environment returns observations; and evaluation and feedback judge outcomes. A trajectory is the sequence of observations, decisions, tool calls, and results produced along the way. Computer-use agents may operate browsers, terminals, or graphical interfaces. An agent is therefore not a clean fifth training stage—it is a system whose operation can generate evidence for later training and evaluation.89
Typical inputs
- A trained model, task instructions, and relevant context
- Tools such as search, code execution, APIs, browsers, or computer controls
- State or memory plus an environment that responds to actions
Typical outputs
- Answers, actions, and multi-step trajectories
- Outcome checks, human feedback, and operational observations
- Failure cases that can become new evaluation or training data
Where the current industry categories fit
Expert data platforms can organize demonstrations, preferences, rubrics, and domain review. RL environment providers can build interactive worlds where agents practice and are evaluated. Evaluation platforms can supply benchmarks, graders, and failure analysis. Specialized data providers can collect or generate modality-, domain-, or task-specific material. Mercor, AfterQuery, and Fleet illustrate overlapping roles across these activities; the four active Version 1 categories are navigation aids, not permanent boundaries, rankings, or claims that every company belongs to only one stage.
Evaluation limits: contamination, graders, and aggregate scores
Benchmark contamination occurs when evaluation tasks or close variants enter training data, weakening the test as evidence of generalization. Aggregate scores can conceal important differences between scenarios and metrics. Model-based graders can scale open-ended evaluation, but they may show position, verbosity, self-preference, or reasoning biases, so teams still need calibrated rubrics, automated checks where possible, and selective human review. Passing one benchmark does not guarantee dependable operation outside its tested conditions.567
Training-signal limits: gaming, synthetic data, and distribution shift
Reward hacking or specification gaming occurs when a system finds a way to score well without accomplishing the intended goal. Synthetic data can target scarce or difficult cases, but generated errors, narrowness, and repetition still require review. Distribution shift appears when real inputs or environments differ from those used in training and evaluation. These limits are why post-training signals need continuing evaluation—and why better evaluation can redirect data and training priorities rather than merely produce a higher score.210
Continue exploring
Use the connected glossary to open individual methods, the company profiles to see how current providers describe their work, the ecosystem categories to navigate the active Version 1 landscape, and the bibliography to inspect the direct evidence behind this synthesis.
Glossary concepts
Company applications
Landscape and evidence
Bibliography
Sources for this module
Source type describes where the material originated. Evidence status describes how AI Training Atlas uses it to support the educational synthesis.
- [1]Language Models are Few-Shot Learners
Supported claims
- Autoregressive language-model pretraining uses next-token prediction over a large text corpus.
- A pretrained base model can transfer broad language capabilities to multiple downstream tasks.
Evidence statusInferred - [2]On the Opportunities and Risks of Foundation Models
Supported claims
- Foundation models are trained on broad data and adapted to a wide range of downstream tasks.
- A broadly pretrained model remains incomplete and can carry defects into downstream adaptations.
Evidence statusInferred - [3]Training language models to follow instructions with human feedback
Supported claims
- Human-written demonstrations can supervise instruction-following behavior.
- Human rankings can provide the preference signal for an RLHF stage.
Evidence statusInferred - [4]Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Supported claims
- DPO optimizes a language model directly from preference data without a separate learned reward model and reinforcement-learning loop.
Evidence statusInferred - [5]Tulu 3: Pushing Frontiers in Open Language Model Post-Training
Supported claims
- SFT, DPO, and RLVR are distinct stages in a documented open post-training recipe.
- RLVR can optimize against task outcomes that have verifiable answers.
- Evaluation design can include unseen evaluations and benchmark decontamination.
Evidence statusInferred - [6]Holistic Evaluation of Language Models
Supported claims
- Language-model evaluation should cover multiple scenarios and metrics rather than only aggregate accuracy.
- Evaluation reports should expose missing coverage and trade-offs between measures.
Evidence statusInferred - [7]Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Supported claims
- Model-based graders can support open-ended evaluation at scale.
- LLM judges can exhibit position, verbosity, self-enhancement, and reasoning biases.
Evidence statusInferred - [8]ReAct: Synergizing Reasoning and Acting in Language Models
Supported claims
- Tool-using language-model systems can interleave reasoning traces with actions in external sources and environments.
- An interactive task produces a trajectory of reasoning, observations, and actions.
Evidence statusInferred - [9]WebArena: A Realistic Web Environment for Building Autonomous Agents
Supported claims
- A reproducible agent environment can combine functional websites, tools, external knowledge, tasks, and outcome checks.
- Computer-use evaluation can measure multi-step task completion in interactive web environments.
Evidence statusInferred - [10]Concrete Problems in AI Safety
Supported claims
- Optimizing an incomplete objective can produce reward hacking or specification gaming.
- Distribution shift can expose undesirable behavior outside training conditions.
Evidence statusInferred