How LLMs WorkA VISUAL GUIDE TO LANGUAGE MODELS
JOURNEY 04 / Behavior is trained, too

From model to assistant

Understand how demonstrations and preferences can shape a pretrained model, without treating one historical recipe as universal.

By the endCompare supervised fine-tuning, reward-based optimization, and direct preference optimization.
Start path: Pretraining: learning from continuations →6 lessons · 36 min total

Mark lessons as read to save your place on this browser.

  1. 1
    6 MIN READING · Not read yet

    Pretraining: learning from continuations

    Start with a model trained to predict text.

    Read the lesson →
  2. 2
    6 MIN READING · Not read yet

    Supervised fine-tuning: demonstrations of behavior

    Learn from examples of desired responses.

    Read the lesson →
  3. 3
    6 MIN READING · Not read yet

    Preference learning: comparing alternatives

    Understand what a preference comparison records.

    Read the lesson →
  4. 4
    6 MIN READING · Not read yet

    RLHF: optimizing a learned reward

    Study a reward-model-based training recipe.

    Read the lesson →
  5. 5
    6 MIN READING · Not read yet

    DPO: learning directly from preference pairs

    Compare a direct preference objective.

    Read the lesson →
  6. 6
    6 MIN READING · Not read yet

    Base model versus assistant

    Separate the resulting model from its interface and tools.

    Read the lesson →