From text to next token.
One prompt. Seven steps. Change the input or temperature, inspect the numbers, and follow the same example all the way through.
Before the first response,
there is learning.
The pipeline above uses fixed weights. This second flow explains how weights come to exist, and why using a model is a different activity from training it.
- 01 / PREPARE
Collect, filter, tokenize
Choose data, remove unwanted material and duplicates where possible, then encode examples. These decisions shape both what the model can learn and what it may miss.
Data quality → - 02 / PRETRAIN
Predict, measure, update
Compute predictions, measure loss against targets, backpropagate gradients, and apply optimizer updates. Repeat over batches and evaluate on held-out material.
Try a real numerical update → - 03 / POST-TRAIN
Shape useful behavior
Demonstrations and preference or reward signals can adapt the model. Recipes vary; not every model uses a separate reward model or the same sequence of stages.
Compare the signals → - 04 / INFER
Use the saved weights
Supply context, run a forward computation, select a token, and continue. Ordinary generation changes the sequence and temporary activations, not the learned parameters.
Training vs inference →
A decoder-transformer teaching path, not a universal architecture or training recipe. See the primary references in the linked lessons and source register.