Weights vs context
A model’s persistent learned numbers and the text supplied to a request are different places for information to influence a prediction.
| Question | Weights | Context |
|---|---|---|
| Form | Learned numerical parameters | Token sequence available to this request |
| How it changes | Training or explicit parameter editing | Adding, removing, or replacing supplied messages |
| Lifetime | Persists in the saved model | Depends on the application and current request |
| Capacity | Distributed representation, not a reliable document archive | Finite window with explicit token costs |
| Retrieval | Cannot ordinarily inspect a weight to recover a citation | Can supply passages with source information |
Putting a new bus timetable in a prompt makes it available to this request. It does not, by itself, update the model for every future user.
Follow the evidence
Read the connected lessons for detailed mechanisms, assumptions, and primary references.
Parameters: the numbers learning changes
Weights shape a model’s computation. They are different from the text you put in a prompt.
Context windows: what is available right now
The current sequence is bounded. Fitting information into it and successfully using that information are separate questions.
Context management: choosing what stays in view
A finite input budget makes selecting, preserving, and refreshing context part of the application’s job.
Retrieval: giving the model evidence
A retrieval system finds relevant material and places it where a language model can use it.