Limits & tradeoffs
Look beyond parameter counts and benchmark headlines. Examine compute, latency, compression, and the difference between generalization and recall.
By the endAsk precise questions about model quality, cost, and evidence.
Start path: Scaling laws: trends, not guarantees →7 lessons · 42 min total
Mark lessons as read to save your place on this browser.
- 16 MIN READING · Not read yet
Scaling laws: trends, not guarantees
Treat scaling results as measured trends under assumptions.
Read the lesson → - 26 MIN READING · Not read yet
Compute: the work behind the words
Account for the work and hardware behind a run.
Read the lesson → - 36 MIN READING · Not read yet
Latency: why a response takes time
Separate time to first token from generation speed.
Read the lesson → - 46 MIN READING · Not read yet
Quantization: fewer bits per number
Explore the accuracy and memory tradeoff of lower precision.
Read the lesson → - 56 MIN READING · Not read yet
Memorization: when training text comes back
Recognize that training examples may be reproduced.
Read the lesson → - 66 MIN READING · Not read yet
Generalization: beyond the examples seen
Ask whether success transfers beyond familiar examples.
Read the lesson → - 76 MIN READING · Not read yet
Evaluation: measuring the behavior you need
Read the benchmark design before trusting the headline.
Read the lesson →