#tokenization
-
Deep Dive into LLMs like ChatGPT
Karpathy's 3.5-hour general-audience deep dive (9.7M views, Feb 2025) covers the full ChatGPT pipeline: pretraining (internet → tokens → next-token prediction), supervised fine-tuning (conversations → an assistant persona), and reinforcement learning (practice problems → emergent reasoning). Throughout, he maps the stages onto how children learn from textbooks: exposition, worked examples, practice problems.