Tool selection by freshness

andrej-karpathy's working heuristic for which LLM feature to reach for, based on what needs to get into the context window (params-vs-context-memory):

Situation	Reach for
Frequent, stable knowledge	the model itself (no tools)
Recent, niche, or changing ("is the market open today?", a new TV season)	web search (Perplexity by habit)
Exact text matters (a paper, a book chapter)	file upload or paste ("don't read books alone")
Arithmetic, data, plots	Python interpreter / Advanced Data Analysis
Hard math or code the fast model got wrong	a thinking (RL-trained) model
Multi-source comparison or literature review	deep research (~10 min of search plus thinking)
Building UI or a diagram	Claude Artifacts; for real codebases, Cursor Composer (vibe-coding)

Two caveats: tool availability differs by app, model and tier, so without a code tool Grok and Gemini hallucinate plausible multiplications. Also, start a new chat per topic, because stale tokens distract the model and cost more. He also keeps an "LLM council": he pays for several providers and asks them all the same question.

Every tool's output is still a draft; see verify-like-a-junior-analyst.

Source: report