Open weights

A model is open-weight when its creator publishes the trained parameters, so anyone can download it, host it, and build on it. With a closed model like ChatGPT or Claude you only get the vendor's app or API.

andrej-karpathy breaks a release into two parts [43:04]. One is the code for the network's forward pass, "a couple hundred lines" and "fairly standard". The other is the parameters, "where the actual value is": for GPT-2 (2019), one list of about 1.5 billion numbers. See model-parameters.

• GPT-2 (OpenAI, 2019) was a base model released with its weights.
• Llama 3.1 405B (Meta) is Karpathy's hands-on base model; he notes companies rarely publish base models, because a base model is "only a step one" before post-training (llm-training-pipeline).
• DeepSeek-R1 [2:35:50, 3:16:06] is "an MIT license model... anyone can download them, anyone can host their own version." Karpathy calls it "unprecedented that a model this strong was released with open weights."

"Open weights" is narrower than open source. The training data and training code usually stay private, and licenses vary: R1 is MIT, while Llama ships under Meta's own license.

Running one takes hardware. Full-precision R1 "you won't run on a MacBook", so people use inference providers that host the weights: together.ai for chat models, Hyperbolic for base models, and Perplexity hosts R1 in its model picker. Smaller models run locally in LM Studio.

A side effect shows up in dhh's interview: a Chinese open-weight model answered a Tiananmen question bluntly, while a US frontier model refused to translate an essay. Each model's refusals come from whoever post-trained it.

Sources: Deep Dive · How I use LLMs · DHH on Lex Fridman