Yann Dubois
Stanford CS PhD student who worked on Alpaca (Llama 7B fine-tuned on 52k LLM-generated instructions), AlpacaFarm and AlpacaEval (GPT-4 as a judge, with a length correction). Dubois's Summer 2024 CS229 guest lecture, "Building Large Language Models", is the archive's most technical walk through the llm-training-pipeline.
The lecture's position is that academia over-invests in architecture while industry wins on data, evaluation and systems. It backs this with scaling-laws: small architecture changes mostly shift a curve's intercept, while better data improves the whole curve. Dubois's advice, after Sutton's "bitter lesson": "do the simple things, do them well, scale them."
Source: CS229 lecture
Linked from
LLM training pipelineModel parameters (weights)Open weightsScaling laws