Build an LLM from Scratch

Most courses explain what a large language model is. This one has you build one, by hand, in Python, until there is no magic left.

You will not import a finished model and call it done. You will construct every piece from the ground up: the tokenizer that turns text into numbers, the embeddings that give tokens meaning, the self-attention that lets words read each other, the transformer blocks stacked into a GPT, the training loop that teaches it, and the sampling that makes it write. By the end, the phrase "large language model" describes something you have actually built.

What makes this course different

• You write real code that really runs. Every lab is a live Python editor in your browser (real NumPy, no setup, no install), with an AI tutor that reviews what you wrote and coaches you toward correct, idiomatic solutions.
• Interactive, not passive. Watch text break into tokens, drag a temperature slider and see the next-word distribution sharpen, click through a transformer block, and press Train to watch the loss fall and the samples turn from gibberish into language.
• Deep enough for a professional, gentle enough for a beginner. Every term is defined the first time it appears, then taken all the way down to the mechanics and the math. Nothing is hand-waved.

What you will be able to do

• Explain and implement tokenization, embeddings, attention, and the full transformer, from scratch
• Read and reason about the internals of any modern GPT-style model
• Build, train, and sample from a small language model in Python
• Understand how the toy you built scales into the assistants used every day, and what pretraining, fine-tuning, and human feedback each add

Who this is for

Anyone comfortable with basic Python who wants to truly understand language models, not just use them. Bring curiosity. Leave with a model you built yourself.