From pixels
to attention.
Don’t just read about neural networks.
Take them apart. Change a number.
See what happens.
Different ways to connect information.MATH
INSIDE ↗
One question.
Three ways to connect.
Which parts of the input should influence the output?
A CNN looks nearby. An RNN carries a state forward. Attention lets positions exchange information directly. The difference is a pattern of computation you can see and calculate.
Start with addition, multiplication and vectors. We’ll build the rest together. Each lab uses small, fixed examples so you can verify every result.
Convolution
A shared filter finds patterns in small neighborhoods.
Best first experiment: find an edge →02 SEQUENCE ↗Recurrence
A hidden state passes information from one step to the next.
Follow a memory through time →03 RELATIONSHIPS ↗Attention
Queries compare with keys. The weights combine values.
Make every connection count →Find the pattern.
One window at a time.
Slide a filter across a small image. Every output cell is a dot product. Change a pixel and watch where the effect appears.
A memory.
Updated every step.
Unroll a tiny RNN. Trace the state, then change its recurrent weight to see how an earlier signal can fade or grow.
Choose what
to pay attention to.
No magic. Just matrix products, a scale factor, and a weighted sum. Inspect a query and follow its numbers all the way through.
Attention was the idea.
This is the architecture.
The original 2017 Transformer is an encoder–decoder for sequence transduction. Follow the data through the blocks, then explore how positions are encoded.
Give each position a signature.
Unmasked self-attention needs positional information to distinguish token order. Explore the sinusoidal positional vectors used in the paper.
Inside residuals & normalization.
Add the shortcut, then normalize one token’s features. Change the numbers and inspect each intermediate result.
What happens when a sequence gets longer?
Explore the quadratic size of dense attention scores without allocating a giant matrix.
Can you predict
what happens next?
Short checks, with the reasoning after every answer. Try the labs again whenever a step feels unclear.
Go to the source.
Original explanations and diagrams. Grounded in research and official documentation.
Your progress & privacy
This site uses no accounts, analytics, tracking cookies or external computation services. Chapter progress and quiz answers are saved in this browser only. Export your progress and import it on another device to continue. All six lab calculations run locally; after the page loads, interacting with the labs sends no server requests. Clearing browser storage removes them. Cloudflare serves the site and may process ordinary request metadata. A private GitHub repository does not hide the HTML, CSS or JavaScript delivered to your browser.