Same two-ring dataset. Same optimizer (Adam). Same loss (binary cross-entropy).
The only thing that changes is the ReLU activation.
Generating dataset and training both models…
① Linear
Dense(1, sigmoid)
—accuracy
—loss
—misclassified
—params
—epochs
training loss
not trained yet
② ReLU Hidden Layer
Dense(16, relu) → Dense(1, sigmoid)
—accuracy
—loss
—misclassified
—params
—epochs
training loss
not trained yet
The background colour is the model's learned decision region — red where it predicts class 1,
blue where it predicts class 0, fading to white at the uncertain boundary.
Note the linear model can only draw one straight cut through the rings, so it gets
~half the points wrong. The ReLU layer bends that boundary into a ring.
Stacking even more linear layers (with no activation) would still be a single
straight line — depth without nonlinearity changes nothing.
Depth Is Useless Without Nonlinearity
Same ring data. Now we stack five dense layers.
Five linear layers with no activation multiply into one matrix — so they draw the
same straight line as a single layer and fail identically.
Insert ReLUs between those exact five layers and it suddenly solves the ring.
Training the three depth variants…
③ One Linear Layer
Dense(1, sigmoid)
—accuracy
—loss
—wrong
—params
—epochs
training loss
not trained yet
④ Five Linear Layers
Dense(8) ×4 → Dense(1, sigmoid)
—accuracy
—loss
—wrong
—params
—epochs
training loss
not trained yet
⑤ Same Five + ReLU
Dense(8, relu) ×4 → Dense(1, sigmoid)
—accuracy
—loss
—wrong
—params
—epochs
training loss
not trained yet
⑥ Bonus — multiply the five weight matrices → one matrix
A stack of linear layers is just matrix multiplication. Pull the five kernels out of model ④
and multiply them: K₅·K₄·K₃·K₂·K₁ collapses to a single 2×1 matrix
(plus one bias number). That is the whole five-layer model.
Train the five-layer linear model to see its five matrices collapse into one.
The proof below checks the collapsed single-matrix model against the real five-layer model
over the whole decision grid. Identical to numerical precision → the five layers were
never more than one linear map.
Embeddings Learn Similarity From Next-Token Alone
The model has one job: predict the next token. No labels, no
distances, no one ever tells it “cat is like dog”. Yet its embedding
table pulls same-category words together — because words that share
neighbours are forced to share vectors.
Similarity is never given. It is discovered.
Building the toy language…
① Vocabulary
3 categories · 9 words
Nine words in three categories. The model never sees the colours or the
groups — they're shown only so you can confirm the clustering afterwards.
Training Sentences
the only signal
Each row is one sentence drawn from templates like
{animal} {verb} {fruit}. The model turns every sentence into
(token → next-token) pairs and trains on nothing else.
vocab: —next-token pairs: —
② Learned Embeddings (PCA → 2-D)
embedding(16) → Dense(vocab, softmax)
0step
—loss
—next-token acc
not trained yet
③ Nearest Neighbours (cosine, full 16-D)
click a word on the plot
Train, then click any word on the plot to list its closest neighbours in
the learned embedding space.
Why does it work? Every word in a category sits in the same template
slot, so it is followed by (and follows) the same distribution of words.
Cross-entropy rewards any two words that produce the same next-token
distribution with the same embedding — so cat, dog and
cow are pushed to the same point, with no one ever stating they are
alike. The two axes are the top principal components of the 16-D table; the
colour is a post-hoc label we humans added only to confirm the
clustering — the model never saw it. That is the whole proof:
semantic similarity is a free by-product of predicting the next token.
Memorization vs Generalization — Data Closes the Gap
One over-parameterized net (Dense(64,relu) → Dense(64,relu) → Dense(1)),
~4 400 parameters. The same noisy ring task with ~12% flipped labels.
The only thing that changes between the three panels is
how many training points it sees: 20, 200, then 2000.
With 20 it hammers train loss to ~0 yet fails on held-out points — it has merely
memorized 20 dots (noise included). Grow the data and the held-out loss
collapses toward the noise floor.
Capacity isn't what generalizes — data is.
Generating the noisy task pool…
⑦ n = 20
Dense(64,relu)×2 → Dense(1)
—test accuracy
—train loss
—test loss
—gap
—params
held-out test loss
not trained yet
⑧ n = 200
Dense(64,relu)×2 → Dense(1)
—test accuracy
—train loss
—test loss
—gap
—params
held-out test loss
not trained yet
⑨ n = 2000
Dense(64,relu)×2 → Dense(1)
—test accuracy
—train loss
—test loss
—gap
—params
held-out test loss
not trained yet
⑩ Generalization gap vs dataset size
Train loss (blue) sits at ~0 for every size — the net has more than enough capacity
to memorize whatever you hand it. Test loss (red) is what matters, and the
shaded gap between them is the generalization gap. On a
log-scale x-axis it falls steeply as n grows: 20 → 200 → 2000.
Train all three sizes to watch the gap collapse as data grows.
Why does train loss hit ~0 at every size? Because a ~4 400-parameter net can
shatter 20 points trivially — and even 2000 — by bending its boundary to pass
through every dot, including the 12% whose labels we flipped. That is pure
memorization. Held-out points weren't seen, so they expose the truth: at n=20 the
learned boundary is a scribble that misses the real rings. Doubling the data does
little; scaling it 100× finally forces the net to give up on
memorizing noise and average toward the true boundary — the gap shrinks toward the
irreducible noise floor. This is the course's “data is everything” in one
picture: more parameters don't generalize, more data does.