MLML Bites
Lessons/Lesson 3

Why one neuron isn't enough

XOR, and the need for layers · 14 min

Your perceptron can learn AND and OR. So let's try XOR — output 1 when the inputs are different. Sounds just as easy. It isn't, and why it isn't is one of the most important ideas in all of deep learning.

Watch a single neuron fail

main.pyPython · numpy

It can't get all four right, no matter how long it trains. Here's the intuition:

A single neuron can only draw one straight line to separate the 1s from the 0s. AND and OR can be split by one line. XOR can't — the two 1s sit on opposite corners. No single straight line separates them.

The fix is the whole idea behind neural networks: stack neurons in layers. One layer carves the space into pieces, the next layer combines those pieces. Two simple lines can do what one cannot. That combining layer is called a hidden layer, and it's exactly what we build next — with backpropagation to train it.

Your turn

Prove XOR is not the neuron's fault by feature engineering. Add a third input feature a AND b to each example (so inputs become [a, b, a*b]) and a third weight. Train on XOR with this extra feature and show the perceptron now reaches 4/4. (This previews what a hidden layer does for you automatically.)

exercise.pyPython · numpy