The perceptron, and why it stalled (1958 to 1969)

In July 1958 Frank Rosenblatt, a psychologist at the Cornell Aeronautical Laboratory, showed reporters a program running on an IBM 704. He called it the perceptron. The idea was simple. Take inputs, multiply each by a weight, add them up, and fire if the sum passes a threshold. Show it examples. When it gets one wrong, nudge the weights toward the right answer. His paper that year in Psychological Review set out the model and the learning rule.

The New York Times ran the story under the headline New Navy Device Learns By Doing, and quoted the Navy's hope that the machine would one day walk, talk, see and write. The Times reported that after fifty tries the program could tell cards marked on the left from cards marked on the right. The hardware version, the Mark I Perceptron, used a 20 by 20 grid of photocells, 400 of them, wired to motor driven potentiometers that held the weights.

The important part is not the hardware. It is that the weights were learned, not set by hand. A proof by Albert Novikoff in 1962 showed that if a straight line can separate the two classes, the update rule finds such a line after a bounded number of mistakes. That is the perceptron convergence theorem. It is the first time anyone had a learning rule with a guarantee behind it.

Rosenblatt wrote carefully about what the model could and could not do. The press did not. The gap between the paper and the headline is the oldest pattern in this field, and you will see it in every lesson of this part.

In 1969 Marvin Minsky and Seymour Papert published Perceptrons, a book of proofs about what a single layer can represent. The famous case is XOR: output 1 when exactly one of two inputs is 1. No single line separates those four points, so no single layer perceptron can learn it, however long you train. They also showed that parity and connectedness need units that look at a number of inputs that grows with the size of the image. The limits scaled badly.

Minsky and Papert knew that stacking layers removes the limit. The problem was that nobody had a learning rule for the layer in the middle. The perceptron rule needs to know the right answer for each unit, and a hidden unit has no right answer written down anywhere. Funding moved to symbolic AI. The field calls the next decade and a half the first AI winter. Rosenblatt died in a boating accident in 1971, and the people who kept working on learning machines did it quietly.

Say that the perceptron stalled on a proof, not on a lack of effort. Then say what the proof left open: credit assignment. When the final output is wrong, how much of the blame belongs to each weight in the middle? Answer that and several layers become trainable, and XOR is a toy. It took until 1986 for the answer to reach a wide audience. That is lesson 0.2.