Chapter 10: The Algorithm That Put Paid to a Persistent Myth

The narrative traces how Geoffrey Hinton, David Rumelhart, and Ronald Williams formalized a mathematically rigorous method for adjusting weights across multiple layers by leveraging the chain rule of calculus to compute error gradients. The chapter establishes the historical context by discussing Frank Rosenblatt's early intuitions about error correction and acknowledges contributions from researchers like Paul Werbos and Shun'ichi Amari whose work preceded the mainstream recognition of the method. Central to the explanation is how gradient descent iteratively minimizes loss functions by moving network parameters in directions that reduce prediction errors, with sigmoid activation functions providing the differentiability necessary for this process. The chapter addresses the XOR problem, which demonstrated that single-layer networks could not learn certain nonlinear mappings and necessitated hidden layers capable of discovering internal representations. A critical discussion examines symmetry breaking through random weight initialization, explaining why careful initialization prevents networks from learning identical features across neurons. The chapter also documents the resistance from symbolic artificial intelligence researchers including Marvin Minsky and Seymour Papert, who initially dismissed neural approaches, and explores how this skepticism temporarily stalled progress in the field. By integrating historical narrative with mathematical intuition, the chapter reveals backpropagation not merely as an algorithmic technique but as a paradigm shift that resolved longstanding theoretical limitations and established the methodological foundation upon which contemporary neural networks, multilayer perceptrons, and deep learning architectures are built.