Chapter 2: We Are All Just Numbers Here...

Beginning with William Rowan Hamilton's quaternion discovery, the chapter introduces fundamental concepts including scalars, vectors, and unit vectors that form the basis for representing data in machine learning systems. The author explains vector operations such as addition, subtraction, and scalar multiplication, then progresses to the dot product and its geometric interpretation, showing how these operations capture meaningful relationships between data points in high-dimensional spaces. A central theme is the geometric insight that classification problems can be solved by finding hyperplanes that separate data into distinct regions, a concept made concrete through the perceptron algorithm. The chapter details how perceptrons use the dot product to determine which side of a decision boundary a data point occupies, enabling binary classification tasks. Matrix notation is introduced as a compact and computationally efficient way to represent and manipulate vectors, facilitating the mathematical formalism required for learning algorithms. The perceptron convergence proof demonstrates that when a linearly separable solution exists, the algorithm will discover it in a finite number of steps, providing theoretical assurance that machines can indeed learn from examples. However, the chapter also explores fundamental limitations of single-layer perceptrons through the well-known XOR problem, which revealed that simple linear separators cannot solve all classification tasks. This limitation motivated the development of multi-layer networks and backpropagation algorithms, marking a turning point in artificial intelligence research that moved beyond early optimism toward more sophisticated approaches capable of learning nonlinear decision boundaries.