A neural network understands no digit whatsoever; it multiplies and it adds. A 7×7 grid yields 49 input numbers, each hidden neuron takes a weighted sum plus a bias and pushes it through an activation function, and four output neurons compete for the top score while softmax turns those scores into confidence. A lone neuron can only draw one straight line, so XOR defeats it: a hidden layer of two neurons combining two lines is what finally separates the cases. For the same sum z, swapping the activation from step to sigmoid to ReLU gives three quite different response shapes.
A neural network understands no digit whatsoever; it multiplies and it adds. A 7×7 grid yields 49 input numbers, each hidden neuron takes a weighted sum plus a bias and pushes it through an activation function, and four output neurons compete for the top score while softmax turns those scores into confidence. A lone neuron can only draw one straight line, so XOR defeats it: a hidden layer of two neurons combining two lines is what finally separates the cases. For the same sum z, swapping the activation from step to sigmoid to ReLU gives three quite different response shapes.
Related topics in Mind & Machine