A neural network is a mathematical function made of layers of simple units: each unit multiplies its inputs by learned weights, adds them up, and passes the sum through an activation function, and the layers feed forward into an output guess. Training compares that guess with the correct answer, measures the error with a loss function, and uses backpropagation with gradient descent to adjust every weight slightly in the direction that shrinks the error, repeated over many examples, per Google's Machine Learning Crash Course and Stanford's CS231n notes. What the network has learned lives entirely in those weight values; the loose analogy to biological neurons is historical, not a description of how brains work.
Reading time
— 4 min
Updated
— Sep 25, 2026
Fact-reviewed
— Sep 25, 2026
Key Takeaways
Key Takeaways
1Each unit in a neural network does something very simple: multiply its inputs by weights, add them up, and pass the result through an activation function.
2Training is a loop: make a guess, measure how wrong it was with a loss function, then use backpropagation to nudge every weight slightly toward a better guess.
3Everything the network "knows" lives in its weight values. The brain comparison is a historical inspiration, not a description of how it works.
The concept
Think of a neural network as a huge panel of volume knobs. Numbers go in one side (the pixels of a photo, say), pass through several rows of knobs, and a guess comes out the other side ("cat, 80% sure"). At the start, the knobs are set randomly, so the guesses are junk. Training shows the network thousands or millions of labeled examples. After each one, it checks how wrong it was and turns every knob a tiny bit in the direction that would have made the guess less wrong. Do that enough times and the knob settings end up encoding patterns, like edges, then shapes, then whole objects. Nobody writes those patterns in by hand. They fall out of the knob-turning.
That "many layers" part is where the term deep learning comes from, covered in more detail in machine learning vs. deep learning. The same multiply-add-squash building block, scaled up to billions of weights and trained on text, is what sits underneath large language models.
Quick check
A network's guesses get better over training. What actually changed inside it?
Worked examples
Example 1: One unit, by hand (baseline case)
A unit gets two inputs, 2 and 3, with weights 0.5 and −1, and a bias of 1. Weighted sum: (2 × 0.5) + (3 × −1) + 1 = 1 − 3 + 1 = −1. A ReLU activation turns anything negative into 0, so this unit outputs 0. That's it. Every unit in even the largest network is doing this same small calculation; the power comes from having millions of them wired in layers.
Example 2: One training step on a digit recognizer (variation)
A network sees a handwritten "7" and outputs 30% for "7" and 50% for "1". The loss function turns that into a single error score. Backpropagation works out, for each weight, whether increasing it would have raised or lowered the error. Each weight moves a tiny step the helpful way. Shown the same image again, the network might now say 34% "7". One step barely matters; tens of thousands of steps across many different sevens are what make it reliable.
Example 3: When the pattern it learned is the wrong one (applied case)
Suppose most training photos of wolves happen to show snow. The network lowers its loss fastest by weighting "white background" heavily, so it starts calling a husky on snow a wolf. Nothing in the math told it to look at the animal. It found whatever pattern reduced the error. This is why the quality and balance of training data matter as much as the network design.
Quick check
In Example 1, why did the unit output 0 instead of -1?
How it works (visual)
The training loop: guess, measure the miss, adjust the weights
Common mistakes
Common Mistakes
✕
Picturing each artificial neuron as a tiny thinking unit.
→ Each unit is a multiply-add-and-squash calculation. Any apparent intelligence comes from millions of these combined, not from any one of them.
✕
Assuming a network that scores well in testing has learned the 'right' concept.
→ It learned whatever pattern reduced the loss on its training data, which can be a shortcut like background color. Test on varied, realistic data.
✕
Thinking more layers always means a better model.
→ Deeper networks need more data and computing power and can overfit. The right size depends on the problem.
Common misconception
“Neural networks work the way the human brain works.”
The name comes from a 1940s-50s simplification of a neuron, and MIT News describes the brain link as loose inspiration. Real neurons use timed electrical spikes, chemistry, and feedback that artificial units don't model, and brains don't learn by running backpropagation over labeled examples. A modern neural network is best understood as a very large, adjustable mathematical function, not a digital brain.
What to do next
What to do next
Work through the single-unit arithmetic in Example 1 yourself with different weights. It makes every later concept concrete.
When you read an AI claim, ask what data the model was trained on, since that's where its learned patterns come from.
Try Google's free Machine Learning Crash Course neural-network module if you want the math with interactive visuals.
Read how the same building block scales up in our explainer on large language models.
FAQ
FAQ
Related terms
Related terms
Neural network
A machine learning model built from layers of simple computing units, each combining its inputs with learned weights, so that the whole stack can map inputs (like pixels) to outputs (like labels).
Weight
A number attached to each connection in a neural network that scales how much one unit's output influences the next unit. Training is the process of finding good weight values.
Activation function
A simple non-linear function (such as ReLU, which turns negative numbers into zero) applied after each unit's weighted sum, which lets stacked layers represent curved, complex patterns rather than only straight-line ones.
Loss function
A formula that scores how wrong the network's output is compared with the correct answer. Training tries to make this number as small as possible.
Backpropagation
The method for calculating how much each weight contributed to the error, working backward from the output layer, so every weight can be adjusted in the direction that reduces the loss.
This entry was researched from public sources and drafted with AI-assisted tools, then edited — errors are still possible. Spot one, or want a topic covered? Read our disclaimer.
KnowThis