A neural network is a mathematical system that transforms inputs through connected processing units. Its design borrows a loose analogy from biological neurons, but an artificial neuron is not a miniature brain cell: it is a compact calculation using inputs, weights, a bias, and an activation function.
What biological neurons inspired the analogy?
Biological neurons receive signals through dendrites, integrate them in the soma, and send output along an axon. Connections between neurons are synapses, whose strengths can change. That arrangement offers a useful way to think about artificial inputs, connection weights, and learning. The University of Toronto’s CSC311 course notes describe the artificial neuron as “far simpler than a real one” and explain that the aim is “a clean mathematical abstraction” rather than biological accuracy.
The distinction matters. Biological signaling includes excitation and inhibition, physical cell dynamics, synaptic plasticity, and recurrent circuits. As the University of Texas Medical School at Houston explains in its Neuroscience Online chapter on neurons and neuronal networks, those features are part of how biological networks operate. They do not mean that brains learn by running machine-learning backpropagation.
What does an artificial neuron calculate?
For an input vector x, corresponding weights w, bias b, activation function f, and output y, a common expression is:
Free tools Windows power users keep installed
One-click scans. No signup required.
#1 Best Overall
y = f(wᵀx + b)
With a single input, the same idea is y = f(wx + b). The one-input form makes each part easier to see; a real unit may combine many inputs. OpenStax’s introduction to artificial neural networks presents both forms.
- Inputs: features from the data, or outputs passed along from earlier units.
- Weights: values that scale each input’s contribution. A positive or negative weight can raise or lower that contribution in the artificial model.
- Bias: an offset added to the weighted sum, letting the unit’s response shift.
- Activation function: a function applied to the sum to produce the unit’s output.
- Output: the resulting value, passed to later units or used as part of the network’s result.
Weights and biases are learned parameters: training adjusts them so the network’s outputs better fit the task. An activation need not be the same at every layer, and the choice affects how the network transforms information.
Why do networks use layers?
Networks organize units into layers. The input layer receives the data; hidden layers perform intermediate transformations; and the output layer produces a result suited to the task, such as a prediction or class score. In a classification model, for example, separate output units may represent classes, with their activations used to select one.
Not every network has a hidden layer. The number of hidden layers can be zero, one, or more, and the connections between units depend on the architecture. A network described as deep commonly has multiple hidden layers, although conventions can differ on whether the input layer counts toward its depth. The NCBI Bookshelf chapter “Fundamentals of Artificial Neural Networks and Deep Learning” discusses these variations.
Recommended Free Tools
Rank #3
Nonlinear activation functions are important in stacked networks. If each layer did only a linear transformation, stacking those layers would still amount to a linear transformation. Nonlinearity lets a network represent more complex relationships and decision boundaries. The right architecture depends on the data and task, required output, connectivity, interpretability, computing resources, and available training data; there is no universally best layer arrangement.
How does training change the network?
In supervised learning, a training example includes an input and a target value. A standard backpropagation training loop uses the current weights and biases to make a prediction, measures its error against the target, and adjusts the parameters. OpenStax outlines the process in its section on training a neural network:
Rank #4
- Forward pass: send the input through the layers using the current parameters and activation functions.
- Calculate loss: use a loss or cost function to score the difference between the prediction and target.
- Backward pass: propagate information about the loss backward through the network to determine how the parameters affect it.
- Update parameters: an optimizer uses that information to change weights and biases in an effort to reduce the loss. Introductory accounts commonly use gradient descent as an example.
- Repeat: run the process over training data until performance is sufficient for the task.
Backpropagation and optimization are related but distinct: backpropagation calculates how the loss changes with the parameters; the optimizer uses those calculations to update them. This is one supervised training setup, not a description of every way neural networks can be trained.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to explore the parts interactively
TensorFlow Playground, linked from OpenStax’s introduction, lets learners adjust features such as hidden layers, neurons, learning rate, and activation choices, then observe training. It is a practical way to see how changing the components affects a model’s behavior.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
