Forward propagation
Forward propagation is the process in a neural network where input data is passed through the layers to produce an output or prediction. It proceeds through the following steps.
- Input: The input data (features) is fed into the network.
- Linear transformation (weighted sum): Each neuron multiplies the input by a weight (which represents the relative importance of the input) and adds a constant value—a bias term, which allows the neuron to activate even when the input values are zero.
- Non-linear transformation (activation): The weighted sum is passed through an activation function, which introduces non-linearity, enabling the network to learn complex relationships rather than just simple ones.
- Layer-by-layer propagation. The activated output becomes the input to the next layer. This sequence of weighted sums and activations continues across the hidden layers, allowing the network to gradually extract and refine patterns from the data.
- Output: The final (output) layer produces the prediction.
Forward propagation produces the network’s prediction. To improve accuracy, the network then needs to learn from its mistakes.
This is done through backpropagation. The network compares its output with the correct, or target, value using a loss function, which measures the error. That error is then propagated backward through the network to determine how much each weight and bias contributed to the result.
Based on this information, the network updates its parameters by applying an optimization method such as gradient descent.
Forward propagation is used in image recognition, natural language processing (NLP) tasks like sentiment analysis, autonomous systems, financial forecasting, speech recognition, etc.
