Underfitting occurs when a machine learning model is too simple to learn the underlying patterns in its training data, resulting in poor performance on both training and new, unseen data.
The core issue is high bias, meaning that the model's assumptions are too rigid to capture real relationships in the data. This is the opposite of overfitting, where a model is too complex and memorizes the training data instead of generalizing from it. Finding the right balance between the two is known as the bias-variance tradeoff.
Underfitting is typically caused by
- choosing a model that is too simple for the complexity of the data,
- stopping training too early before the model has learned meaningful patterns,
- using too few or poorly chosen features that do not capture the relevant factors, and
- applying excessive regularization that constrains the model too heavily.
Addressing underfitting usually involves increasing model complexity, improving feature engineering, extending training time, or reducing regularization. In some cases, switching to a more capable algorithm altogether, such as moving from linear regression to a deep learning model, is the most effective solution.