Classical machine learning refers to a set of algorithms that predate deep learning and form the foundation of modern AI. Unlike deep learning, which uses large neural networks to automatically extract patterns from raw data, classical ML models typically require human input to define the relevant features before training.
Classical ML models are generally faster to train, cheaper to run, and easier to interpret than deep learning alternatives. Because their logic is more transparent, it's easier to understand why a model produced a particular output. This interpretability is one of the key advantages that explainable AI (XAI) draws on.
The most common classical ML algorithms include
- logistic regression, which classifies input data into discrete categories using a linear boundary;
- support vector machines (SVMs) that separate data into categories by finding the most effective dividing boundary;
- decision trees, which make predictions by splitting data based on feature values and random forests that combine
- multiple trees to improve accuracy; and
- gradient boosting (XGBoost, LightGBM), which builds models sequentially, with each new model correcting the errors of the previous one.
Classical ML covers both supervised learning and unsupervised learning. It remains widely used in production environments, particularly when datasets are small, computational resources are limited, or when the priority is speed and interpretability over raw performance.