A base model, also called a foundation model, is a large AI model trained on vast amounts of data and later adapted to various downstream tasks.
Base models are trained using self-supervised learning, which allows them to develop broad, transferable knowledge at scale. Once pretrained, they can be fine-tuned on smaller, task-specific datasets to improve performance in a particular domain.
Most state-of-the-art models also go through an alignment phase after fine-tuning, using techniques like reinforcement learning from human feedback (RLHF) to make outputs more accurate, helpful, and safe.
There are risks associated with base models, such as inheriting biases from training data, hallucinating plausible-sounding but incorrect information, and the high computational cost—both financial and environmental—of training them at scale.