Objective function

An objective function is a mathematical function that defines the goal of a problem by assigning a numerical value to possible outcomes. It specifies what needs to be minimized or maximized so that a system can determine how good or bad a particular result is relative to that goal.

In optimization problems, the objective function provides a clear target that guides decision-making and parameter updates. By evaluating different outcomes against this function, an algorithm can systematically adjust variables to move toward the best possible solution.

In machine learning (ML), the loss function is a specific type of objective function that is typically minimized. During model training, the model repeatedly checks the loss value to evaluate how well it is performing. If the value is high, the model continues adjusting its parameters using gradients, which shows the direction and size of change needed to reduce the error. As the loss decreases, it signals that the updates are effective and the model is moving closer to the desired outcome.

Objective functions aren’t limited to ML and are used across mathematics, engineering, economics, operations research, etc., to formalize goals and drive optimization processes.