A convex function is a loss function where the curve bends upward consistently and looks like a smile. The opposite is a concave function, where the curve bends downward. Think of an upside-down U shape.
In machine learning, a function's shape reflects how easy it is to optimize a model's parameters during training.
When the loss function is convex, there is only one point where the error is at its minimum, and the training algorithm will reliably find it. When it is non-convex, there are multiple dips, and the algorithm can get stuck in one that looks like the best solution but isn't.
Share