Generalization

Generalization is the ability of a machine learning model to perform well on new, previously unseen data drawn from the same distribution as the training data. A model that achieves 95% accuracy on both training and test sets has generalized well, it learned patterns that transfer beyond the specific examples it saw. This is the fundamental goal of machine learning: building models that capture underlying structure rather than memorizing examples. The generalization gap is the difference between training and test performance. A large gap indicates overfitting; the model learned training-specific details that don't apply to new data. Zero gap suggests the model might have more capacity available, it could potentially fit training data better and still generalize. Factors affecting generalization include training data quantity and quality, model architecture and capacity, regularization techniques, and the match between training and deployment distributions. Distribution shift, when real-world data differs from training data, is a major generalization failure mode in production systems. Deep learning's success stems from unexpectedly strong generalization: despite having enough parameters to memorize training data, neural networks often learn generalizable features instead. Understanding why this happens is an active research area called deep learning theory.