Generalization

Generalization is whether a model does well on new data from the same distribution as the training set. A model at 95% on both train and test has generalized. It learned patterns that transfer, not a list of memorized rows.

That is the actual goal of machine learning. The generalization gap is train performance minus test performance. A large gap is overfitting: the model learned details that only exist in the training sample. A gap near zero can mean leftover capacity. You might still fit the train set better and still transfer.

What moves generalization: how much data you have and how clean it is, the architecture and its size, regularizers, and whether train and production look alike. Distribution shift is the production failure. Real traffic is not the training set. The model looked good in the lab and fails in the wild.

Deep learning often generalizes better than the parameter count would suggest. Networks have enough weights to memorize, and they still learn features that transfer. Why that happens is an open research problem, usually filed under deep learning theory. Until that is settled, you still measure the gap on held-out data. That number is the one that matters.

Generalization is test performance on new samples from the same distribution. That is the actual goal of training.