Definition of overfitting in modeling

Summary

In mathematical modeling, overfitting is when an analysis matches a specific dataset too closely—often by capturing noise rather than the underlying structure—so it performs poorly on additional or unseen data. An overfitted model typically has more parameters (or flexibility) than the data can justify, which allows it to “memorize” the training examples instead of learning generalizable patterns. Overfitting is closely tied to model complexity and the training/selection process. It can occur when the model is chosen based on training performance rather than validation or test performance, and it is more likely when training is done for too long or when data are scarce. A common symptom is decreasing training error while validation (or test) error increases, indicating that the model is fitting idiosyncrasies of the training set rather than learning trends that generalize.

Key Points

• 1

  Overfitting occurs when a model fits training data too closely, often by learning noise, leading to worse performance on unseen data.

• 2

  An overfitted model is typically overly complex (e.g., too many parameters relative to the amount of data), enabling memorization of the training set.

• 3

  Overfitting is more likely when model selection/training focuses on training performance instead of generalization (e.g., validation error rising while training error falls).