A type of Distribution Shift
Often, the cause of an error is explained by the finiteness of the training set. In such cases, additional training examples will solidify the model. However, in many practical scenarios, the model starts to make errors because of concept drift.
Concept drift is a fundamental change in the statistical relationship between the features and the label.
Imagine your model predicts whether a user will like certain content on a website. Over time, the preferences of some users may start to change, perhaps due to aging, or because a user discovers something new (I didn’t listen to jazz three years ago, now I do!). The examples added to the training data in the past no longer reflect some user’s preferences and start hurting the model performance, rather than contributing to it. This is concept drift. Consider it if you see a decreasing trend in model performance on new data.