Adaptive Cross-Validation Under Concept Drift for Time Series Forecasting

Abstract

Time-series forecasting often involves non-stationary data, making i.i.d. validation unreliable and fixed-window protocols vulnerable to leakage and biased error estimates. We propose Adaptive Time-Series Cross-Validation (ATSCV), a drift-aware evaluation framework that uses statistical change-point detection to partition each series into contiguous, approximately stationary regimes, followed by forward-chaining folds that respect those boundaries. By aligning train–validation splits with distributional changes (with emphasis on covariate shift), ATSCV yields leakage-controlled, regime-consistent evaluations and more realistic estimates of out-of-sample performance. We evaluate ATSCV on five equity time series (INTC, META, NVDA, ORCL, TSLA) and four model classes (Linear, RNN, LSTM, GRU), using RMSE and MAE. ATSCV reduces RMSE and MAE typically by 30–50% relative to a drift-blind baseline on four of five assets, while revealing one challenging case (TSLA) where frequent regime changes limit cross-regime transfer. Beyond improving accuracy, the protocol stabilizes model rankings and reveals asset-dependent behavior. Overall, the results indicate that drift-aligned evaluation provides more realistic generalization estimates and clarifies when apparent performance is driven by regime dynamics rather than model capability.