Overfitting and Extrapolation in Tree Ensembles: Two Distinct Failures in Short-Sample FDI Forecasting

Abstract: Tree ensembles are recommended for macroeconomic forecasting because they capture non-linear structure, and they are known in principle to be unable to predict outside the range of target values seen in training. This paper documents both properties in one short-sample forecasting problem. Comparing training and test error for three models forecasting Nigerian FDI one quarter ahead on 62 observations, gradient boosting achieves a training RMSE of 2.80 against a test RMSE of 983.64, a ratio of 351 to 1. Random forest and support vector regression degrade far less (2.24 and 2.61 times respectively) from much looser training fits. The extreme ratio for gradient boosting is straightforward overfitting on 49 training observations. Separately, the training targets are strictly positive, running from 58.72 to 3,084.90, while the test period contains four negative quarters and five observations below the training minimum. A tree ensemble trained only on positive targets cannot emit a negative prediction, because its output is a weighted average of training-set leaf values. Both effects are present and they are distinct. We separate them, and note that on a full walk- forward comparison gradient boosting on an enhanced feature set was the best of twelve specifications, so neither effect supports a general claim about tree ensembles in this domain.

Keywords: Gradient Boosting; Random Forest; Overfitting; Extrapolation; Short Samples.