Automatic time series modeling and forecasting: A replication case study of forecasting real GDP, the unemployment rate and the impact of leading economic indicators

Figures & data

Figure 1. Time series plot of leading indicator, unemployment rate and unemployment claims

Figure 2. Coefficient estimates, 120-month rolling window, VAR(AIC) model, composite leading indicator

Figure 3. P-values of coefficient estimates, 120-month rolling window, VAR(AIC) model, LEI

Figure 4. Coefficient estimates, 120-month rolling window, VAR(AIC) model, weekly unemployment claims

Figure 5. P-values of coefficient estimates, 120-month rolling window, VAR(AIC) model, weekly unemployment claims.

Table 1. Autometrics analysis of levels of real GDP data

	Plain AR(1) model	Autometrics model I	Autometrics model II
Lagged real GDP	1.004 (significant at 1%)	0.619 (significant at 1%)	1.006 (significant at 1%)
Lagged real GDP partial R^2	99.9%	56.8%	99.9%
Number of indicators	0	30 (all significant at 1%)	28 (9 significant at 5%, 10 significant at 10%
RSS	591.309	168.732	195.091
AR 1–2 test (p-value)	0.00	0.17	0.64
Normality test (p-value)	0.00	0.23	0.01
Heteroscedasticity test (p-value)	0.00	0.14	0.32

Table 2. Autometrics analysis of levels real GDP data, LEI included

	AR(1) Model + leading indicator lags	Autometrics model I	Autometrics model II
Lagged real GDP, lag 1	0.9942 (significant at 1%)	0.7010 (significant at 1%)	1.3574 (significant at 1%)
Lagged real GDP, lag 1, partial R^2	99.7%	67.2%	77.2%
Lagged real GDP, lag 2	n.a.	n.a.	−0.3615 (significant at 1%)
Lagged real GDP, lag 2, partial R^2	n.a.	n.a.	19.7%
Contemporaneous LEI	n.a.	n.a.	1.3572 (significant at 5%)
Contemporaneous LEI partial R^2	n.a.	n.a.	3.51%
Lagged LEI, lag 1	2.1675 (significant at 1%)	n.a.	n.a.
Lagged LEI partial R^2	3.31%	n.a.	n.a.
Number of indicators	0	27 (all significant at 1%)	21 (7 significant at 5%, 8 significant at 10%)
RSS	552.914	184.280	237.785
AR 1–2 test (p-value)	0.00	0.36	0.07
Normality test (p-value)	0.00	0.24	0.08
Heteroscedasticity test (p-value)	0.16	0.01	0.47

Table 3a. Autometrics analysis of levels unemployment (U) data, LEI and weekly unemployment claims included, 1994–2018

	AR(1) Model	Autometrics model #1 with indicators	Autometrics model #2 with indicators and weekly claims	Autometrics model #3 with indicators and LEI
Lagged U	0.6564 (significant at 1%)	0.9680 (significant at 1%)	0.7734 (significant at 1%)	0.6782 (significant at 1%)
Lagged U partial R^2	42.6%	98.6%	91.1%	80.5%
Lagged LEI, lag 1	n.a.	n.a.	n.a.	−0.0456 (significant at 1%)
Lagged LEI, lag 5	n.a.	n.a.	n.a.	0.0198 (significant at 1%)
Lagged LEI, lag 12	n.a.	n.a.	n.a.	−0.0192 (significant at 1%)
Lagged LEI, lag 1, partial R^2	n.a.	n.a.	n.a.	29.9%
Lagged LEI, lag 5, partial R^2	n.a.	n.a.	n.a.	13.7%
Lagged LEI, lag 5, partial R^2	n.a.	n.a.	n.a.	14.3%
Lagged claims, lag 5	n.a.	n.a.	0.0017 (significant at 1%)	n.a.
Lagged claims, lag 6	n.a.	n.a.	0.0015 (significant at 1%)	n.a.
Lagged claims, lag 5, partial R^2	n.a.	n.a.	10.3%	n.a.
Lagged claims, lag 6, partial R^2	n.a.	n.a.	7.2%	n.a.
Number of indicators	0	47 (significant at 1%)	60 (significant at 1%)	65 (significant at 1%)
RSS	337.859	22.3469	12.7725	4.9023
AR 1–2 test (p-value)	0.00	0.65	0.04	0.37
Normality test (p-value)	0.00	0.00	0.05	0.02
Heteroscedasticity test (p-value)	0.00	1.00	0.09	0.27

Table 3b. Autometrics analysis of levels unemployment (U) data, LEI and weekly unemployment claims included, 1959–2018

	Autometrics model #4 with indicators	Autometrics model #5 with indicators and weekly claims	Autometrics model #6 with indicators and LEI	Autometrics model #7 with indicators, LEI and weekly claims
Lagged U	0.9966 (significant at 1%)	0.8568 (significant at 1%)	0.8898 (significant at 1%)	0.5334 (significant at 1%)
Lagged U partial R^2	99.9%	94.4%	95.3%	43.9%
Linear trend	n.a.	n.a.	0.0051	n.a.
Linear trend partial R^2	n.a.	n.a.	28.9%	n.a.
Lagged LEI, lag 1	n.a.	n.a.	−0.0297 (significant at 1%)	n.a.
Lagged LEI, lag 1, partial R^2	n.a.	n.a.	29.2%	n.a.
Lagged LEI, lag 12	n.a.	n.a.	n.a.	−0.0370
Lagged LEI, lag 12, partial R^2	n.a.	n.a.	n.a.	20.1%
Lagged claims, lag 1	n.a.	0.0023 (significant at 1%)	n.a.	0.0026 (significant at 1%)
Lagged claims, lag 1, partial R^2	n.a.	36.4%	n.a.	17.4%
Number of indicators	31 (28 significant at 5%, 3 significant at 10%)	118 (all significant at 5% or lower)	95 (all significant at 5% or lower)	95 (90 significant at 5% or lower)
RSS	14.0580	5.8088	6.7993	5.6877
AR 1–2 test (p-value)	0.13	0.29	0.29	0.26
Normality test (p-value)	0.38	0.85	0.70	0.93
Heteroscedasticity test (p-value)	0.20	1.00	1.00	1.00

Table 4. Performance summary for models for levels of unemployment data, with and without explanatory variables and indicators, 1959–2018

Estimated model	RSS	AR 1–2 (p-value)	Normality (p-value)	Heteroscedasticity (p-value)
AR(1)	22.903	0.00	0.00	0.00
AR(1) with indicators	14.058	0.130	0.38	0.20
AR(1) and lagged claims	17.978	0.00	0.00	0.00
AR(1), lagged claims and indicators	5.809	0.29	0.85	1.00
AR(1) and lagged LEI	21.42	0.00	0.00	0.00
AR(1), lagged LEI and indicators	6.799	0.29	0.70	1.00
AR(1), lagged LEI and lagged claims	15.294	0.00	0.00	0.00
AR(1), lagged LEI, lagged claims and indicators	5.688	0.26	0.93	1.00

Table 5. Forecasting performance comparison, US unemployment rate

Model	RMSE	MAE	d0	d1	pF
	Composite leading indicator, 60-month rolling window
AR(4)	0.746	0.730	0.067	0.988	0.037
BMA	0.747	0.748	0.098	0.983	0.001
ADA-AR	0.740	0.721	0.044	0.992	0.200
ADL	0.728	0.722	0.079	0.986	0.007
	Composite leading indicator, 120-month rolling window
AR(4)	0.734	0.729	0.089	0.986	0.009
VAR(AIC)	0.702	0.714	0.025	0.996	0.674
ADA-AR	0.742	0.728	0.070	0.989	0.043
ADL	0.699	0.717	0.036	0.994	0.431
	Weekly unemployment claims, 60-month rolling window
AR(2)	0.746	0.731	0.067	0.989	0.031
BMA	0.774	0.765	0.134	0.976	0.000
ADA-AR	0.740	0.722	0.068	0.988	0.026
ADL	0.744	0.727	0.072	0.987	0.018
	Weekly unemployment claims, 120-month rolling window
AR(4)	0.725	0.714	0.078	0.986	0.007
VAR(AIC)	0.711	0.720	0.109	0.981	0.000
ADA-AR	0.729	0.715	0.059	0.990	0.052
ADL	0.701	0.702	0.091	0.984	0.000

RMSE and MAE are the root-mean-squared and the mean absolute forecast errors relative to the naïve (no-change) model.

d₀ and d₁ are the estimates from the Mincer–Zarnowitz regression of the actual value on the forecast and under the forecast efficiency assumption these estimates should not differ significantly from 0 and 1, respectively.

p_F is the p-value of the joint F-test for the null hypothesis of d₀ = 0 and d₁ = 1, computed using a HAC covariance matrix.

AR(p) is a univariate autoregressive model of order p, BMA is the bivariate trend-stationary Theta model, ADA-AR is the adaptive averaging AR model, VAR(AIC) is the bivariate vector autoregressive model of order selected by the AIC and ADL is the adaptive learning forecast.