Predicting Excess Stock Returns Out of Sample: Can Anything Beat the Historical Average?
Source: Campbell, J. Y. & Thompson, S. B. (2008) · Review of Financial Studies 21(4), 1509–1531 · DOI: 10.1093/rfs/hhm055
TL;DR
A constructive reply to Goyal & Welch (2006/2008), who showed that predictive regressions of the equity premium fail to beat the prevailing historical mean out of sample (OOS). Campbell & Thompson show that imposing weak, theory-motivated restrictions — forcing the slope coefficient to have the sign theory predicts and forcing the equity-premium forecast to be non-negative — turns negative OOS R² values positive. The OOS predictability is small but economically meaningful for a mean-variance investor.
Problem it solves
Naive OLS predictive regressions overfit in short, volatile return samples and can produce economically absurd forecasts (e.g., negative equity premia). Goyal & Welch documented that, as a result, the historical-average benchmark beats most predictors OOS — calling stock-return predictability into question. The method recovers usable predictability without abandoning OOS discipline.
The method
Estimate standard predictive regressions of monthly (and annual, via overlapping data) excess returns on a predictor (D/P, E/P, smoothed E/P, B/M, ROE, T-bill rate, long yield, term spread, default spread, inflation), expanding the estimation window.
Apply two restrictions, sequentially and jointly:
1. Coefficient sign restriction — set the slope to zero whenever its estimate has the "wrong" (theory-contradicting) sign.
2. Forecast restriction — set the equity-premium forecast to zero whenever it goes negative.
Optionally go further with steady-state valuation-model restrictions (zero-intercept, unit-slope on the valuation ratio), removing the need to estimate the mean from a short, noisy sample.
Evaluate with an OOS R² comparable to the in-sample R², benchmarked against the prevailing historical mean.
Assumptions & inputs
Monthly CRSP total-return data since 1927; predictor series through end of 2005. OOS evaluation begins in 1927 or 20 years after a predictor first becomes available, whichever is later (subsamples 1927–1956, 1956–1980, 1980–2005).
Requires a defensible sign prior and a valuation-model link for the steady-state version.
How to use it
Headline result: monthly OOS R² runs from −0.66% to 0.32% with no restrictions, −0.45% to 0.43% with the sign/forecast restrictions, and 0.24% to 0.97% with the zero-intercept/unit-slope steady-state restrictions — the restrictions improve every regression considered.
Translate small R² into economic value: a mean-variance investor with the paper's parameters (monthly buy-and-hold Sharpe ≈ 0.108, annual ≈ 0.374) gains a proportional return increase of about R²/S² (e.g., 0.43%/1.2% ≈ 36%) — so a tiny R² can justify a meaningful tilt.
Limitations & pitfalls
The OOS R² is genuinely small; sign/forecast restrictions help least at the annual frequency, where some ratios still don't beat the mean.
Restrictions inject a prior — wrong priors bias forecasts; the method does not resolve the persistence/Stambaugh-bias inference problems, only the OOS-forecast stability.
Results are sample- and predictor-dependent (e.g., the dividend-price ratio weakens post-1980 as buybacks rise).
Key references
Campbell, J. Y. & Thompson, S. B. (2008) — Predicting Excess Stock Returns Out of Sample — Review of Financial Studies
Goyal, A. & Welch, I. (2008) — A Comprehensive Look at the Empirical Performance of Equity Premium Prediction — Review of Financial Studies
Rapach, D., Strauss, J. & Zhou, G. (2010) — Out-of-Sample Equity Premium Prediction: Combination Forecasts and Links to the Real Economy — Review of Financial Studies
Provenance: verified/generated from the paper's full text.
