Category — Forecasting
[Editor’s note: J. Scott Armstrong and Kesten C. Green, first time guest posters, are leading researchers in the field of forecasting. Scott Armstrong is a Professor at the Wharton School of the University of Pennsylvania and Kesten Green is a Senior Research Fellow at the Business and Economic Forecasting Unit at Monash University]
We have recently proposed a model that provides forecasts that are over seven times more accurate than forecasts from the procedures used by the United Nations Intergovernmental Panel on Climate Change (IPCC).
This important finding, which we report in an article titled “Validity of climate change forecasting for public policy decision making” in the latest issue of the International Journal of Forecasting, is the result of a collaboration between climate scientist Willie Soon of the Harvard-Smithsonian Center for Astrophysics, and ourselves.
In an earlier paper, we found that the IPCC’s approach to forecasting climate violated 72 principles of forecasting. To put this in context, would you put your children on a trans-Atlantic flight if you knew that the plane had failed engineering checks for 72 out of 127 relevant items on the checklist?
The IPCC violations of forecasting principles were partly due to their use of models that were too complex for the situation. Contrary to everyday thinking, complex models provide forecasts that are less accurate than forecasts from simple models when the situation is complex and uncertain.
Confident that a forecasting model that followed scientific forecasting principles would provide forecasts that were more accurate than those provided by the IPCC, we asked Willie Soon to join us in developing a model that was more consistent with forecasting principles and knowledge about climate.
The forecasting model we chose was the so-called “naïve” model. The naïve model assumes that things will remain the same. It is such a simple model that people are generally not aware of its power. In contrast to the IPCC’s central forecast that global mean temperatures will rise by 3?C over a century, our naïve model simply forecasts that temperatures next year and for each of 100 years into the future would remain the same as the last years’.
The naïve model approach is confusing to non-forecasters who are aware that temperatures have always varied. Moreover, much has been made of the observation that the temperature series that the IPCC use shows a broadly upward trend since 1850 and that this is coincident with increasing industrialization and associated increases in manmade carbon dioxide gas emissions.
In order to test the naïve model, we simulated making annual forecasts from one to 100 years in the future starting with 1850’s global average temperature as our forecast for the years 1851 to 1950. Then we repeated this process updating for each year up through 2007. This produced 10,750 annual average temperature forecasts for all horizons. It was the first time that the IPCC’s forecasting procedures had been subject to a large-scale test of the accuracy of the forecasts that they produce.
Over all the forecasts, the IPCC error was 7.7 times larger than the error from the naïve model. [Read more →]
November 23, 2009 7 Comments