Summary
Forecasting is an essential input to the work of the MPC, and, increasingly since the advent of the ‘One Bank’ strategy, into the work of the Financial Policy Committee and the Prudential Regulation Committee. The publication of forecasts, which occurs quarterly in each Inflation Report, is also an important output of the MPC, and forms a key part of the Committee’s communication strategy.
In this paper, the IEO details the work programme initiated towards the end of 2014 to provide Court with a better basis for evaluating the Bank’s forecast performance.
Findings
In general, the IEO’s empirical results suggest that there are a number of areas where the Bank’s core forecasts have performed well from a statistical standpoint.
Recommendations
The IEO’s work did, however, suggest two aspects of performance that merit further investigation. First, the IEO observed a tendency for some two year ahead projections to be less accurate than one year ahead projections, particularly so in the 2010–14 period. The second area of note relates to the Bank’s unemployment rate forecasting record: of the eleven variables evaluated, the unemployment rate forecasts performed the least well against our statistical tests, including tests of unbiasedness and efficiency. More broadly, the IEO’s empirical work, together with the numerous interactions the IEO had with producers and (internal) users of the Bank’s macroeconomic forecasts as part of this study, suggest that the following recommendations could assist the Bank in improving its forecasting capability: learn more from other forecasters and models; learn more systematically from the past; challenge conventions more; and provide more support for non-MPC internal users of the Bank’s forecasts.
Follow up
In September 2018, the IEO reported back to Court on the implementation and impact of the 2015 IEO review of forecasting performance. James Mitchell of Warwick Business School, who had supported the original review, had been involved in the latest work. This noted that the original recommendations had been implemented, but that the Bank could still go a little further in integrating the new tools into its processes. Following the 2015 Review some 'non-structural' models had been introduced as a source of challenge, and outputs were routinely shown to the MPC as a way of cross checking the main forecast. Some but not all members found them helpful, but there was no desire to develop more models of this sort. Internally they had not been integrated into the forecast process as a source of challenge. Another potential source of external challenge – formal MPC meetings with external academics – hadn’t been scheduled for a couple of years and would be reintroduced. The 2015 Review had asked MA to improve its forecast evaluation techniques. Staff had developed an innovative approach to spotting changes in forecast performance relatively quickly. MA was also asked to promote wider Bank’s understanding of the forecast – which is used in other areas, for example as the baseline for stress testing. Regular briefing of FPC members on the forecast had been well-received, and technical secondments had improved understanding at the staff level. Overall the IEO was content that all of the 2015 recommendations had been adopted and implemented. It was not clear what impact they had had, or whether fuller implementation of the proposals would have made any difference to forecast performance, given the relatively short period of time that had passed and the significant shock to the economy in Summer 2016.
Evaluating forecast performance