ABSTRACT

The Central Bank of Brazil annually rewards the top 5 private forecasters of the long run price index “aiming at improving their predictive ability.” We review the evidence of their performance and cannot reject the hypothesis that the forecasters enter the top 5 list randomly.

Subject Areas:

Behavioral Economics, Macroeconomics

Keywords:

Inflation Forecasting, Professional Forecasters, Central Banks, Luck

1. Introduction

As expected from a central bank, the Central Bank of Brazil considers inflation expectations as a key input for monetary policy decisions. A report from February 2019 of its Monetary Policy Committee (a.k.a. Copom) reads: “The committee members reiterated that the next steps in the conduct of monetary policy will continue to depend on the developments in economic activity, the balance of risks, and inflation expectations.” Inflation expectations are a fundamental ingredient in the implementation of Brazil’s inflation targeting regime and here market participants’ surveys play an important role [1] . In its Focus Report (https://www.bcb.gov.br/publicacoes/focus), the Central Bank of Brazil summarizes the statistics calculated considering market expectations collected through the Friday prior to their release every Monday. The report provides weekly forecasts for price indices, economic activity, exchange rate and its base interest rate (called Selic). Of note, the forecasts come from the market participants, not from the central bank. Focus ranks the top 5 forecasters “aiming at encouraging research participants to improve their predictive ability and recognize their analytical efforts” (https://www.bcb.gov.br/controleinflacao/focustop5ranking). This work reviews the past top 5 rankings in the performance of inflation forecast to evaluate whether the Central Bank of Brazil is rewarding “predictive ability” or, alas, plain luck.

2. Materials and Methods

Table 1 shows the top 5 private forecasters of the Brazilian long run price index (called IPCA) for each of the years from 2009 to 2017, and Table 2 presents a summary of the top 5 entries. The methodology employed by the Central Bank of Brazil to measure the forecast error in the third column of Table 1 is explained in its website (https://www.bcb.gov.br/content/monetarypolicy/Top5Institutions/Top-5-Forecasting-Institutions-Short-and-Medium-Run-Rankings-and-Consolidation-of-Methodology.pdf). Figure 1 displays such forecast errors. The year 2015 shows an atypical behavior with larger forecast errors (Table 1). Of note, 2015 was a troubled year that culminated in the process of impeachment of the Brazilian president that began on 2 December 2015. Table 2 suggests we should investigate whether hitting the top 5 once, with an 87.5 percent chance, and twice with a 12.5 percent chance can be explained by luck alone with no need for a narrative of “predictive ability” or forecast talent. We turn to this problem next.

3. Analysis

Consider this analytical framework. If in a given year there are N competitors, from whom n = 5 enter the group of the top 5 forecasters, we assume such an entry is random, and the proper sample design is a simple random sampling without replacement coupled with an analysis based on the hypergeometric distribution [2] .

Source: Central Bank of Brazil.

How many N competitors there are unknown, but considering that the population of competitors is large enough, this analysis can be approximately performed with N = infinity [2] . Thus, under the assumption of randomness, our problem is to assess whether the allocation to the top 5 occurs through a simple random sampling over a population of infinite size.

Now consider our data from the Brazilian private forecasters who enter the top 5 list at least once between 2009 and 2017 (Table 1). From our data, 35 forecasters hit the top 5 in just one occasion, which meant 87.5 percent. Only five hit the top 5 twice, which meant 12.5 percent, and no one entered the top 5 list more than two times (Table 2).

If the sampling assortment process is random with the probability of assortment p constant, we should learn its random nature from the very data in Table 2. Let X be a random variable for the number of times a forecaster hits the top 5 in the time window of nine years. In this situation, it is appropriate to assume X follows a one-displaced binomial distribution [3] given by:

$P\left(X=x\right)=\left(\begin{array}{c}9-1\\ x-1\end{array}\right)p^{x-1}{\left(1-p\right)}^{9-x}$ ,

where $x=1,\cdots ,9$ , with 9 referring to the time window of nine years, and p to the probability of a forecaster to enter again the top 5 list.

The maximum likelihood estimator of probability p is

$\widehat{p}=\frac{\overline{X}-1}{9-1}=0.015625$

where $\overline{X}=1.125$ is the sample mean of the distribution of X. Table 3 displays the observed probabilities from Table 2 along with the expected probabilities found from this model.

Table 4 shows observed and expected frequencies of the number of times a forecaster enters the top 5 list during the same time window. The columns for 2

and for 3 or >3 in Table 3 are pooled in Table 4 to allow for computing the chi-square statistic.

With one degree of freedom, we find ${\chi }^2=0.017$ (p-value = 0.897). Therefore, there is no statistical evidence to reject the hypothesis that the forecasters enter the top 5 list randomly.

4. Discussion

The robustness of our finding could be assessed by extending the sample to consider the experiences of more countries. There are data of inflation forecasts from selected economists for 130 countries from 2015 to 2017 in FocusEconomics (https://www.focus-economics.com/about-us). However, we could not perform an analysis similar to that in Section 3 because the population of forecasters is mixed in such data, and disaggregated data by country are not available.

Can inflation be predicted? Possibly not. Prices depend on demand and supply and monetary prices depend on the supply of money. A central bank can then rely on the quantity theory of money and change the base rate to stabilize the prices through its monetary policy. This will be straightforward as long as the central bank can know with certainty the price expectations of private market participants. The problem is that central banks cannot. In the dynamics of hyperinflations, for example, current and expected inflation diverge at an increasing rate. And in the monetary policy regime of inflation targeting, forecasters may anchor on an increasing current inflation rather than the announced target [4] .

Inflation may end up random after all, and forecasters may not be that good at recognizing randomness. Therefore, their prediction errors cannot be completely removed. Misperception of randomness can be due to cognitive biases or not [5] , but it does exist. Flipping a coin, the chance of heads three times in a row is $\frac{1}{2}\times \frac{1}{2}\times \frac{1}{2}=\frac{1}{8}=0.125$ . And the chance of tails three times in a row is 0.125, too. Because the occurrences “heads three times in a row” and “tails three times in a row” are disjointed, their probability is 0.125 + 0.125 = 0.25. Similarly, the probability of four heads or four tails in a row is 0.125. This is probability theory. Contrary to this, in an iconic experiment participants were asked to write down a series of 150 random coin tosses. Rather than 25 percent, only about 15 percent of the sequences they produced had heads or tails three times in a row. And only 3 percent had four heads or four tails in a row, rather than 12.5 percent [6] . All in all, the Central Bank of Brazil’s pursuit of “improving the predictive ability of private forecasters” might just be wishful thinking.

5. Conclusion

The Central Bank of Brazil annually rewards the top 5 private forecasters of the long run price index “aiming at improving their predictive ability.” We review the evidence of performance for nine years, from 2009 to 2017. Forecasters entered the top 5 list once, with an 87.5 percent chance, and twice with a 12.5 percent chance. We then analyzed these odds and could not reject the hypothesis that the forecasters entered the top 5 list randomly. This suggests the Central Bank of Brazil may be rewarding luck rather than predictive ability.

Acknowledgements

Financial support from CNPq, Capes and FAPDF is acknowledged.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

Cite this paper

Da Silva, S., Matsushita, R. and Fernández, G. (2019) Inflation Forecasting: Rewarding Luck Rather Than Skill? Open Access Library Journal, 6: e5801. https://doi.org/10.4236/oalib.1105801

References