Indrek Saul: Cost of Estonian institute's change of methodology a cool billion

How much can one small methodological change cost the economy? According to Indrek Saul, the "price tag" of the Estonian Institute of Economic Research's methodology shift is in the ballpark of a billion euros.
My job is to help companies identify growth opportunities and put them into action. Accelerating the growth of a country and a business is, in broad terms, quite similar. First, it's necessary to understand what drives GDP growth — such as private consumption, exports and so on. Second, one must understand the factors that influence those drivers. For instance, private consumption is primarily affected by consumers' incomes, levels of spending and saving and their confidence in the future.
Consumer confidence has been in a very strange place over the past three years — unusually low and not aligned with the objective macroeconomic picture. This has been written about repeatedly by Kaspar Oja, Tõnu Mertsina, Andres Sutt, Mihkel Nestor, Jürgen Ligi and myself.
News of a new methodology
The recent revelation about the Estonian Institute of Economic Research's (EKI) mishandling of the consumer barometer methodology finally sheds light on this mystery. In May 2022, EKI made a significant methodological change in its consumer barometer, switching from telephone interviews to online surveys.
An analysis by the SALK think tank showed that the distortion caused by this change accounts for more than half of the dramatic decline in consumer sentiment reported by the institute. Respondents in online panels rate their economic situation an average of 16.7 points lower than those surveyed by phone, yet the institute has continued for a third year to present the data as if no methodological change had occurred.
This isn't just a statistical issue. Distorted data affects the investment climate and companies' strategic decisions.
Confidence indicator as a budgeting driver
Yes, the distorted picture originated with the change in methodology — without any public disclosure, without explaining to decision-makers what impact this change might have had on the results. And unfortunately, the impact wasn't limited to just an average decline. The consumer confidence barometer has a direct influence on the state budget and economic forecasts.
From the Ministry of Finance's 2025 summer economic forecast, we read:
"Macroeconomic forecast, point 1.2, Domestic Demand: Consumer confidence has been low and the downward trend has yet to reverse. The growth in residents' purchasing power will support increased consumption starting in 2026. Improving economic growth prospects will also boost private sector investment confidence."
The country's official economic policy thus directly relies on EKI's distorted data, shaping forecasts and policy measures based on false assumptions. In turn, this cycle adds further fuel to austerity and tax hike policies, which in themselves deepen pessimistic expectations.
Detailed data confirms effect of methodology flaw
The consumer barometer's composite confidence indicator is calculated as the arithmetic average of four components: families' economic situation over the past 12 months, the forecast of families' economic situation for the next 12 months, the forecast of the country's economic situation for the next 12 months and families' forecast for purchasing durable goods over the next 12 months.
Let's look at how assessments of the families' economic situation changed before and after the methodology shift. The average value of this indicator from April 2021 to April 2022 was –5. From May 2022 to May 2023, the average dropped to –33. A significant difference.
Before the methodological change, the average downward trend was –0.7 points per calendar month. After the change, there was a slight upward trend of +0.3 points per month. However, the decline between May and June alone followed a trend of –10 points per month — fifteen times greater than in the previous period.

If we cut the two months of anomalous decline from the graph and replace them with the earlier trend, the picture looks entirely different. The margin of error is 18 points. Eighteen! That's one hundred percent!
How is it possible that no one at EKI thought to question why, in the two months following the methodological change, families rated their economic situation 15 times worse — only to suddenly rate it just as good as before? It's the same nonsense we saw at Statistics Estonia, where it didn't occur to anyone to investigate what caused the unusually high wage growth.
One possible explanation is that poor indicators conveniently fit Peeter Raudsepp's narrative that "everything is bad." He'll surely object and argue that this is the real and more accurate picture. Which raises an interesting question in itself: does that mean the previous measurements were wrong?
Excel skills leave a lot to be desired
As a side note, here are some observations about the Excel file of the consumer barometer that's shared with the public. The formatting contains several mistakes typical of a beginner-level Excel user. Below is a brief overview of basic rules for formatting and using tables — rules that EKI has clearly violated:
- Format your data range as a Table using Ctrl+T. This makes the data dynamic, gives it a clear structure and simplifies referencing.
- Avoid empty rows and columns. Breaks in the data table disrupt its logic and lead to analytical errors.
- Structure the data top-down. Excel tools like PivotTables, Power Query and charts rely on a vertical data structure, which also makes the data more readable and reliable.
- Format dates correctly. Dates must be entered as Excel date values, not as text, and they should not be split into separate columns for month and year. Doing so hampers sorting, filtering and analysis.
Is EKI keeping positive indicators up its sleeve?
The EKI barometer includes other interesting indicators as well — such as families' financial situation, car purchases, home purchases or construction and home or apartment renovations. These are inherently much more objective measures than, say, opinions about how the family or the country will fare over the next twelve months. These "reserve" indicators, which are not used in calculating the composite index, tell a very different and far more positive story about the state of households and the economy.

Once again, the question arises: why does the composite confidence index include the three most subjective indicators when the survey actually collects more objective data? Why is the composite index calculated based on the most negative indicators?
For example, the assessment of families' ability to purchase durable goods over the next 12 months remained relatively unchanged before and after the methodology shift. Most notably, the evaluation of the family's financial situation didn't change significantly at all. In fact, consumers' assessments of their ability to buy a car or a house or to renovate a home or apartment also stayed consistent.
That doesn't square at all with the indicators that form the basis of the composite index: the families' and the country's economic outlook supposedly deteriorated catastrophically, yet the family's financial condition, ability to purchase durable goods and plans for buying or renovating a home remained stable.
"So what, we're still in a recession!"
Yes, we are in a downturn. But the real question isn't whether we're in a downturn — it's how deep that downturn actually is. And what kind of economic growth or contraction we might have seen if consumer confidence had been higher.
If the index didn't exist, how would consumers know whether their family's financial situation is good or bad? That judgment — good or bad — is formed solely and exclusively through comparison with a benchmark. That benchmark might be the family's own past financial situation or information gathered from friends and acquaintances about theirs.
When the state decides to measure the economic situation of thousands of households and the media reports on it regularly, it inevitably begins to influence how consumers perceive their own family's financial standing.
The EKI's methodological error set off a self-reinforcing cycle of negativity:
- a low barometer reading generates pessimistic moods and expectations;
- pessimistic expectations affect consumer purchasing behavior and reduce spending;
- reduced spending is reflected in weaker economic performance;
- weaker economic performance deepens pessimistic expectations;
and the cycle repeats.
Yes, we are whining our way into more poverty
In a recent Äripäev broadcast, EKI Director Peeter Raudsepp once again claimed: "People aren't whining themselves into poverty — they're poor and that's why they whine."
But why are people poor? Because the economy isn't growing. And why isn't the economy growing? Is it connected in some way to consumer confidence — in other words, to this very "whining"?
According to a study by SALK, due to the methodological change, EKI has effectively "whined" us into being 16.7 points poorer. In his response to Delfi, Raudsepp first belittles SALK's analysts with a dismissive line: "Some little SALK group taking an interest..." and then skillfully spins the conversation away from what is clearly a systemic measurement error:
"As a hypothesis, one could suggest that the change in data collection methods caused a shift of 10 points… As a thought experiment, we could position ourselves 10 points higher, which would still leave us in a downward trend."
I find it deeply troubling that the head of a scientific institution — an institute, no less — responsible for measurements critical to the country's economic well-being seems unaware that consumer confidence directly affects economic performance and that large shifts in confidence can tip an economy into contraction or growth.
The correct question to ask is:
"How would Estonia's economy have developed if the Institute of Economic Research hadn't reported record-low, Europe's lowest consumer confidence in the summer of 2022?"
Multiple studies have examined the relationship between the consumer confidence index and economic growth. For instance, Borisov (Borisov, L. (2022) in "Consumer confidence and real economic growth in the eurozone." Baltic Journal of Economic Studies) found that a one-point change in the European CCI leads to a 1.1 percentage point change in economic growth. A study from TalTech (Valgemäe, 2020) found that in Estonia, a one-point change in the local TKI index leads to a 0.04 percentage point change in GDP.
The CCI error caused by the methodology change stands at –1.6 points and the TKI error at –11 points. The corresponding effects on GDP: –1.8 percent and –0.44 percent per quarter. Based on Borisov's analysis, the impact of the measurement error on GDP amounts to €450 million annually; according to Valgemäe's model, €110 million annually.
€0.33– €1.3 billion in three years
Looking at the actual quarterly economic growth figures from the past three years — which have mostly fluctuated between –0.4 percent and +0.3 percent — it's entirely plausible that this error was the underlying cause of the prolonged, stagnant decline.
Did Peeter Raudsepp screw up the economy?
In one respect, Peeter Raudsepp is certainly right — consumer confidence has indeed been in decline. But as we've established in this discussion, had consumers been presented with a fairer picture of the general and prevailing sentiment, the economic downturn would likely have been milder — or perhaps we might have even seen modest growth.
It's also true that the actions of the ruling coalition and the government have eroded consumer confidence. But these kinds of qualitative debates aren't helpful on their own. An analyst's job isn't just to ask whether some strategic action affects growth. In my day-to-day work in strategy, I often see how framing the question that way leads to constant missteps: "But this activity affects the company's growth."
An analyst's — and a strategist's — task is to ask: "How much? To what extent?" That's the only way to sort through the countless things one could do and prioritize the ones one must do. The top priority must always be the actions with the greatest impact on growth — not those we subjectively consider important or have elevated based on gut feeling.
Change of Estonian consumer confidence index in time
To study the "behavior" of the consumer confidence index (TKI), I used the same method I applied at the start of the COVID-19 pandemic to uncover major flaws in how infection rates were being measured. This involves transforming the time series of TKI values into a time series of changes in TKI values and then identifying anomalous changes — those that exceed the upper or lower bounds of standard deviation.

What do we see? In 2020 and 2021, the consumer confidence index fluctuated widely in both directions, but all of those shifts can be reasonably explained by the waves and lulls of the COVID crisis. There were no large, one-sided shifts in the index level; movements in both directions were more or less balanced.
As already mentioned, the index began drifting downward in January, but the decline was moderate — on average, 2.5 points per calendar month.
The real blow came in May, when EKI changed its survey methodology. Instead of dropping by 2.5 points, the index fell four times that in a single month and, in June, it dropped three times more than in the months before. As I noted earlier, two-thirds of that decline — 11 points — was due to the methodology change. If we remove the systematic error from the CCI, the index at the time of the methodology change should have looked like this:

Cost of government's taxes rally in the billions
We can see that the consumer confidence index has experienced four additional major negative shifts that, unlike during the COVID crisis, were not followed by similar positive corrections:
- May 2023: the index dropped by 4.7 points after the coalition's plans for new taxes became public;
- October 2023: the Riigikogu began processing the tax bills and the index fell by 5.1 points;
- August 2024: the motor vehicle tax law was published and the index dropped by 8.0 points;
- July 2025: the 24-percent VAT rate came into force, with a 3.7-point decline.
Let's calculate the monetary impact of these changes in the confidence index on GDP by accumulating the index shifts and their financial consequences.

When all the heavy blows to consumer confidence are added up, their total impact on the TKI amounts to –21.5 points. The monetary impact: between €250 million and €1.6 billion per year, or 0.6 to 4 percent of GDP annually.
How much did the government hope to make through tax hikes?
The goal of raising taxes was to bring more money into the state budget as follows:
- €113 million in 2025
- €751 million in 2026
- €784 million in 2027
- €822 million in 2028
It's a solid plan on paper, but considering how heavily this tax maneuvering has undermined consumer confidence, it's far from certain that the monetary value of lost economic growth won't end up exceeding the expected additional tax revenue.
Finally
The methodological change at the Institute of Economic Research could cost us between €110 million and €450 million annually in lost economic growth. The government's tax rollercoaster, meanwhile, may be costing us €250 million to €1.6 billion per year.
To Peeter Raudsepp's credit, we can say he's not the biggest public enemy — but unfortunately, he and the institute have still "talked down" the economy to the tune of hundreds of millions of euros a year.
As for the barometer itself, it's in serious need of an overhaul — so that consumer confidence and the actual state of the economy are once again in alignment.
--
Editor: Marcus Turovski










