No single indicator explains the whole reality: GDP does not summarize a country's well-being; unemployment does not describe the entire labor market. But no one would stop measuring unemployment because it is an imperfect statistic. The value of an indicator is not to be perfect, but to allow us to see if we are advancing or regressing, and, from there, to decide. That is why the tools that measure reality are a governance infrastructure. PISA tests are one of them. And this week we have learned that Catalonia has lost the possibility of comparing its 2025 results with those of other countries and the past.PISA has known limitations. It neither measures everything that matters in school nor does everything it measures depend exclusively on what students know. But these limitations do not nullify the value of comparison. Precisely because they remain stable from one edition to another, the value of PISA is not a snapshot of a single year, but rather the ability to know if an educational system improves or deteriorates over time. And a trend can only be interpreted if all the snapshots have been taken with the same criteria.Well, this time comparability has gone out the window. The OECD allows for the exclusion of some students in very specific situations, such as certain disabilities or a very recent enrollment without language proficiency, but it sets a ceiling: a maximum of 5%. In Catalonia, 23% have been excluded, almost five times more. But the irregularity is not excluding anyone; what breaks the sample is having done it massively and without anyone checking. There's no need for bad faith: it is enough not to supervise. And the result is that Catalonia has lost precisely what made the test valuable: the possibility of rigorously comparing itself with its past and with others.
And it is here where the error ceases to be a technical anecdote. When a sample ceases to be representative, it's not just that the average turns out a little skewed; what you lose is the ability to know how much you are wrong. A well-made statistic does not give just a figure, but also its margin of error. With representativeness broken, this margin ceases to be reliable, and the data ceases to be useful for decision-making. But the problem is not just statistical. The students who may be excluded from the test due to disabilities or insufficient language proficiency are not a random cut of the classroom: they are often those who face the most difficulties. They are, precisely, those whom the system should support better. An average that rises because we have removed them from the calculation does not tell us that the school is doing better; it tells us that we have stopped looking at a part of the problem that we had to solve.Data not only describes reality; it also determines which reality is visible to those who govern. And governing a more complex society – with more diverse classrooms and deeper inequalities – requires reliable information to know what works and what doesn't. The problem is not so much having missed a PISA edition: there are other ways to take the pulse of the educational system, and a single cycle would hardly have altered any trend. A government can afford to make a mistake in a policy and correct it the next day. But it cannot damage the instruments it needs to understand reality.
