Upside down in the PISA tests: an error by Catalonia will make it impossible to compare results with other years and territories

BarcelonaSerious setback in the Catalan education system. The results of the PISA tests in Catalonia, which will be made public in early September, will not be comparable with other territories or with previous years' editions. In other words, the PISA report will not allow us to see if the level of Catalan students has improved or worsened. The reason is what the Department of Education has described as a "technical, isolated, and isolated incident." Broadly speaking, Catalonia has not correctly applied the criteria set by the OECD – the initiators of the PISA tests – to justify that a student should not undergo the tests, which has resulted in the sample of students who did take the exams not being representative of Catalan students aged 15. The Government received an alert about the error in March and communicated it this Friday, July 24. An investigation has been opened to clarify the causes of this fiasco.

As explained by the Director General of Curriculum and Professional Development, Sònia Fernández Cuenca, there has been an error in the exemption index, that is, in the percentage of students who have been considered, either due to physical, cognitive, and emotional disabilities or lack of language proficiency, as not having to take the PISA exams. Specifically, the OECD sets the percentage of exempt students at 5%, and in the 2025 tests in Catalonia, this percentage soared to 23.2%. In other words, the number of students who have not taken the PISA tests in Catalonia is too high for the exam results to represent the actual level of students born in 2009. A priori, the sample for the PISA tests in Catalonia was 2,545 students, and of these, up to 591 were decided not to take the exams: six for physical disability, 92 for insufficient language proficiency, and up to 493 for cognitive, emotional, or behavioral disability.

All of this will make the results unrepresentative and, according to the Department of Education and the Ministry of Education, they cannot be compared with the grades from other territories or with Catalonia's own results in previous years.

However, this does not mean that Catalonia will not receive the results of the PISA tests. In fact, the average obtained by Catalan students who have taken the test will be used to determine Spain's results in the OECD report because, according to the Government, the error in the sample does not affect the representativeness at the state level. "If the sample is not balanced, the results are not comparable, but they are valid," defended Fernández Cuenca. He also assured that errors like the one perpetrated in Catalonia are "a phenomenon that the OECD itself has had to manage on other occasions".

"We are facing a technical error, not a problem with the students' level or the quality of the education system. There has been no manipulation or concealment of data," insisted the department representative on several occasions during the appearance she made out of "responsibility and transparency".

What happened?

Although the department has assured that the reasons for the error are still being investigated, they have put forward several facts that warn that "these are not conclusions", but rather provide contextual information surrounding the error. One of these "contextual" facts is that, as Fernández Cuenca detailed, the PISA tests coincided in a very short period with the end-of-stage tests for 4th year of ESO —the old basic competencies— and Catalan tests have broader criteria than those of the OECD when deciding which students are exempt from taking the exam. This could have led educational centers themselves to have "confusion" when applying the correct criteria to each test.

Here it should be taken into account that, as Education has described, although the OECD is involved in the circuit to determine which students pass the tests and which are exempt, along with two companies, Spain and the communities, the final step in applying the exemption criteria is up to the educational centers themselves. Neither the Government nor the ministry validates the final selection a posteriori.

Another fact that the Director General of Curriculum has admitted is that the application of the PISA tests – from March 31 to May 16 – coincided with a "transition process" between the Superior Council for Evaluation (the body that had been responsible for applying the tests for years) and the Agency for Evaluation and Prospective of Education (APE, the new body on which the evaluations of the education system depend). "At the time of the events, the agency's creation process was still underway," admitted Fernández Cuenca.

However, a few hours after these statements, the same agency – which depends on the Generalitat – issued a statement recalling that "APE began its activity on July 7, 2025, that is, after the management and application of PISA 2025 in Catalonia". Nevertheless, they also explain that "given the functions" attributed to the agency, it has become aware of the incident and "is fully aware of the impact: the loss of representativeness and statistical reliability of the results of the Catalan sample". Furthermore, they defend that since the agency began operating, all tests have been conducted with guarantees in the samples used.

And now what?

According to the Government, in March the OECD notified the ministry of an error in the Catalan sample, and they communicated this to the Director of the Evaluation Agency. From here, "immediately" the department opened "preliminary informative proceedings of a confidential nature" and this process to gather information "is still open". "I can state that it is an isolated case", nevertheless, the Director General of Curriculum has justified. Furthermore, she has also assured that the Government "has evidence" that it is an isolated case because this year state, international (PIRLS) and Catalan tests have also been carried out and in all of them the exemption rates "have been within the margin of the threshold set by each test".

Education has also insisted that its priority now is to "ensure that [an error like this] is not repeated". In this regard, Fernández Cuenca explained that "today" an "until now non-existent" body has been created to organize and apply each test of the system by making "protocols, action circuits and alert thresholds for each of the tests". The objective of this body, called the Commission for the Coordination of Tests and Evaluation Studies, will be that "the validation of exemptions —when a student is exempted from taking exams— is always done and for all tests". "Until now this depended on informal circuits and now it will correspond to a stable order that will have a specific protocol to manage any incident with sufficient advance notice", assured the representative of the department.

Ustec requests an audit

The reactions to the blunder with the OECD tests have not been long in coming. Junts and PP have already requested the appearance of the Minister of Education Esther Niubó for the error in the tests and have labeled the act as "incompetence". USTEC has also issued a strong statement against the Government, reproaching them for hiding information for months, as the executive became aware of the incident last March. Furthermore, the union has demanded an independent audit, the assumption of responsibilities, and the end of the Government's agreement with the OECD. "Inclusion does not consist of making vulnerable students disappear from statistics," warned the majority union in the sector, which has called for a strike on September 8, the first day of school.