The alteration of the PISA sample in Catalonia far exceeds that of any country in the latest report

BarcelonaThis Friday, the Department of Education communicated that, due to an error, Catalonia's results in the PISA tests will not be valid for comparison with other territories nor with Catalan results from previous years. The reason is that when deciding how many students did not take the tests due to learning difficulties, Catalonia exceeded the limit: it discarded 23% of the students in the sample when the OECD marks that the maximum number of exempted students cannot exceed 5%.

In the explanation of the facts, the Director General of Curriculum and Professional Development, Sònia Fernández Cuenca, assured that errors like the one that occurred in Catalonia are "a phenomenon that the OECD itself has had to manage on other occasions." In this regard, in the latest PISA report of 2022, sixteen countries exceeded the threshold of exempted students set by the OECD, but none of them – except Ukraine, which was already in full war and was at 26% exempted – exceeded 11% of excluded students. In other words, the percentage of students that Catalonia has excluded from the PISA exams this year doubles the proportion of the country that exempted the most in the last tests: Denmark.

Regarding Spain, in the latest PISA report it was among the territories with the lowest exemption rates, with 4.02%, just above Italy (3.07%), France (3.72%), and Japan (2.52%).

Returning to the territories that exceeded the limit of excluded students, despite there being fifteen of them, none of them were excluded from the international report – as will happen with Catalonia – because with the exceptionality of the post-pandemic period of covid-19, the OECD decided to keep them and make annotations in their sections to warn that the results could be altered. In this case, the OECD itself warns that having a very high percentage of exempted students can lead to an increase in results.

From dyslexia to an increase in students with disabilities

When the OECD detects that a country has a higher rate of excluded students than permitted, the PISA Adjudication Groupexamines all cases and also asks the country in question to carry out a bias analysis to determine the reliability of its sample.

In the case of the PISA 2022 tests, the reasons for exceeding the percentage of excluded students are diverse. In the case of Denmark – the country with the highest percentage of excluded students – it was detected that there had been an increase in students with diagnosed dyslexia. Furthermore, these students used electronic support devices to read and the PISA tests could not be adapted for them.

In contrast, in the Netherlands (8.4% exemptions), the majority of excluded students had a physical or intellectual disability, for which no adaptation was available in the test. In Latvia (7.9%), exclusions were due to a significant number of students studying remotely at the time of the PISA exams. In Sweden (7.4%), the reasons for the excess of excluded students could not be determined. And in Norway (7.3%) and Australia (6.9%), the reason for exclusion was a combination of students with an intellectual disability and a considerable number of students who had a "limited command" of the language in which the tests were applied.

In the tests for the year 2025, as detailed by the Department of Education on Friday, in Catalonia, out of an initial sample of 2,545 students, it was decided that up to 591 would not take the exams: six for physical disability, 92 for insufficient language proficiency, and 493 for cognitive, emotional, or behavioral disability.