How was it decided which students did not take the PISA tests?

BarcelonaThe Catalan education system has experienced a real upheaval: an error in the sample of students who took the PISA tests will make Catalonia's results in the OECD tests unrepresentative and incomparable with the scores of other countries or with the results that Catalan students achieved in previous editions. In other words, the PISA tests will not serve to know if Catalan schools have improved or worsened compared to the last report in 2022, which was a disaster for Catalonia, nor to know the level of Catalan students compared to that of boys and girls from other territories. Broadly speaking, the problem is that there was an error in deciding which students should be excluded from the tests due to learning difficulties: Catalonia exempted 23% of the students who were part of the sample, when the OECD sets a maximum of 5% of participants to be exempted. But what is the process for deciding which adolescents take the PISA tests?

As explained by the Department of Education, of the 2,545 students selected for the sample, it was decided that 591 would not take the exams: six for physical disability, 92 for insufficient language proficiency, and up to 493 for cognitive, emotional, or behavioral disability.

Center Census

The process begins with the OECD – the organization that promotes PISA tests worldwide – hiring a company to select the schools that will take the tests. This company is responsible for receiving the entire list of schools eligible to take the exams, in this case, all those with students born in 2009. The company receives this list from the Ministry of Education, which in turn has received the list from the autonomous communities. This allows for the census of schools that have students from 2009 – who are between 14 and 15 years old at the time of the test –. At this stage, special education centers are already excluded because the OECD dictates it.

Sample of schools

Once the school census is available, it is the company contracted by the OECD that is responsible for making the sample selection, that is, deciding which educational centers in the country will take the tests. The selection is made taking into account the characteristics of each center – for example, its complexity – so that the sample is representative of the entire educational system. This list of 51 Catalan centers, both public and subsidized, is sent to the ministry, which then forwards it to each autonomous community.

Student selection

The next step consists of the selected centers sending to the ministry all students born in 2009 and who, therefore, are likely to undergo the PISA tests. On this list, however, each institute already marks which of these students on the list have special educational needs. This entire census is sent back to the OECD.

With this student census, another company contracted by the OECD comes into play. This is the one responsible for creating the random sample of students who, a priori, will take the exams. However, there is still one more step: the decision of which students will be exempt from taking the PISA tests is made at the time of applying the exam.

In this last phase, a national company – in this case from Spain – goes to the educational center with the list of students provided by the OECD and who, according to the sample, are to take the test. Once there, "the educational professionals who know the students", in the words of the representative of the Ministry of Education, decide with the criteria provided by the OECD whether their student will take the test or be excluded. "The educational center decides with prior criteria," they emphasize from the ministry.

Without review

This Friday, the Department of Education and the ministry acknowledged that, after the center's selection, there is no review of how or how many students have been discarded. "The company contacted by the OECD cannot say whether the selection made by the centers is correct or not," defended the representative of the Ministry of Education. In fact, the Department of Education assured that it did not know that there was an error in the PISA test sample until last March when the OECD informed it that the selection was incorrect and that, therefore, the results of Catalonia in the PISA tests will not be valid for evaluating the evolution of the level of Catalan students.