Jesús Moral: "The PISA chronology points to a very serious error by the Department of Education"

BarcelonaJesús Moral (1959, Còrdova) is a mathematics professor and has worked for almost two decades in the Department of Education. He has been head of the Center Organization service, secretary of the School Council of Catalonia, secretary of the Higher Council for the Evaluation of the Educational System, and deputy director general of Innovation and Training. In 2018, he was part of the department's team responsible for organizing the PISA tests. In a conversation with ARA, he analyzes the error that has invalidated the inclusion of Catalonia's results in the international tests and which, therefore, makes it impossible to compare the level of Catalan students with that of other territories and with previous years.

Did you imagine that a situation like this could happen?

— No, I didn't expect it. I followed the minister's appearance very carefully, but I lack data, I lack a lot of information to understand what happened. The PISA tests are very important, they cannot be underestimated, and their procedure is very regulated and has a lot of credibility. There are different supervision and control mechanisms so that strange things like what happened do not happen, there are many organizations involved and, therefore, they are very rigorous and very strict in the procedure.

How does it work exactly?

— The Organisation for Economic Co-operation and Development (OECD), which drives the PISA tests, samples and selects the schools that will participate. This selection is communicated to the National Institute for Educational Evaluation and, together with the Department of Education, work is done to verify that the sample is truly representative. In the event that a school cannot participate, there are substitute schools also chosen by the OECD with the same characteristics so as not to alter the sample. We convened the schools, explained the conceptual framework of the test so that they would know the application criteria and also the exemption criteria set by the OECD.

One of the hypotheses is a possible confusion.

— Yes, due to the coincidence in time between this international evaluation (which took place between March 31 and May 16) and the end-of-stage tests for ESO (on April 10 and 11). It is true that the OECD exemption criteria are different from local criteria, which include other aspects such as dyslexia or emotional vulnerability, among others, while PISA only considers physical disability, insufficient language proficiency, or cognitive, emotional, or behavioral disability. We were very careful not to make them coincide. In the years 2015, 2018, and 2022, the 4th year ESO tests were in February and PISA tests were in April-May.

The Ministry of Education states that it was aware since April 2025 of the OECD's doubts about the criteria.

— The timeline of events directly points to a very serious error in the management of the tests, both by the Superior Council for Evaluation and the department. We should have been very attentive to avoid these incidents and to ensure the tests do not come out biased. It used to be done by the Superior Council for Evaluation, which is now transitioning to the Agency for Evaluation and Prospecting of Education (APE), which has assumed its competence, and I don't know if this has led to the accompaniment of participating centers being neglected. A thorough study is carried out of the centers in the sample, their census, their social composition, the students' language proficiency, and their learning disorders, and then it is validated by the National Institute for Educational Evaluation. One must be very rigorous and apply the criteria correctly. It is strange, I eagerly await the conclusions of the inquiries opened by the department. A serious alert about exclusion rates is normally escalated to higher levels and a solution is sought so as not to jeopardize the validity of the test.

Were there ways to detect the error before?

— PISA has a weak point which is the lack of real-time validation before starting the test. But it is the department's responsibility to properly transmit information to the centers. The exemption criteria for the test are different from those of our evaluations, but the department must know it, take responsibility, and anticipate possible incidents. It is their obligation. They must mobilize inspectors and inform directors and teachers. In 2018 we even created a Moodle with the students so they would be familiar with the test before taking it.

What consequences do you think this error will have?

— Not having reliable data has a very serious impact on the education system because there will be no indicators to guide public policies. The latest results indicated that we were doing badly, and whether or not there was a recovery, we will no longer know. Furthermore, there is the problem of discredit and distrust. Educational centers have had a very complicated year and now this serious incident is added to it. In addition, Catalonia has lost the census tests for basic competencies, because now they are sample-based, which is why we are in a complex moment and there is no reliable information to make decisions. We must take advantage of this to open a deep debate on the evaluation model we want in Catalonia because we cannot afford further mismatches in an educational emergency situation like the one we have.