The OECD responds to the experts who found an error in PISA: "The conclusions remain valid"

BarcelonaThe infinity of data analyzed by the PISA report and the many ways of analyzing them continue to cause a stir. After researchers from EsadecPol –the center for economic policies at Esade– and Innovamat Research Lab warned in a study on Wednesday that there was an error in the PISA report which meant that the claim that wealthy students had lost more level than vulnerable ones was false, the OECD has responded. In a written response provided to the media, the OECD –the organization that conducts the PISA tests around the world– downplays the matter: it points out that the difference between the conclusion of its report and that of the Spanish researchers "reflects a methodological choice and not a methodological error." Despite admitting that they have not yet "had the opportunity to review the research in detail," they do defend that "the conclusions of the PISA volume remain valid and consistent."

According to the researchers from EsadecPol and Innovamat, the drop in level of students with a high socioeconomic status, which the OECD itself described as a "new phenomenon" and a "surprise," was due to a change in the way students are classified and not a decrease in points in languages and mathematics. In this sense, in its statement, the OECD acknowledges that "different approaches" to defining the socioeconomic situation "can produce different results and that, therefore, one must act with caution when interpreting changes in performance gaps associated with the socioeconomic context."

From the organization, they defend that the team of Spanish researchers conducted the analysis by comparing the vulnerability items that have been repeated throughout the latest editions of PISA. In contrast, the OECD researchers have used aspects that have been updated based on expert advice to decide year by year which are most relevant for defining the students' socioeconomic situation. In this way, they insist that according to "this basis" the PISA conclusions are "valid and consistent with the methodology used."

The drop is not limited to vulnerable students

While not invalidating the analysis done by the Spanish researchers, OECD sources make it clear that "a conclusion that seems to hold across different approaches is that the declines in performance are not limited to students from families with greater socioeconomic disadvantages, but are also substantial among students from more affluent families with higher educational levels".

Far from confronting the EsadecPol and Innovamat researchers, the OECD explains that their "findings" can constitute "a useful starting point to delve into this issue". And they add: "The OECD encourages researchers around the world to explore the relevant questions raised by the initial publication of the PISA results and to use this data to strengthen the empirical base that allows for improving the educational outcomes of young people all over the world".