The OECD responds to the experts who found an error in PISA: "The conclusions remain valid"

BarcelonaThe infinite amount of data analyzed by the PISA report and the many ways to analyze it continue to cause a stir. After researchers from EsadecPol –Esade's center for economic policies– and Innovamat Research Lab warned on Wednesday in a study that there was an error in the PISA report which meant that the assertion that wealthy students had lost more level than vulnerable ones was false, the OECD has responded. In a written response sent to the media, the OECD –the organization that organizes PISA tests around the world– downplays the matter: it points out that the difference between the conclusion of its report and that of the Spanish researchers "reflects a methodological choice and not a methodological error". Although admitting that they have not yet "had the opportunity to review the research in detail", they do maintain that "the conclusions of the PISA volume remain valid and consistent".

According to the EsadecPol and Innovamat researchers, the drop in level of students with a high socioeconomic status, which the OECD itself described as a "new phenomenon" and a "surprise", was due to a change in the way students are classified and not to a drop in points in language and mathematics. In this sense, the OECD in its statement acknowledges that "different approaches" to define the socioeconomic situation "can produce different results and that, therefore, one must act with caution when interpreting changes in performance gaps associated with the socioeconomic context".

The organization maintains that the team of Spanish researchers conducted the analysis by comparing the vulnerability items that have been repeated throughout the latest editions of PISA and, on the other hand, the OECD researchers have used aspects that have been updated based on expert advice to decide year by year which are more relevant to define the socioeconomic situation of students. In this way, they insist that according to "this basis" the PISA conclusions are "valid and consistent with the methodology used".

The drop is not limited to vulnerable students

While not invalidating the analysis carried out by the Spanish researchers, OECD sources make it clear that "a conclusion that seems to hold across different approaches is that performance declines are not limited to students from more socioeconomically disadvantaged families, but are also substantial among students from more affluent families with higher levels of education".

Far from confronting the EsadecPol and Innovamat researchers, the OECD explains that their "findings" may constitute "a useful starting point for delving into this issue". And they add: "The OECD encourages researchers around the world to explore the relevant questions raised by the initial publication of the PISA results and to use these data to strengthen the empirical basis that will allow for the improvement of educational outcomes for young people worldwide".