Is it the AI's fault?: what the PISA report says about how it has affected the results
Spanish students are above the European and OECD average in the use of tools like ChatGPT
BarcelonaIt is one of the main concerns of families, teachers, and professionals: that students use artificial intelligence (AI) to study – or to have to do less and get good grades easily –. It is also one of the novelties of the PISA report, which has analyzed country by country what use students make of this technology and how it impacts their academic performance. However, analyzing the results, it cannot be concluded that there is a clear relationship between using AI and achieving – or not – new knowledge.
It is true that in the report, the father of the PISA tests himself, Andreas Schleicher, is clear in stating that "students who use AI chatbots for specific tasks such as summarizing texts obtain lower scores in science, on average, than those who do not." But he also admits that everything depends a lot on how this technology is used. "Just as we don't get in shape by watching sports but by doing sports, learning does not occur through the consumption of content, but as a productive cognitive struggle of the mind with new material. When technology allows or improves this cognitive struggle, students advance. When technology short-circuits the productive struggle of learning, it undermines the development of students," describes Schleicher. And, in fact, he speaks out in favor of AI: "If we do it right, AI becomes a scaffold, not a crutch. A tool for thinking, not a substitute," insists the father of PISA.
This difference in the impact on results depending on the use made of AI is also observed in the ranking of countries. For example, Japan is the country in the PISA ranking where there are more students who "never (or almost never) use AI chatbots" to do homework and, at the same time, it is the fourth country with the best academic results in the world. And Singapore, for example, which occupies second place in mathematics, reading comprehension, and science, is also the second country that has the fewest students who never use AI to do work at home.
It also happens with Macau (China), third with the best results and also one of the territories where the fewest students say they do not use AI. And it is equally true that Estonia and the United Kingdom, which are the countries in Europe with the best results, or Canada and China, which also occupy the world top 10, are the countries with the most students who do not use artificial intelligence.
Data at a European scale also shows the extent to which whether AI is good or bad depends on the type of use students make of it. In this sense, in Estonia 58% of students say they use AI at least once a week to "help them learn". In Spain, which scored 50 points lower than Estonia –the equivalent of two school years–, this proportion is almost identical: 54%. And in the United Kingdom, 31%.
Spain, above Europe and the OECD in AI usage
Beyond specific comparisons, the report points out how, in general terms, Spanish students use AI more in all areas than the European and OECD averages. For example, four out of ten adolescents in the country say they use AI more than once a week to summarize texts they have to read, while the European average is just over 30% and the OECD average does not reach 30%. Spanish students also exceed the average when it comes to using tools such as ChatGPT or Gemini in all areas: to carry out tasks such as researching new topics, writing papers, or to help them learn.
Although the authors of the report insist that there is not enough data to establish a direct relationship between the use of AI and the decline in results – in fact, they also point out that there are green shoots in countries where it is observed that when this technology is taught to be used appropriately in high school, the impact on results improves – there are cases in which a significant difference is evident. Across the OECD, in science, students who use AI every day to summarize texts or write essays score about 40 points lower than those who never use AI for these tasks. In contrast, those who use AI to help them learn every day or once a week obtain better results than those who never use it.