What the world can learn from the latest PISA test results

FOOTBALL fans must wait four years between World Cups. Education nerds get their fill of global competition every three. The sixth Programme for International Student Assessment (PISA), a test of the science, maths and reading skills of 15-year-olds from across the world, was published by the OECD club of mainly rich countries on December 6th. Its results have telling lessons for policymakers worldwide.

Some 540,000 pupils in 72 countries or regions—each of whom had finished at least six years of school—sat similar tests last year. The OECD then crunched the results into a standardised scale (see chart 1). In the OECD the average result for each subject is about 490 points. Scoring 30 points above that is roughly akin to completing an extra year of schooling.

Singapore, the consistently high-achiever in PISA, does even better: it is now the top-performing country in each subject area. The average pupil’s maths score of 564 suggests Singaporean teens are roughly three years ahead of their American peers, with a tally of 470.

Other East Asian countries also score highly across most domains, as they have done since PISA was launched 15 years ago. Japan and South Korea have above-average results in science and maths, as do cities such as Hong Kong and Macau, both autonomous territories of China, and Taipei, the capital of Taiwan.

Elsewhere, Canada and Finland have reading scores as high as Hong Kong’s. Then there is Estonia: its science results are indistinguishable from Japan’s and its maths scores are akin to South Korea’s. It is now equal with Finland as the top performer in Europe. In turn Finland, which topped the first PISA, is still an above-average performer, but its scores have fallen since at least 2006.

Opponents of PISA argue that trying to make sense of all this is like trying to hear oneself over the noise of an obstreperous classroom. They note that education is about more than doing well in tests. And besides, some critics add, there is little useful to learn from the results, since it is parents alone that encourage swots. John Jerrim of University College London suggests that the only way some countries could catch up with the East Asian powerhouses is through more “tiger mothers” and “widespread cultural change”.

PISA has flaws. It is one of many standardised tests, and tests are not all there is to learning. But it matters. It is the most influential research report in education for good reason. It offers informed guidance on what policymakers should do to fix their school systems. Just as importantly, it tells them what not to do.

It points out that among poorer countries the amount of public spending per pupil is associated with higher test scores. But in richer states that spend more than about $50,000 per pupil in total between 6 and 15 this link falls away (see chart 2). The pupils of Poland and Denmark have, in effect, the same average results in the science tests even though Denmark spends about 50% more per pupil.

Another potential waste of money, if only from the perspective of PISA results, may be sending children to private school. Across the OECD pupils in public schools score lower in science than students in private schools do. But this is not the case once you account for the economic and social background of pupils.

And while poverty is strongly associated with low scores, it is not destiny. In the OECD poor pupils are nearly three times more likely than their rich peers to have less than the basic level of proficiency in science. Those pupils with foreign-born parents tend to do even worse. Nevertheless, 29% of poor pupils score among the top quarter of children across the OECD. In Singapore, Japan and Estonia nearly half of the poorest pupils do.

Money isn’t everything

That hints at another finding: achievement and greater equality are not mutually exclusive. In Canada, Denmark, Estonia, Hong Kong and Macau pupils have high average scores, with only a weak link between results and children’s backgrounds.

One reason for Estonia’s gain is demographic loss. Over the past 20 years the population of young people has declined faster than the number of teachers. There is now one teacher for every 12 pupils, down from closer to 20 two decades ago. Although in general reducing class sizes is not the most cost-effective response, Estonian pupils have benefited from the demographic shift, which has made it easier to give pupils, especially laggards, extra help.

But Estonia has also taken a deliberately inclusive approach, argues Mart Laidmets, a senior official at its ministry of education. It tries to avoid at all costs having pupils repeat years of school. Holding pupils back can help. But too often it is used as an excuse not to teach difficult kids. It may also reflect bias or discrimination. In countries such as Russia, Slovakia and the Czech Republic, poor boys are especially prone to being kept back a year, despite decent academic achievements.

Estonia, like Finland and Canada, also tries to keep selection by ability to a minimum. It delays “tracking” children into academic or vocational routes until they are 15 or 16 years old. Mr Laidmets argues that it helps pupils find jobs later in life, since better maths and literacy make it easier for them to adapt to changes in the labour market and to earn new skills.

By contrast, where pupils are diverted from an academic track at an early age, whether towards a vocational school or a less rigorous class in the same school, the gap between rich and poor children tends to be wider. In the Netherlands pupils at vocational schools have results equivalent to about three years less of schooling than their peers at general schools. “The more academically selective you are the more socially selective you become”, says Andreas Schleicher, the head of education at the OECD.

All of which suggests what countries should not do. But are there any sure-fire tips from the best performers? Or is their success just down to pushy parents and tuition after school?

Culture matters but so, too, does policy, says Lucy Crehan, author of “Cleverlands”, a new book on PISA-besting countries. She points out that most of these states delay formal schooling until children are six or seven. Instead they use early-years education to prepare children for school through play-based learning and by focusing on social skills. Then they keep pupils in academic courses until the age of 16. Even Singapore, which does divert some pupils to a vocational track at the age of 13, ensures that pupils in those schools keep up high standards in reading and maths.

Top performers also focus their time and effort on what goes on in the classroom, rather than the structure of the school system. For while test scores and pupils’ economic background are linked across the OECD, so too are specific things that the best schools and teachers do (see chart 3).

The top performers treat teachers as professionals and teachers act that way as well. They have time to prepare lessons and learn from their peers. They tend to direct classroom instruction rather than be led by their pupils. Their advancement is determined by results, not by teachers’ unions. There are high expectations of nearly every student and high standards, too.

A keener Argentina

The teenagers who took the PISA tests in 2015 were influenced by many years’ worth of policies. And focusing on the consistently high performers means that lessons from those that have made recent improvements are neglected.

The city of Buenos Aires had the largest jump in overall scores from three years ago. On average its pupils scored 475 in science (up 51 points), 475 in reading (46 points) and 456 points (38 points) in maths.

For Esteban Bullrich, the minister in charge of education from 2010 to 2015, the initial aim was to make sure that pupils were being taught. Teachers were spending 12-15 days per year on strike, or about 7% of the time they should be in class, according to his calculations. To try to reduce those absences he first made his mobile-phone number public and began fielding calls directly from angry teachers. He extended the school day.

Then he offered teachers something of a deal: higher salaries in exchange for taking their job more seriously. The grip of unions in deciding on promotions was loosened. And he made teacher training more rigorous and practical.

Another impressive mover, albeit more of a tortoise than a hare, is Portugal. Since 2006 it has steadily improved its scores across each subject by about a year of schooling, overtaking the United States as it went from a middling to an above-average performer.

There are three reasons for Portugal’s steady progress, says Nuno Crato, the country’s former education minister. First, it began to care about results, introducing new standardised tests. Second, a new curriculum with higher standards was introduced from 2011. Third, it has reduced the amount of streaming by ability, keeping its use “temporary and partial”. Struggling pupils may get extra tuition but teachers will try to keep them in the same classes as their peers.

For Portugal to become an educational powerhouse, argues Mr Crato, it also needs “better-prepared teachers”. This is hard when some teaching unions oppose their members having to pass exams before they are allowed into classrooms. What better education does not necessarily need is bigger budgets, he says. Portugal’s improvements have come despite severe cuts to public spending. “Money matters but it is not decisive,” adds Mr Crato.

Progress can also be spotted even among countries whose overall scores have remained flat. The economic background of the average American pupil matters much less to their overall test scores than in earlier editions of PISA. Mr Schleicher puts this down to reforms such as President George W. Bush’s No Child Left Behind Act, which made local governments more accountable for the results of poorer pupils.

The PISA results are not all happy tales of plucky reform. Australia is one of several countries whose results have dipped. Its average score in maths has fallen from 524 to 494 since 2003, equivalent to a year of schooling. Australia is one of the few rich countries where pupils do not have to take maths in their leaving exams. (Malcolm Turnbull, the prime minister, is trying to make states change this.) It is also a result of a declining quality of teaching, suggests John Hattie of the University of Melbourne. Successful applicants to teacher-training courses have lower results in their school exams than in the 1980s and 1990s.

Nevertheless, while some countries rise and others fall, many are just like England. Its results have barely budged since 2006. (Scotland’s have plummeted.) The average result for OECD countries has similarly hardly changed since the tests began. This may reflect well on the test-setters: it would be worrying if the results swung wildly from edition to edition.

Yet it still reflects poorly on many policymakers. Mr Bullrich says PISA is like an X-ray of a country’s education policy. It is not a full picture of your health but it can help you spot where things are sickly. Sadly, too many countries are dodging essential therapy.