China Power

What Shanghai’s PISAs Really Say

Recent Features

China Power

What Shanghai’s PISAs Really Say

Were Shanghai’s PISA scores really a Sputnik moment? Or did they just take the tests more seriously?

Last week, the Organisation for Economic Co-operation and Development (OECD) released its 2009 Programme for International Student Assessment (PISA) results, with Shanghai 15-year olds outperforming their peers from Singapore, South Korea, and Finland by a considerable margin in math, science, and reading ability. The world’s reaction was a bit over-the-top, with one American in the New York Times comparing Shanghai’s PISA results to the Soviet Union’s launch of Sputnik.

For many Chinese and Westerners in China, there were two, simultaneous reactions. First, with generous government funding, some of China’s best universities, a dominant middle class, and a progressive and cosmopolitan culture, Shanghai was lauded as having China’s best education system, but people also noted it couldn’t be considered representative of China as a whole. (Chinese like to say that Shanghai is closer to Europe than it is to Beijing.) The second reaction was: How did Shanghai cheat? 

Shanghai could have selected its 5000 best students to take the test, or it could have gotten early access to the test questions and prepped its students. The local teachers who graded PISA could have been generous, or just reported inflated scores. There could have been translation issues as well. Yet the New York Times reporter Sam Dillon scoured the details for evidence of cheating, and this was the best he could find: ‘Shanghai students apparently were told the test was important for China’s image and thus were more motivated to do well.’

Okay, so Shanghai took the test seriously—that’s it?   

But what if I were to say that Shanghai taking the test seriously could in fact explain why it did so well? In his book Outliers, Malcolm Gladwell points out that there’s a direct correlation between the ability to take tasks seriously and high test scores:      

‘When students sit down to take the Trends in International Mathematics and Science Study (TIMSS) exam, they also have to fill out a questionnaire.  It asks them all kinds of things, such as what their parents’ level of education is, and what their views about math are, and what their friends are like. It’s not a trivial exercise. It’s about 120 questions long. In fact, it is so tedious and demanding that many students leave as many as ten or twenty questions blank.’

Now, here’s the interesting part. As it turns out, the average number of items answered on that questionnaire varies from country to country. It’s possible, in fact, to rank all the participating countries according to how many items their students answer on the questionnaire. Now, what do you think happens if you compare the questionnaire rankings with the math rankings on the TIMSS? They are exactly the same. In other words, countries whose students are willing to concentrate and sit still long enough and focus on answering every single question in an endless questionnaire are the same countries whose students do the best job of solving maths problems.   

As a teacher, I think 90 percent of teaching is getting students to focus, listen, and concentrate in class.  And if you think that’s easy, it’s obvious you’ve never dealt with 15 year-olds, who no matter their nationality or culture must deal with the rampaging confusion of adolescence: the onslaught of sexual awareness and a fast-growing body, the early search for identity and meaning, and the demands of parents and teachers.  So, considering all this, it’s quite a bureaucratic feat to get the world’s 15 year-olds to even sit down for a test administered by an organization they’ve never heard of, let alone to take PISA seriously enough to want to do well on it.     

In my school, we had a couple of students who by mid-term were failing, but once we threatened to fail them (in China, no one fails) went overnight from the bottom of the class to near the top because they suddenly focused in class and did their homework.   

And I believe that those 5000 Shanghai students were even more motivated than my two students because Chinese take international competitions and their image abroad as life-or-death matters. Chinese can simultaneously be xenophobic, and desperate for international recognition and praise. Remember the 2008 Beijing Games, and that scary, manic determination to win the most gold medals? It seems as though China constantly needs these international awards—Olympic gold medals and PISA high scores—as much as it needs 8 percent GDP growth and new skyscrapers to validate itself in its own eyes.   

Is it possible that China’s bureaucratic obsession with international awards and praise is to divert people’s attention from real-life problems—China’s unsustainable asset bubbles, endemic corruption and crime, the unemployment rate for university graduates—that show that China is in fact a house of cards just waiting to come tumbling down?