According to Kevin Drum, the countries whose students are most overachieving on the 2011 TIMSS relative to the 2012 PISA are Israel, Russia, the U.S., Kazakhstan, Hungary, and Korea. Those most overachieving on the 2012 PISA are Norway, New Zealand, Slovenia, Australia, Finland, and Turkey.
The main difference between those sets of countries seems to be about cognitive inequality. The TIMSS-overachieving countries have large numbers of smart people, especially in the technical fields. The Russian language is well known to be overrepresented on the Internet relative to the world’s Russian-speaking population. Russian and Israeli scientists are well-known across the world. Meanwhile, the PISA-overachieving countries listed above are much less famous for their smart people, especially scientists and technologists. Yet, it is clear the Russian and Israeli people are not as good at creating First World countries as, say, the Norwegians, Australians, and New Zealanders.
So scoring well on PISA is better correlated with a country’s ability to build a good society for its citizens, while scoring well on TIMSS is better correlated with a country’s will to power, especially in military, in technology, and on the Internet. America, Russia, Israel, and Korea are well known for their powerful militaries. Of the PISA-overachieving countries, only Turkey (Turkiye Delenda Est) and, to a lesser extent, Finland, are known for their militaries, and they haven’t fought a major war since the 1940s. Meanwhile, Korea fought in Vietnam and remains technically at war, Israel’s history of warfare does not need to be summarized here, and Russia won wars in Chechna, Ossetia, and Donbass in the past fifteen years alone. A country which is always under the threat of war will have an education system built for it. Given the TIMSS is meant to measure school curriculum, which can be altered by the state to a far greater extent than a nation’s IQ, an improved education system, especially in the technical fields, those most vital to the military, will be most manifest in improved scores on the TIMSS tests. TIMSS does stand for Trends in International and Science Study, after all.
I also suspect that much of the difference lies in sampling: where the tests are taken. The 2012 PISA, for example, is certainly more representative of Kazakhstan than the 2011 TIMSS, where a higher-scoring population, probably Russian (I haven’t looked), is almost certainly overrepresented. The 2007 TIMSS was even more unrepresentative of Kazakhstan (4th grade) and Hungary (8th grade, but not 4th grade), placing their average mathematics scores between those of Japan and Russia. Nobody who seriously understands the nature of those countries can believe this is representative. The 2012 PISA is also almost certainly more representative of Russia’s and Israel’s IQ- the PISA-based estimate of Russia’s IQ (in the mid-90s) seems about right, as well as the PISA-based estimate of Israel’s IQ, especially given the society the Israeli leadership has created over the past six decades (more on that in a later post). Most likely, the residents of the big cities-Moscow, St. Petersburg, Tel Aviv- tend to be more overrepresented in at least some TIMSS-taking countries, with low-average-IQ ethnic minorities in both countries (Arabs in the case of Israel, native Siberians and Caucasians in the case of Russia) being underrepresented (though Arabs in Israel were fairly represented in the 2007 TIMSS). Indeed, the 2007 TIMSS was much more consistent with the 2012 PISA than the 2011 one was for Israel’s ranking, suggesting sampling in the 2011 TIMSS was far less fair. So while both tests should be used, TIMSS’s sampling should be usually considered as more suspect, while PISA’s questions should be considered less indicative of a country’s population’s technical competence.