There has been considerable public discussion about Australia’s relatively poor performance in recently released international student achievement tests, with one newspaper describing the results as
‘Australia’s disaster in education’. These results have also called into question the Government’s aim to be in the
top five countries on reading, mathematics and science by 2025. But just how bad are these results?
The
results released on Tuesday 11 December by the Australian Council for Educational Research (ACER) are the 2011 outcomes from the International Association for the Evaluation of Educational Achievement’s
Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS).
TIMSS has been conducted at Year 4 and Year 8 on a four-year cycle since 1995. Australia has participated in all five cycles. In 2011, of the 52 countries that participated in the Year 4 assessment, Australia was ranked 18th in mathematics and 25th in science. Year 8 students performed better, with Australia ranking 12th in both mathematics and science, of the 45 countries participating. The Australian average score for each of these groups has not changed significantly over the five cycles except for Year 4 mathematics where the 2011 result is significantly higher than for 1995 (when Australia was ranked 11th compared to 18th in 2011).
How valid these sorts of comparisons are is questionable given the increasing number of countries participating in these studies. For example, the number of countries participating in the Year 4 assessments has doubled since 1995 – 26 countries participated in 1995 compared to 52 countries in 2011. On the other hand, while Australia’s scores in mathematics and science have generally stagnated, those from a number of other countries have improved substantially over this period. Australia’s performance in mathematics and science has been of ongoing concern to governments, with the
2012-13 Budget including further measures in this area.
PIRLS assesses students’ reading achievement at Year 4 and has been run on a five-year cycle since 2001. Australia participated for the first time in 2011 and was ranked 27th out of 48 countries. Given that Australia has not previously participated in PIRLS, it is not possible to assess whether this result represents a decline in literacy standards at Year 4.
So does this mean Australia’s aspiration to be a ‘top five’ country by 2025 is not achievable? Not necessarily. The usual benchmark used to assess Australia’s relative educational performance is the Organisation for Economic Co-operation and Development’s (OECD)
Programme for International Student Assessment (PISA). PISA is conducted every three years, and assesses the performance of 15-year-olds in reading, mathematics and science. In the most recent
PISA results (2009), Australia was ranked 9th in reading, 15th in mathematics and 10th in science.
In comparing the results from these various tests it is important to note that they are not testing the same things. PISA assesses a student’s ability to apply their knowledge and skills to real-life problems and situations, whereas TIMSS and PIRLS have a curriculum focus – that is, what students have learnt, rather than how they apply their knowledge. This difference in emphasis is potentially significant in how school systems are rated, with problem-solving skills, for instance, better rewarded in the PISA testing than in TIMSS and PIRLS.
In speculating how Australia’s 2011 TIMSS and PIRLS results might translate into its PISA rankings in future years, it is relevant to consider how previous cohorts have fared. Due to the different timing of the various assessments, it can be possible to track the same cohort of students in each country through TIMSS Year 4 and 8 and then PISA, and from PIRLS to PISA, given that the Year 4 assessment average age is 10, for Year 8 is 14 and PISA is 15. Different countries have participated in each of the studies over the years, however a number have been involved in all surveys and so their relative performances can be compared.
The table below compares the rankings of countries in both mathematics and science for TIMSS in Year 4 in 1995 and Year 8 in 1999, with the PISA results for 2000. While the top two countries in both subject areas in PISA (Japan and Korea) also performed well in all the TIMSS assessments, the pattern for the other high performers in PISA is much less clear. New Zealand was towards the bottom of the rankings for all four TIMSS assessments but was ranked third in PISA mathematics and sixth in PISA science.
| Country | Mathematics | Science |
| | TIMMS 1995
Yr 4 | TIMMS 1999
Yr 8 | PISA 2000 | TIMMS 1995
Yr 4 | TIMMS 1999
Yr 8 | PISA 2000 |
| Japan | 3 | 5 | 1 | 2 | 4 | 2 |
| Korea | 2 | 2 | 2 | 1 | 5 | 1 |
| New Zealand | 20 | 21 | 3 | 16 | 19 | 6 |
| Finland* | | 14 | 4 | | 10 | 3 |
| Australia | 11 | 13 | 5 | 5 | 7 | 7 |
| Canada | 13 | 10 | 6 | 9 | 14 | 5 |
| United Kingdom/England | 17 | 20 | 8 | 8 | 9 | 4 |
| Czech Republic | 6 | 15 | 18 | 7 | 8 | 13 |
| United States | 12 | 19 | 19 | 3 | 18 | 14 |
*Finland did not participate in 1995
A similar comparison for
PIRLS Year 4 assessments in 2001 compared to
PISA in 2006 shows the same pattern, with the top three countries from PIRLS (Sweden, the Netherlands and England) only rating 10th, 11th and 17th in the PISA reading assessment, while New Zealand again performed poorly in the PIRLS tests, but was ranked fifth in PISA. Bulgaria, which ranked fourth in the PIRLS assessment (and is one of the countries ranked above Australia in the 2011 results), was 42nd in PISA, scoring well below the OECD average.
In summary, Australia’s relatively ‘poor’ showing in the latest rankings may not be that significant in its quest to be among the top five countries in PISA testing. However there are other challenges in achieving this goal which have been addressed in an earlier
FlagPost.