CHAPTER 5

CHAPTER 5

International best practice for standardised testing

5.1        The inquiry sought to assess the effectiveness of NAPLAN in an international context to ensure Australia is using educational best practice to maintain and improve our global educational standing.  As well as being informed by the submissions it received, the committee considered findings from a number of Organisation for Economic Co-Operation and Development (OECD) publications.  These publications were also referenced by the Department and ACARA in their submissions to the inquiry.

5.2        The OECD report, Synergies for Better Learning: An International Perspective on Evaluation and Assessment in Education, highlights the importance of education in a global world:

Economic activity has become globally interconnected on an unprecedented scale... This growing integration of economies has an impact on strategies for national competitiveness, innovation, employment and skills. The emergence of the "knowledge society" and the strong skill bias in technological change have increased the value of education as a determinant of social and economic outcomes; this raises the payoff to good performance and amplifies the penalty for poor performance. The quality of education is necessary to achieve economic competitiveness in a context of global economic competition.[1]

5.3        With specific regard to assessment and evaluation the report stresses the importance of being able to measure the benefits of achieving educational goals for individual students, but also on the broader community:

"Well designed evaluation and assessment activities are expected to ensure that: each student is provided with quality and relevant education; the overall education system is contributing to the social and economic development of the country; and each school agent is performing at their best to deliver efficient education services. A corollary of this is that educational goals place increasing emphasis on equity objectives, which enlarges the scope for evaluation and assessment activities.[2]

5.4        However the OECD report raises the controversy of whether to publish the data garnished from testing. In some countries national assessments cannot be used to rank schools. This differs from Australia, where the media publishes school rankings or 'performance tables' drawing on officially published data, although this practice is not supported by the Australian government. The OECD report states that:

[E]vidence on the effect of publishing student exam or assessment results is mixed, with some studies showing a positive relationship with student performance results, but others showing unintended strategic behaviour by schools, teachers and parents.[3]

5.5        One of the key factors in whether to publish data or not is how it is used. Australia is considered as making a high use of the results of evaluation and assessment for development (accountability) because most of the components of its evaluation and assessment framework are systematically linked to actions for development. A key challenge is to find the right balance between accountability and the development functions of evaluation and assessment: 

While transparency of informed, high-quality data, and the accountability of school agents are essential for a well-functioning evaluation and assessment system, it is important to ensure that the existing data and information are actually used for development and improvement. This requires reflection on designing mechanisms to ensure that the results of evaluation and assessment activities feed back into teaching and learning practices, school improvement and education policy development.[4]

5.6        Both the Department and ACARA cited OECD publications to support their argument that NAPLAN represents international best practice for standardised testing. ACARA submitted that:

In 2010 the OECD undertook a review of Australia's evaluation and assessment processes as part of an international study into evaluation and assessment in education. The report on this review affirmed that NAPLAN is technically sound and results are credible among school agents.[5]

5.7        The Department referred to the OECD report, Review of Evaluation and Assessment Frameworks for Improving School Outcomes, which analyses the strengths and weaknesses of different approaches, and offers policy advice on using evaluation and assessment to improve the quality, equity and efficiency of education.[6] According to the Department's submission:

The report notes that Australia's National Education Agreement (effective 25 July 2012) has reinforced the role of evaluation and assessment as key tools to achieve quality and equity in Australian education, in particular with the introduction of the NAPLAN and the establishment of reporting requirements for all schools.[7]

5.8        The OECD report also noted the Government's opposition to the publication of data to create league tables, while identifying the publication of NAPLAN results in general as:

...a best practice example of public accountability through the reporting of standardised student assessment results at the school level for use by parents, government officials, the media and other stakeholders.[8] 

5.9        A number of submitters disagreed with the Department of Education, Employment and Workplace Relations and ACARA and pointed to alternative international approaches to standardised testing. For example, the Australian Literacy Educators' Association (ALEA) referred to the United Kingdom's standardised 'high stakes' testing regime similar to NAPLAN, which after analysis, had not been found to have improved student outcomes in English.[9]  ALEA also quoted research into the No Child Left Behind program in the United States as an example. This program has as its centrepiece a requirement for all states[10] to develop standards that are measured through state wide testing.[11] The research cited by ALEA contended that substantial gains in the 1990s that were realised through educational reforms rather than testing, stalled or declined under the No Child Left Behind program.

5.10      In comparison, many submitters[12] cited the Finnish education system as one that does not rely on standardised testing but has nonetheless achieved excellent results in numeracy and literacy.

5.11      The Australian Primary Principles Association (APPA) suggested that the success of the Finnish system could be traced to the trust it placed in teacher professionalism, rather than in standardised testing. APPA further noted a number of key elements, including an approach connected to inclusiveness and creativity, a high degree of special education support for students, and collective responsibility of teachers for developing curriculum and diagnostic assessments instead of prescribed curriculum and standardised high stakes testing.[13]

5.12      A number of submitters compared NAPLAN to the Programme for International Student Assessment (PISA)[14] in terms of its methodology and results. 2012 PISA results indicate Australia has higher than OECD average results in a number of areas, including overall performance. However, it also shows this is declining in the areas of mathematics and reading, with results for science showing a neutral trend with neither improvement nor decline. Student anxiety is around the OECD average with a lower than OECD average effect on performance.[15]

5.13      Professor Kaye Stacey, (former Chair of the International Mathematics Expert Group for the OECD PISA study in 2012), submitted a comparison of PISA and NAPLAN, focussing on mathematics. Professor Stacey submitted OECD's assessments focus on providing knowledge and skills 'that are likely to be important for knowledge economies in the future'.[16] However she suggested that NAPLAN assessments are focussed on more basic skills:

NAPLAN does not provide an adequate model for the school mathematics curriculum. It does not promote complex thinking, or reasoning, or explaining mathematical ideas.[17]

5.14      The committee has noted a number of areas in the PISA 2012 Results in Focus Report where Australia performs above the OECD average. For example, Australia has achieved an above average performance in both mathematics and equity in education opportunities.[18] The results also indicate that on average across OECD countries, 8% of students are top performers in reading; Australia has more than 10% of students that are top performers.[19] The PISA 2012 results indicate that Australia is one of several countries - including Finland - that achieve above OECD average mean performance and have a weak relationship between socio-economic status and student performance.[20]

5.15      Another comparison that can be made between PISA and NAPLAN is the quality of the data it produces about schools, and how useful it is in considering the needs of students and schools as a whole. Much of the debate around NAPLAN has centred on the way information it collects is used as a diagnostic tool, and the corresponding way in which it may affect student stress levels. The quality of data produced by testing and the degree to which it can be interrogated to produce meaningful information that will assist parents, schools and governments to improve individual students' as well as overall school performance is considered critical to the success of standardised testing.

Committee View

5.16      It is the committee's view that Australia is performing well at an international level, as demonstrated by the most recent OECD report outlining the 2012 PISA results, and that NAPLAN does not appear to be inconsistent with international best practice. However, OECD data highlights a number of areas where Australia's overall performance is declining or is below OECD average, and areas where NAPLAN could be improved.

5.17      The committee believes it is important to consider how NAPLAN could focus on twenty-first century skills and testing that requires students to undertake a deeper analysis in responding to questions. Data collected should both be meaningful and able to be interrogated to produce reports that will result in better outcomes for students and schools.

Senator Sue Lines
Chair

Navigation: Previous Page | Contents | Next Page