CHAPTER 5
International best practice for standardised testing
5.1
The inquiry sought to assess the effectiveness of NAPLAN in an
international context to ensure Australia is using educational best practice to
maintain and improve our global educational standing. As well as being
informed by the submissions it received, the committee considered findings from
a number of Organisation for Economic Co-Operation and Development (OECD) publications.
These publications were also referenced by the Department and ACARA in their
submissions to the inquiry.
5.2
The OECD report, Synergies for Better Learning: An International
Perspective on Evaluation and Assessment in Education, highlights the
importance of education in a global world:
Economic activity has become globally interconnected on an
unprecedented scale... This growing integration of economies has an impact on
strategies for national competitiveness, innovation, employment and skills. The
emergence of the "knowledge society" and the strong skill bias in
technological change have increased the value of education as a determinant of
social and economic outcomes; this raises the payoff to good performance and
amplifies the penalty for poor performance. The quality of education is
necessary to achieve economic competitiveness in a context of global economic
competition.[1]
5.3
With specific regard to assessment and evaluation the report stresses
the importance of being able to measure the benefits of achieving educational
goals for individual students, but also on the broader community:
"Well designed evaluation and assessment activities are
expected to ensure that: each student is provided with quality and relevant
education; the overall education system is contributing to the social and
economic development of the country; and each school agent is performing at
their best to deliver efficient education services. A corollary of this is that
educational goals place increasing emphasis on equity objectives, which
enlarges the scope for evaluation and assessment activities.[2]
5.4
However the OECD report raises the controversy of whether to publish the
data garnished from testing. In some countries national assessments cannot be
used to rank schools. This differs from Australia, where the media publishes
school rankings or 'performance tables' drawing on officially published data,
although this practice is not supported by the Australian government. The OECD
report states that:
[E]vidence on the effect of publishing student exam or
assessment results is mixed, with some studies showing a positive relationship
with student performance results, but others showing unintended strategic
behaviour by schools, teachers and parents.[3]
5.5
One of the key factors in whether to publish data or not is how it is
used. Australia is considered as making a high use of the results of evaluation
and assessment for development (accountability) because most of the components
of its evaluation and assessment framework are systematically linked to actions
for development. A key challenge is to find the right balance between
accountability and the development functions of evaluation and assessment:
While transparency of informed, high-quality data, and the
accountability of school agents are essential for a well-functioning evaluation
and assessment system, it is important to ensure that the existing data and
information are actually used for development and improvement. This requires
reflection on designing mechanisms to ensure that the results of evaluation and
assessment activities feed back into teaching and learning practices, school
improvement and education policy development.[4]
5.6
Both the Department and ACARA cited OECD publications to support their
argument that NAPLAN represents international best practice for standardised
testing. ACARA submitted that:
In 2010 the OECD undertook a review of Australia's evaluation
and assessment processes as part of an international study into evaluation and
assessment in education. The report on this review affirmed that NAPLAN is
technically sound and results are credible among school agents.[5]
5.7
The Department referred to the OECD report, Review of Evaluation and
Assessment Frameworks for Improving School Outcomes, which analyses the
strengths and weaknesses of different approaches, and offers policy advice on
using evaluation and assessment to improve the quality, equity and efficiency
of education.[6]
According to the Department's submission:
The report notes that Australia's National Education
Agreement (effective 25 July 2012) has reinforced the role of evaluation and
assessment as key tools to achieve quality and equity in Australian education,
in particular with the introduction of the NAPLAN and the establishment of
reporting requirements for all schools.[7]
5.8
The OECD report also noted the Government's opposition to the
publication of data to create league tables, while identifying the publication
of NAPLAN results in general as:
...a best practice example of public accountability through the
reporting of standardised student assessment results at the school level for
use by parents, government officials, the media and other stakeholders.[8]
5.9
A number of submitters disagreed with the Department of Education,
Employment and Workplace Relations and ACARA and pointed to alternative
international approaches to standardised testing. For example, the Australian
Literacy Educators' Association (ALEA) referred to the United Kingdom's
standardised 'high stakes' testing regime similar to NAPLAN, which after
analysis, had not been found to have improved student outcomes in English.[9]
ALEA also quoted research into the No Child Left Behind program in the
United States as an example. This program has as its centrepiece a requirement
for all states[10]
to develop standards that are measured through state wide testing.[11]
The research cited by ALEA contended that substantial gains in the 1990s that
were realised through educational reforms rather than testing, stalled or
declined under the No Child Left Behind program.
5.10
In comparison, many submitters[12]
cited the Finnish education system as one that does not rely on standardised
testing but has nonetheless achieved excellent results in numeracy and
literacy.
5.11
The Australian Primary Principles Association (APPA) suggested that the
success of the Finnish system could be traced to the trust it placed in teacher
professionalism, rather than in standardised testing. APPA further noted a
number of key elements, including an approach connected to inclusiveness and
creativity, a high degree of special education support for students, and
collective responsibility of teachers for developing curriculum and diagnostic
assessments instead of prescribed curriculum and standardised high stakes
testing.[13]
5.12
A number of submitters compared NAPLAN to the Programme for
International Student Assessment (PISA)[14]
in terms of its methodology and results. 2012 PISA results indicate Australia
has higher than OECD average results in a number of areas, including overall
performance. However, it also shows this is declining in the areas of
mathematics and reading, with results for science showing a neutral trend with
neither improvement nor decline. Student anxiety is around the OECD average
with a lower than OECD average effect on performance.[15]
5.13
Professor Kaye Stacey, (former Chair of the International Mathematics
Expert Group for the OECD PISA study in 2012), submitted a comparison of PISA
and NAPLAN, focussing on mathematics. Professor Stacey submitted OECD's
assessments focus on providing knowledge and skills 'that are likely to be
important for knowledge economies in the future'.[16]
However she suggested that NAPLAN assessments are focussed on more basic
skills:
NAPLAN does not provide an adequate model for the school
mathematics curriculum. It does not promote complex thinking, or reasoning, or
explaining mathematical ideas.[17]
5.14
The committee has noted a number of areas in the PISA 2012 Results in
Focus Report where Australia performs above the OECD average. For example,
Australia has achieved an above average performance in both mathematics and
equity in education opportunities.[18]
The results also indicate that on average across OECD countries, 8% of students
are top performers in reading; Australia has more than 10% of students that are
top performers.[19]
The PISA 2012 results indicate that Australia is one of several countries -
including Finland - that achieve above OECD average mean performance and have a
weak relationship between socio-economic status and student performance.[20]
5.15
Another comparison that can be made between PISA and NAPLAN is the
quality of the data it produces about schools, and how useful it is in
considering the needs of students and schools as a whole. Much of the debate
around NAPLAN has centred on the way information it collects is used as a
diagnostic tool, and the corresponding way in which it may affect student
stress levels. The quality of data produced by testing and the degree to which
it can be interrogated to produce meaningful information that will assist
parents, schools and governments to improve individual students' as well as
overall school performance is considered critical to the success of
standardised testing.
Committee View
5.16
It is the committee's view that Australia is performing well at an
international level, as demonstrated by the most recent OECD report outlining
the 2012 PISA results, and that NAPLAN does not appear to be inconsistent with international
best practice. However, OECD data highlights a number of areas where
Australia's overall performance is declining or is below OECD average, and
areas where NAPLAN could be improved.
5.17
The committee believes it is important to consider how NAPLAN could
focus on twenty-first century skills and testing that requires students to
undertake a deeper analysis in responding to questions. Data collected should
both be meaningful and able to be interrogated to produce reports that will
result in better outcomes for students and schools.
Senator Sue Lines
Chair
Navigation: Previous Page | Contents | Next Page