The conduct of the postal survey
Introduction
3.1 This chapter canvasses a number of issues raised during this inquiry
including:
-
Any precedents for surveys of this kind;
-
Accessibility and participation in the survey;
-
Scrutiny of the survey; and
-
Collection and destruction of the forms.
Precedents for surveys of this kind
3.2
At the public hearing on 17 August 2017, Mr Jonathan Palmer, Deputy
Australian Statistician at the Australian Bureau of Statistics (ABS) described
the scope of the Australian Marriage Law Postal Survey (postal survey):
A single question will be asked: should the law be changed to
allow same-sex couples to marry? We will ask all eligible Australians—and they
are the people who, as at the end of 24 August, would be entitled to vote as
per the electoral roll—to provide a yes or no answer on a strictly voluntary
basis. The point I would like to emphasise here about the process is that it's
been designed to provide both secrecy and integrity. The vast bulk of
Australians who are eligible for this survey will receive and return their
survey form by post. They'll be sent a letter with instructions, a survey form
and a reply-paid envelope. The barcode on the survey form will be used for
marking purposes only. We're calling it a single-use, anonymous marking code.
No person who sees or has any access to the completed forms will know both the
name of the eligible Australian and the related single-use code. The survey
form will not have any name or address on it. Only one response can be made by
each eligible Australian. When the form is scanned, the barcode will be
recorded in a separate file for marking purposes and the survey response will
be recorded in another file. There will never be an electronic file containing
both the survey response and the bar code. The form's encaptured images will be
destroyed within 60 days of our publishing the statistics on 15 November.[1]
History of plebiscites and postal
surveys in Australia
3.3
Plebiscites and postal surveys have rarely been used at a national level
in Australia. In a 2011 research paper, the Parliamentary Library stated that
there have been three national plebiscites:
-
1916: military service conscription (defeated)
-
1917: reinforcement of the Australian Imperial Force overseas
(defeated)
-
1977: choice of Australia's national song ('Advance Australia
Fair' preferred.)[2]
3.4
Postal surveys are almost unprecedented, with only one occurring
nationally. In an answer to a question on notice, the Department of Finance
described the 1997 Constitutional Convention Election as 'a national voluntary postal process to elect
delegates to the Constitutional Convention to determine whether Australia
should become a republic'.[3]
3.5
However submitters gave evidence that the postal survey was
fundamentally different from other endeavours undertaken by the ABS. The
committee was told in a submission by an ABS staff member, in their private
capacity, that the directives to the ABS were so prescriptive as to 'violate
the guidelines and principles of official statistics'.[4]
3.6
The committee were told that the ABS has collected data on social issues
for over forty years:
The ABS has been conducting large social surveys since the
1970s, and it's familiar with undertaking voluntary statistical surveys that
seek views and opinions of Australians about issues such as their
self-perceived wellbeing, social experiences and society in general. I thought
I would share just two examples for you of instances where we've sought
opinions...So, from a personal safety survey, we asked: how safe or unsafe do you
feel walking alone in your local area after dark? And we published statistics
on that question. From a survey on disability, ageing and carers, we asked: do
you feel you need more support or an improvement in your situation to aid your
role as a carer? And the answer to that was a yes or no question.[5]
Timetable of the survey mail-out
3.7
A timeline of key events during the postal survey is outlined below in
Table 3.1. The committee understands that the ABS and other government
departments and agencies have met each of these milestone dates.
Table
3.1—Timeline of key events during the postal survey[6]
| 24 August 2017 |
Commonwealth Electoral Roll
closed. |
| 12 September 2017 |
Mailing of forms and
collection process began. |
| 13 September 2017 |
Legislation providing
additional safeguards during the marriage law survey passed the Parliament.[7] |
| 25 September 2017 |
All survey packages should be
delivered by this date. |
| 20 October 2017 |
Final day to request or pick
up replacement forms. |
| 27 October 2017 |
Final day to post forms to
ensure that they are included in the final survey results. |
| 7 November 2017 |
Postal survey closed. |
| 15 November 2017 |
Postal Survey results
announced and published. |
The Australian Bureau of
Statistics' capability to deliver
3.8
When the ABS were announced as the lead agency to conduct the postal
survey, many questions were raised about the ability of the ABS to manage this
process in light of some of the difficulties associated with the 2016 Census.[8]
Mr Palmer acknowledged these concerns:
Another issue I would like to touch on is concerns that have
been expressed about our [ABS's] capacity to deliver on undertaking this
survey. In response to that, we say, 'Yes, we can.' We have been given an
adequate appropriation—$122 million—through an advance to the Finance Minister
determination. The 2016 census online form outage did hurt the public
reputation of the ABS. We acknowledge that. But we're seeing some rebuilding of
that reputation following the release of a quality census dataset. I can assure
you that we've reflected and learnt from the lessons of the 2016 census and
we're applying those experiences and lessons to the conduct of this exercise.
We've established a team of more than 40 skilled professionals, including staff
seconded from the Australian Electoral Commission and other agencies. We have
systems, processes, technical expertise, and we will partner with high-calibre
suppliers and vendors. Undertakings as large and as complex as the Australian
Marriage Law Postal Survey entail risks. We will manage these risks and issues
well. And I would like to place on record my appreciation for the work done by
the task force so far. We've done an enormous amount.[9]
3.9
This came at a cost to the ABS's other operations. The committee was
informed by the ABS that it diverted staff from other projects, including the
2021 Census in order to be able to deliver the postal survey.[10]
3.10
Earlier in the report, the committee discussed the source of the funding
for the postal survey—an Advance to the Finance Minister of $122 million. In
announcing the result of the postal survey, the Australian Statistician, Mr
David Kalisch noted that the postal survey was delivered under budget:
...while costs are still being tallied, the ABS is confident
the final cost for the survey will be under $100 million, many million dollars
less than the available budget.[11]
3.11
Throughout the inquiry, the committee has held government departments
and agencies, the ABS in particular, to account to ensure that the postal
survey was completed as intended. The committee has focused on a number of key
areas which are described in more detail through the remainder of this chapter.
Accessibility and participation in the survey
3.12
The postal survey had a national participation rate of 79.5 per cent.
Breaking down the results by state and territory, age and sex returned a
participation rate range between 70 and 80 per cent. Notably, participation
rates for males aged 20–24, 25–29 and 30–34 years all fell below 70 per cent.[12]
Challenges in remote areas
3.13
On 17 August 2017, Mr Palmer acknowledged that many people in remote
parts of the NT did not have street addresses and that the postal survey would
be a different process to the census in which ABS employees attend people's
homes to collect information on census night in remote localities.[13]
3.14
Mr Palmer reassured the committee that the ABS was finalising an
inclusion strategy to ensure a high participation rate in the postal survey:
I'm not yet able to tell you what our inclusion strategy will
be, but we're looking at all the options for reaching those people and giving
them an opportunity.[14]
3.15
At its committee hearing on 7 September 2017, Mr Kalisch noted that
the ABS had finalised and published a 'comprehensive list of inclusion
strategies' on its website, and acknowledged that:
The ABS recognises that Aboriginal and Torres Strait Islander
communities in remote areas may experience particular challenges in
participating in the Australian Marriage Law Postal Survey.[15]
3.16
The strategy for remote and rural locations included early despatch of
survey materials for remote locations,[16]
'the use of pick-up locations, Department of Human Services' agents, access
points and remote service centres'.[17]
Mr Palmer elaborated on the remote strategy at the 15 September 2017 public
hearing:
If remote Aboriginal and Torres Strait Islander people don't
receive a form, there are still a number of options available to them over the
four weeks from 25 September to 20 October. We will be providing the
opportunity for forms to be collected in 27 different regional locations—places
like Karratha, Mount Isa, Coober Pedy, to name some—and 200 other remote
locations. These locations will be advertised on the ABS website and
communicated directly to remote communities and promoted through local radio
and other media. In addition, people with access to phone or the internet can
respond using our paperless response options. Material will be translated into
a number of different Aboriginal and Torres Strait Islander languages for radio
advertising and to support the field work.
To reach those communities we are also working with the
Department of Human Services to provide additional opportunities. We are going
to leverage over 600 DHS agents' access points and remote service centres across
Australia. Most of these locations can provide self-service computer access or
telephones to freely access the survey information line on the ABS website and,
indeed, those online response options of the online form and the telephony
response. These facilities allow a person to request new materials from the ABS
and to participate in the survey online, if that's their requirement, from 25
September.[18]
3.17
Notwithstanding the strong participation rates nationally and the ABS
inclusion strategies, the overall participation rate in the Northern Territory
(NT) was 58.4 per cent—significantly lower than the national participation rate
of 79.5 per cent. When the NT is broken down into its two federal electorates,
the statistics become even starker. The more remote electorate of Lingiari had
a participation rate of 50.1 per cent. That is, nearly half of the people in
the Lingiari electorate did not respond.[19]
When broken down by age group, less than 40 per cent of eligible 20–24 year
olds responded.[20]
The Lingiari electorate is classified as rural with the majority of the
enrolment being outside the major provincial cities[21]
with 41.7 per cent identifying as Aboriginal and Torres Strait Islander and
24.5 per cent of households speaking a non-English language (with the top five
responses being Aboriginal languages).[22]
3.18
The low participation rate for the electorate of Lingiari in the postal
survey is consistent with the turnout history of the electorate at the 2016
federal election where Lingiari also recorded the lowest enrolled voter turnout
of all federal elections.[23]
3.19
The difficulty of delivering the postal survey to eligible remote
citizens was raised in correspondence tabled at the committee's public hearing
on 17 August 2017 from the NT Chief Minister, Hon Michael Gunner to the Prime
Minister, Hon Malcolm Turnbull MP. In this correspondence, the Chief Minister
raised his concerns about 'Territorians, particularly remote Indigenous
Territorians, being disenfranchised from voting in the postal plebiscite'.[24]
Mr Gunner elaborated:
Only 82% of Territorians are presently enrolled meaning that
more than 25 000 eligible voters are not enrolled. There is also a high
level of mobility amongst Indigenous electors.
Significantly, for a very large number of our remote
communities there are poor or non-existent postal services. For many
communities, the nearest post office is in the next regional town which could
be hours away.
Furthermore, there is a high proportion of Indigenous
electors who do not speak English as a first language and I am convinced that
there has been very little information about in-language material being made
available for electors in the plebiscite...
I find it very difficult to comprehend how the Australian
Government and the Australian Bureau of Statistics is reaching out to local
communities to encourage people to enrol, to update their enrolment details and
provide adequate and in-language material to encourage people to cast and
return their vote for this important societal issue.[25]
3.20
In its submission, Tangentyere Council Aboriginal Corporation
(Tangentyere) noted that 'Town Camp street addresses are not recognised as
enrollable addresses by the [AEC]'. [26]
This has a number of serious consequences for Town Camp residents:
Not having a street address means that with the exception of
residents with a separate post office box no resident receives mail from the
Australian/Northern Territory Electoral Commission. This in turn leads to
reduced enrolments, obsolete enrolment details and poor rates of participation.[27]
3.21
Tangentyere shared its observations of low participation in Town Camps in
the Alice Springs area:
The evidence suggests that very few Town Camp residents
received the 'Marriage Equality Postal Survey' without the intervention of
Tangentyere and the flexibility of the Australian Bureau of Statistics.
Tangentyere canvassed residents from Ilperle Tyathe, Aper-Alwerrknge, Mount
Nancy, Anthelk-Ewlpaye, Nyewente, Akngwertnarre, Ewyenper Atwatye, Yarrenyty
Arltere (Larapinta Valley), Anthepe, Inarlenge, Ilyperenye, Ilparpa (New
Ilparpa), Mpwetyerre, Karnte and Lhenpe Artnwe and was informed that other than
those with [post office] boxes at Australia Post nobody received the postal
survey.[28]
3.22
Furthermore, Tangentyere noted that this situation 'would be systemic
across remote Aboriginal communities, outstations and Town Camps throughout the
Northern Territory'.[29]
Updating of the electoral rolls
3.23
Mr Palmer acknowledged that the electoral roll presented risks to the
postal survey process:
Another risk is that the electoral roll might not be an
accurate reflection of all eligible Australians. We're mitigating that risk by
undertaking a major communications campaign and working with the AEC
[Australian Electoral Commission]—well, the AEC is undertaking an update of the
electoral roll.[30]
3.24
The committee heard that where a person has not notified the AEC of a
new address before the closure of the electoral roll that a new form could only
be provided to that person when they bring it to the attention of the AEC. Mr
Palmer explained what would happen if a survey form was sent to the wrong
address:
They might have realised that their postal address might not
be current, in which case they can contact our information line and ask for us
to send a form to another address. We'll issue them with a new code and they
can use that code. The code that was on the survey form that was sent to the
wrong address, if you like, becomes invalid and won't be processed.[31]
3.25
Mr Andrew Brooks raised his concerns about his postal survey being
despatched to a previous address and questioned whether an updated version of
the electoral roll was being used:
When the postal survey was first sent out between Tuesday 12th
September and 25th September, I did not receive mine at my enrolled
addressed. It was not until my mother who was visiting my grandparents at my
old registered address notified me that my postal survey was there. I have been
enrolled to vote in the Parkes electorate since 2010 and even ran and voted for
myself with no problems and had no issues voting in subsequent Federal
Elections. My electoral roll details have not been registered in Blaxland since
before 2010. I contacted the ABS and same sex survey hotline who passed off my
concerns by trying to claim that I had enrolled too late after the 24th August
deadline despite knowing I was enrolled in Parkes.
I subsequently verified on the AEC website that I am enrolled
in Parkes and to make sure checked my old address in Blaxland in case there was
a double up on the system. There was none.[32]
3.26
Despite these concerns, the committee heard that the AEC had
processed 'nearly a million
transactions in the lead-up to the close of the roll for
this survey'.[33]
To put this into
perspective, 'this is approximately 250 000 more enrolment transactions than
for the close of rolls period for the 2016 election'.[34]
Lost and stolen forms
3.27
At its 7 September 2017 public hearing, the committee sought assurances
from Australia Post that despatched and returned forms would not be lost or
stolen from the mail system and that they would be delivered on-time. Ms
Christine Corbett, then Interim Chief Executive Officer of Australia Post
stated that:
Our service performance standard is over 98 per cent of
letters are delivered on time or early. This is independently audited. We have
very strong confidence in our ability to deliver the mail, as we have done for
centuries. If anyone is concerned that they have not received a postal survey,
the ABS has robust processes in place for a survey to be then sent out to any
individual...
You are looking at 1.3 per cent that are not on time. When
you then look at a delay of a further day or so on top of that, you are very
close to 100 per cent.[35]
3.28
About 10 days later, the committee became aware of the alleged theft of
postal survey forms in Melbourne.[36]
3.29
Fairfax Media reported that:
When prep teacher Kerry Ford returned to her Brunswick home
from a holiday on Monday, what she found behind her carport made her angry and
upset.
Seventeen addressed and unopened same-sex marriage postal
vote envelopes had been dumped in front of her children's cubby house.
The letters were addressed to houses in the surrounding
streets.
"I think it happened last week, some of them have been
eaten by snails," she said. "Our postal votes have also been
stolen."[37]
3.30
Stolen forms were also reported in other locations including from Mona
Vale on Sydney's northern beaches.[38]
3.31
The deficiencies in the complaints handling process are discussed in
Chapter 4.
Overseas voters
3.32
The ABS also answered questions about the postal survey process for
eligible overseas based voters. The committee were informed that overseas
voters in the first instance would be provided with the option to complete
their postal survey online. Mr Palmer explained:
...we have tried to make it even [easier] for those people by
sending them a secure access code straight up rather than requiring them to
contact us. So those people who had an overseas address on the roll will get a
letter with a 16-digit secure access code and, from the 25th [of
September 2017], they'll be able to access the online form or the telephone.[39]
Scrutiny of the survey
3.33
The survey was not subject to the level of scrutiny usually applied to
an electoral process.
3.34
Mr Palmer described to the committee how the ABS would ensure the high
quality and integrity of the statistics gathered during the postal survey and
what elements that this would focus on:
We will have an overarching assurance
process...It will ensure that everyone who was on the electoral roll was
given an opportunity to participate; that every valid form returned was
processed and counted; that no person was able to participate more than once;
that controls were in place to reduce, detect and remove fraudulent activity;
that appropriate security arrangements were maintained around the data
throughout the survey; that there was a separation of the response from
personal information which was maintained throughout the survey; that all
physical and electronic data was destroyed at the end of the survey; and,
finally, that there were opportunities provided to external observers who were
able to assure around the interpretation of marks made on survey forms. So
we've got that, if you like, end-to-end coverage of steps in the process to
ensure integrity. Those controls will be reviewed and reported on by
independent auditors and assessors. Their report on these controls will be
included in our quality statement when we publish the statistics on 15
November.[40]
3.35
Notwithstanding these arrangements, Mr Kalisch made the point that the postal
survey is different to an election in that 'there are not the traditional
scrutineering processes you would be familiar with for an election'.[41]
3.36
In place of scrutineers, the ABS made allowance for observers. Observers
had more limited access to survey materials than scrutineers would have in an
election, and were subject to far stricter eligibility requirements:
The external observers are people who will be nominated by
two parliamentary committees to participate in the survey process in a
particular way. The role of an observer will be to witness the interpretation
of survey responses in order to report on their observations of the integrity
of this key aspect of the survey process. We're requiring that observers be
over 18 years of age. They'll be required to undergo a police check and sign a
deed of confidentiality.[42]
3.37
Mr Ian Brightwell, a former Chief Information Officer and Director of
Information Technology at the NSW Electoral Commission commented on the
integrity and quality control processes at the ABS. Mr Brightwell noted that
the ABS published online its procedures around the use of observers during the
postal survey. Interestingly, Mr Brightwell contacted the ABS call centre to
discuss these procedures in mid-August and was 'told by operators on both
occasions that they did not believe there were any procedures available and did
not know when they would be available'.[43]
3.38
Furthermore, Mr Brightwell pointed out that 'observers would only be
allowed to observe key steps in the survey process' rather than the entire
process. He also noted that the independence of this scrutiny process would be
undermined in that observers would report directly to the Australian
Statistician rather than to an independent body such as a parliamentary
committee.[44]
3.39
On 15 November 2017, the ABS published a summary of results of the
external observers' observation program. The ABS described this process:
External observers provided feedback on the coding decision
[yes/no] made by the [ABS] on 606 991 survey returns.
3.40
Of the 311 318 automatically coded Yes or No survey forms that were
examined by the external observers, there was only one instance where both
observers disagreed with the ABS coding decision equating to 0.0003 per cent of
these forms.[45]
Table 3.1 below summarises the results from the external observer program.
Table 3.1—Summary of
results from postal survey observation program[46]
3.41
At the same time, the ABS also
released a quality and integrity statement. Importantly, this statement noted that
the 'Australian Marriage Law Postal Survey (AMLPS) statistics meet the
requirements of the Census and Statistics (Statistical Information)
Direction 2017 issued by the Treasurer on 9 August 2017'.[47] The ABS noted that
the 'collection and processing
of survey responses was undertaken with rigour and the results accurately
reflect the views of survey respondents'.[48]
Furthermore:
An overall participation rate of 79.5%, with consistent
distribution across age groups, gender and geography, is a strong indicator of
quality. This high participation rate was achieved because of Australians'
significant interest in this topic combined with the statistical collection
design and process which promoted participation and ensured quality and
integrity, in particular:
-
A simple survey form containing a single question, supported with
straightforward instructions that made it easy for participants to understand
and respond.
-
The very high proportion of participants complying with the
form's instructions, enabling accurate coding of responses (with accuracy
reviewed by external observers).
-
Rigorous survey methods that included quality controls and
integrity checks which were subject to independent review and assurance.
-
Protections against fraud that included mechanisms to guarantee
only one response was counted for each participant.[49]
Collection and destruction of survey forms
3.42
The committee heard that all forms 'will be destroyed within 60 days of
[the ABS] publishing the statistics on 15 November' and that 'there will never
be an electronic file containing both the survey response and the barcode'. The
committee understands that destruction of the forms occurred within 60 days.[50]
Navigation: Previous Page | Contents | Next Page