Audit Report No. 28 2008-09
Chapter 4 Quality and Integrity of the Department of Veterans’ Affairs
Income Support Records
Introduction
4.1
The Repatriation Commission was officially established on 1 July 1920
with the passing of the Australian Soldiers’ Repatriation Act 1920, to
provide support for veterans, widows and their families.[1]
This Act was replaced by the Veterans’ Entitlements Act 1986 (the Act),
which retained the functions of the Repatriation Commission. Under the Act, the
Commission delegates its powers to the Department of Veterans’ Affairs (DVA) to
grant pensions and other benefits to veterans and their dependants, and certain
other eligible people.
4.2
The service pension provides regular income support for people with
limited means of income and is broadly equivalent to the Centrelink age and
disability pensions. However, it is payable five years earlier than the age
pension in recognition of the effects of war. The two critical criteria
required of claimants in the Act are to be a veteran and have rendered
qualifying service. Age and residency requirements are also mandatory for some
pensions. There are rules related to assets and income that also affect pension
eligibility and pension amount.
4.3
The total required to deliver this output in 2007-08 was $44,744,000.[2]
DVA’s client population has declined over the past three financial years. At 30
June 2008, the income support pensioner population was 4.6 per cent less than
was reported in the previous financial year. There has been a corresponding
downwards trend in the department’s workload.[3]
4.4
In order to better manage the services provided to Australia’s declining
population of war veterans and the expected decrease in its workload and
funding, DVA undertook a major restructure in 2005-06. This restructure, known
as oneDVA, was the foundation for a new approach to the way DVA delivers
its services to veterans. The reorganisation involved a move away from variable
State-based practices and the expansion of geographically dispersed teams
resourced along national business lines. The department’s strategy to
standardise its operations across all service areas was complemented by the
introduction of the first phase of a new ‘oneVoice’ telephone service
environment, the Veterans’ Service Centre (VSC).
4.5
The department’s administration of income support benefits relies on a
combination of complex older heritage and new information technology (IT) systems.
These systems contain extensive electronic records of personal and other
information about DVA’s clients and are used to process income support claims
and other related work. In February 2008, the department’s client databases
held 1,580,546 total client records, of which 264,248 involved an income
support payment.
4.6
In parallel with DVA’s restructure, the department also recognised the
need to modernise its IT and reduce its expenditure on maintenance of heritage
systems.[4] As part of this
initiative, the department purchased Cúram, an off-the-shelf, integrated IT
application framework designed for service delivery environments. This
IT-enabled business change offered the potential for better functionality of
DVA’s IT systems based on up-to-date business rules, more reliable information
underpinning decision-making and reporting, and increased convenience for
veterans.
4.7
At the time of the audit DVA was taking a phased approach to
implementing Cúram, with data migration (from the department’s heritage IT systems
into Cúram) scheduled over a number of years. The initial phase of the
department’s modernisation of its IT systems occurred in 2006, when several
projects were implemented. This involved the migration of over 1.5 million
records containing client personal information from heritage databases into the
new Cúram environment. Despite DVA’s considerable project planning, data
testing and cleansing to prepare the heritage data for transfer into Cúram,
unanticipated data incompatibility and integration issues emerged during the
migration of the data. To enable the transfer of the data into Cúram, heritage
records with blank date of birth fields were populated by DVA with ‘dummy’
data.[5]
4.8
The department has a major initiative to clean up data integrity errors.
This is the Data Integrity Cleanup Exercise (DICE) project which predominantly revolves
around correcting:
n data errors that were
transferred from heritage systems into Cúram; and
n newly created
problems that arose as a result of incompatibility issues between heritage and
Cúram systems during data migration.
4.9
The department also has an ongoing program for data cleanup – the Data
Integrity Problems (DIPs) work. DIPs activity is directed to correcting complex
data errors unable to be resolved directly via the existing applications.
4.10
The cost of implementing Cúram was to be offset by ongoing savings in
administration and program costs arising from the improved IT framework and
de-commissioning of relevant heritage systems.[6]
The Audit[7]
Audit scope and objective
4.11
The objective of the audit was to examine the quality and integrity of
DVA’s income support records and to report on the effectiveness of the
department’s management of the data and how it impacts on service delivery. The
audit included an examination of:
n DVA’s management of
the data including the quality of data stored on its client databases, the
processing of claims and payment processing;
n the accuracy,
completeness and reliability of DVA’s electronic income support records; and
n the impacts the
quality of data has on service delivery, and related issues of customer support
and feedback.
4.12
The audit focused on data integrity issues associated with the various
types of service pension, income support supplement, the social security age
pension and other related allowances. The accuracy and completeness of records
of selected mandatory fields and other key fields that underpin the integrity
of DVA’s income support records were examined.
4.13
The ANAO’s data extraction and analysis encompassed 1,580,546 records in
DVA’s production environment which included the records of 264,248 income
support clients who were in payment.[8] While the audit did not
directly examine the accuracy of individual payments, it examined underlying
data integrity issues that can impact on the accuracy of payments, such as the
current status of client asset and income information.
4.14
The audit also included a limited number of reviews of paper files and
consideration of documentation associated with DVA’s IT governance,
particularly in relation to data management.
Overall audit conclusion
4.15
The ANAO made the following overall audit conclusion:
The Department of Veterans’ Affairs (DVA) has been undergoing
significant change since 2005-06. Following a review of its service delivery
arrangements, DVA adopted a new business and information technology (IT)
strategy, in recognition of the declining population of war veterans and
expected 30-50 per cent decrease in the department’s workload over the next 10
years. A key element of DVA’s IT strategy is to reduce expenditure on the
maintenance of IT systems over time, by eventually decommissioning its heritage
systems. The purchase of Cúram, an off-the-shelf IT product designed for social
welfare environments, was a key part of this strategy. DVA identified Cúram as
an enabler for its new model of business operation, known as oneDVA.
A challenge for DVA in this environment is balancing the
resources required to maintain its heritage IT systems relative to its
investment in new IT capability with its greater functionality. While Cúram is
designed to be the ‘source of truth’ for client personal data and to provide a
platform to better manage DVA’s data in the future, it is still in the early
stages of implementation. Income support data is not yet scheduled for migration
into Cúram, and DVA continues to be dependant on the integrity of the data
stored in heritage systems for administering income support payments. In this
environment, the department relies heavily upon the corporate knowledge held by
a few key staff about its IT systems and business processes.
Overall, the poor quality of the data in DVA’s electronic
databases is affecting the efficiency and reliability of the department’s
decision-making, and its internal and external reporting. While in most cases
there was sufficient evidence in DVA’s multiple systems and the hard copy
customer records examined to support its clients’ eligibility for income
support benefits, the audit revealed:
n the department’s
management of electronic data and data integrity issues was not effective;
n key fields in many
electronic records were not accurate, complete or reliable; and
n inaccurate recording
and reporting of complaints and compliments in the department’s Feedback
Management System.[9]
Management of data and data integrity issues
4.16
The ANAO found numerous data integrity problems during the audit,
including:
n DVA clients with more
than one Unique Identification Number (UIN) continuing to increase since the
previous audit of DVA’s administration of Repatriation Health Cards in 2003-04;
n DVA not having a
complete electronic record of the qualifying service details for 41 per cent of
veterans eligible for the age service pension;
n exempt assets of
clients being disregarded beyond the legislated exemption period for the
purpose of the assets test;
n fragmentation of
client information across multiple records; and
n cases of pension
misclassification, requiring DVA to further analyse the raw data to ensure the
information reported was meaningful and reliable.
4.17
The ANAO found that these and other data integrity issues identified by
the audit increase the risk of DVA providing untimely advice and incorrect
payments, and reduce the department’s capacity to provide assurance that the
right person is receiving their correct entitlement. To mitigate this risk, the
department has a range of administrative processes and checks in place.
However, these add to the costs of administering the program. This situation
also limits the ability of the department to garnish the dividends of oneDVA.
4.18
According to the ANAO, the quality of DVA’s data would be substantially
improved through the development and deployment of an organisation-wide data
integrity improvement strategy, underpinned by stronger governance
arrangements. In recent years, DVA has reviewed both its IT governance and
committee structure and identified similar issues to those reported by the ANAO
in this audit, including a need for greater clarity across the department for
the authority, ownership and control of data management and data integrity
issues. Strengthening oversight arrangements would assist DVA to monitor the
progress of the strategy, as well as align elements with the roll-out of Cúram
and recognise the interdependencies within the oneDVA initiative. DVA’s
progress would also be facilitated by setting targets and timeframes for
reviewing records, and better utilising the opportunities presented by client
and department initiated contacts, including compliance and review work, to
improve data integrity.
Accuracy, completeness and reliability of key fields in client records
4.19
The ANAO found that DVA’s policy and procedural controls for new claims
processing and updating of client records would benefit from review and
consolidation. In particular, data entry controls and support materials should
be standardised across the department’s State offices and the Veterans’ Service
Centres (VSC), consistent with the oneDVA strategy. This would help to
reduce data input errors and support consistency in decision-making and client
records management generally. Additional assurance would be gained from a
greater focus on data input standards and controls, and procedural compliance
around claims processing and updating of client records.
4.20
Veterans/clients receiving the maximum pension or receiving a part
pension (and considered by DVA to be low risk), are generally not reviewed
through DVA’s review program measures unless pensioners notify of a change in
their circumstances. This has potential service delivery impacts when cases are
not reviewed for a number of years as clients could incur unexpected debts or
by underpaid for a significant period. Retrospective adjustments applied to
pensions over a number of years does create higher administrative costs for
DVA, when having to account for multiple changes to a pensioner’s
circumstances.
4.21
In 2004, DVA introduced its Enhanced Compliance Program (ECP) in order
to manage the risk of pensioner non-compliance.[10]
The ECP targets cases profiled by DVA as high risk with no recent review
activity, or cases with potentially volatile income and assets. As well as
achieving higher than expected benefit payment savings, the ECP has been
successful in updating the current status of a client’s circumstances. However,
this program, coupled with the program of two-yearly reviews of pensioners
receiving less than the maximum pension rate, reviews less than 7 per cent of
the total income support population. Overall, more than 70 per cent of clients
have not had a review that updates all of their previously submitted asset
information, for eight years or more.
4.22
DVA has a range of IT controls and assurance programs, including the ECP
and client contact activities, which present an opportunity to improve data
integrity. These compliance activities and other direct contacts with clients
provide opportunities to implement cost-effective arrangements to validate or
correct client information. Improved data integrity would provide greater
assurance that DVA’s clients are receiving their correct income support
entitlement and associated services.
Management of feedback data and service delivery impacts
4.23
The ANAO indicated that the unreliability of DVA’s client feedback data
limited the department’s capacity to effectively utilise the intelligence
gathered from complaints and compliments to assist in setting client service
priorities and to systematically monitor and generate reliable public reports.
DVA’s proposed new feedback management system was found to be in the early
planning stages, with the projected timetable and resources yet to be defined.
DVA advised that, in the meantime, its existing Feedback Management System
(FMS) would continue to be used with its known data quality shortcomings.
4.24
The ANAO concluded that, in the interim, it would be prudent for DVA to
raise departmental staff awareness of the need for all feedback to be recorded
in the existing FMS and to ensure compliance with the department’s Procedural
Policy-Handling Feedback from the Veteran Community. In doing so, the focus
should be on awareness raising and highlighting the value DVA places on client
feedback and the intelligence gathered from complaints, as a driver of
improvements to business and service quality. This would assist in ensuring
that client feedback is accurately recorded and appropriately managed day-to-day
and ensure its availability for long term business and service delivery
improvement purposes.[11]
ANAO recommendations
4.25
The ANAO made the following recommendations:
Table 4.1 ANAO Recommendations, Audit Report No. 28
2008-09
1.
|
To improve the governance of income support data, the ANAO
recommends that the Department of Veterans’ Affairs:
·
clarifies the authority and role of its Data Integrity
Sub-Committee responsible for resolving issues related to data ownership and
enhancing the integrity of data held in the department’s databases; and
·
develops an agency-wide strategy, assigning ownership for data
management and integrity issues to business areas.
DVA response: Agreed
|
2.
|
The ANAO recommends that the Department of Veterans’
Affairs:
·
documents a controls framework for income support and systems
based business rules; and
·
evaluates its current IT controls and assurance activities to
determine which elements are most efficient and effective in improving data
integrity.
DVA response: Agreed
|
3.
|
To enhance the quality and integrity of income support
records, the ANAO recommends that the Department of Veterans’ Affairs:
·
strengthens its data entry system and procedural controls by
implementing a standard set of national procedures for income support client
records management;
·
validates customer data when interacting with clients and
during income support assurance activities; and
·
updates incorrect data in all key fields of active electronic
client records prior to granting a payment or benefit.
DVA response: Agreed
|
4.
|
The ANAO recommends that the Department of Veterans’
Affairs implements interim measures to enhance the quality of its feedback
data while designing the new feedback system to manage stakeholder feedback
by:
·
improving compliance by staff with the department’s current
procedural policy; and
·
raising staff awareness of the need for all complaints and
compliments data to be recorded in the department’s existing Feedback
Management System.
DVA response: Agreed
|
The Committee’s review
4.26
The Committee held a public hearing on Monday 15 June 2009, with the
following witnesses:
n Australian National
Audit Office (ANAO);
n Department of
Veterans’ Affairs (DVA).
4.27
The Committee took evidence on the following issues:
n current status of the
Cúram project;
n data integrity;
n complaints;
n update of clients’
asset records;
n incomplete electronic
records of service; and
n risks associated with
the reliance on corporate knowledge held by a few key staff members.
Current status of the Cúram project
4.28
The Department of Veterans’ Affairs reported to the Committee that since
the Audit Report was presented the Cúram project has been put on hold, largely
due to financial considerations. A review of the modernisation of the
Department’s information technology systems is being undertaken to establish
‘which is the best platform and what is the best solution to our needs going
forward’.[12]
4.29
DVA was anxious to stress to the Committee that the current difficulties
with regard to the implementation of the modernisation of the agency’s information
technology systems in no way impact on the correctness or accuracy of payments
made to veterans. The General Manager, Support Division, was quick to point out
that none of their accountability processes have been compromised:
… the current state of our systems in no way fetters our
capacity to inform government or inform anybody else about the current state of
DVA’s payments, benefits, expenditure and so forth.[13]
Recommendation 8
|
|
The Committee recommends that the Department of Veterans’
Affairs (DVA) report back to the Committee at the conclusion of the review of
the agency’s information technology systems and provide the Committee with
details of the system chosen, the implementation plan and how the plan will
address the Australian National Audit Office’s recommendations.
|
Data integrity
4.30
The ANAO identified links between the integrity of DVA’s data and its
data entry controls and recommended that the agency take specific steps to
mitigate the risk of errors and improve quality.[14]
The recommendation calls for a standard set of national procedures, the
validation of customer data whenever there is client contact and updating of
incorrect client data before granting a payment or benefit.[15]
4.31
DVA reported that it has instigated a review of its procedures which was
to be concluded in September 2009 and that identified changes will be implemented
as soon as possible. The Department maintains that it already validates data
during client contact and that staff are continually being made aware of the
need to follow this procedure. The National Manager, Income Support, assured
the Committee that this is an ongoing process:
… we sent out a business line to all system users and client
contact staff just last week reinforcing the message that they follow the Departmental
protocols of updating all the client data information, which picks up gender,
title, date of birth, name, address and contact details for the client.[16]
Complaints
4.32
The Committee expressed concern in the rise in complaints and drop in
client satisfaction with complaints handling in 2006-07 identified in the
Report. This is despite a reported overall high rate of client satisfaction
with the Department’s services. Although they did not question the veracity of
the findings, the ANAO detailed a number of weaknesses in the agency’s
complaint handling procedures and reporting.
4.33
DVA explained that the total figures masked varying levels of
satisfaction with the Department between clients covered under the Veterans
Entitlement Act 1986 (VEA), the Military Rehabilitation and Compensation
Act 2004 (MRCA) and the Safety, Rehabilitation and Compensation
Act 1988 (SRCA). Clients under the VEA tend to be more satisfied with the
service than those under the MRCA and SRCA. DVA provided a number of reasons
for this higher dissatisfaction among the latter group:
It is a system where there is a greater emphasis on
rehabilitation, they are younger individuals, they are people who have left the
Australian Defence Force (ADF), which they had seen as potentially their career
for life, and they are people who are clearly in a situation where they need a
lot more work and assistance in getting them back to the position they were in
prior to their injury and their discharge from the ADF.[17]
4.34
The Committee requested a disaggregation of the complaints data for
2006-07 and 2007-08 to allow a comparison to be made between both years. The
disaggregated figures are available in Appendix D.
4.35
The Committee sought further information on the steps being taken to
rectify the dissatisfaction being expressed by clients under the MRCA and SRCA.
The Department explained that it has set up a younger veteran’s task force to
examine the issues and develop solutions. DVA has identified communication
channels as of particular concern to younger clients and is exploring the use
of new technology to facilitate better communication between these clients and
the Department. Examples of reforms that are improving service delivery to
clients include:
n a single claim form
for all clients no matter which piece of legislation they are eligible under;
n more efficient
transfer of medical and service records from Defence to the Department; and
n a single health
examination form which reduces the need to consult separate doctors or
practitioners.[18]
Update of clients’ asset records
4.36
The service pension for DVA clients is calculated using two tests: an
income test and an asset test.[19] The ANAO found that
while changes to income are monitored through regular reviews and data matching
with Centrelink and the Australian Taxation Office, identifying changes to the
value of assets is a more difficult task.[20] The audit report found
that ‘more than 70 per cent of client’s asset records have not had all of their
previously submitted asset items fully updated for eight years or more.’[21]
4.37
The Committee expressed concern that this backlog could be detrimentally
affecting the income of veterans. DVA reported that in accordance with ANAO
recommendations, they were undertaking a review of their data entry system and
procedural controls, scheduled to be completed by September 2009.[22]
Recommendation 9
|
|
That the Department of Veterans’ Affairs (DVA) provide the
Joint Committee of Public Accounts and Audit (JCPAA) with a copy of the
review of the DVA’s data entry system and procedural controls as soon as
practicable after the tabling of the Committee’s report.
|
Incomplete electronic records of service
4.38
Qualifying service (QS) is the criteria used to establish a veteran’s
eligibility for a service pension. The ANAO found that details of service are
recorded on paper files but that 41 per cent of electronic records were
incomplete with regard to these details.[23] As DVA advised that
paper files are not used after the initial assessment, ANAO concluded that
future decisions could be based on incomplete records which would increase the
margin for error.[24]
4.39
DVA told the Committee that the transfer of information from paper
records to the electronic database is resource intensive and not a high
priority.[25] The Department explained
that once qualifying service is established there is rarely any need to access
the details:
… we do not consider the transfer of those to electronic
files as being a priority because we just do not access those files once we
have got the person in payment and eligible and we are confident of those
particular details extracted from the paper files.[26]
4.40
The Committee nonetheless concurs with the ANAO that “when information
is fragmented across different records, this leads to inefficient
administration and increases the risk of error.”[27]
The Committee expects new entrants into the system with qualifying service to
be entered into electronic records as a matter of course, and reiterates ANAO
Recommendation No. 3, which calls for stronger data entry systems and
procedural controls, better validation of customer data, and a commitment to
updating incorrect data.
Corporate knowledge and key staff
4.41
Throughout the Audit Report the ANAO highlighted the reliance by the DVA
on the corporate knowledge of a few key staff to both interpret raw data and
negotiate its IT system.[28] The Committee expressed
concern that this presents a considerable risk if this knowledge is lost as
staff retire or leave the organisation and asked what the Department is doing
to mitigate the risk.
4.42
DVA assured the Committee it has noted the Report’s finding and
explained that the risk is chiefly to do with the heritage IT systems. As the
information on these systems is migrated to newer systems, the heritage systems
will be used less and less with a consequent decrease in the number of staff
familiar with them.[29] The issue is being
considered in the review of ICT currently being undertaken by the Department.[30]
Conclusion
4.43
The Committee acknowledges that the ANAO found no instance of incorrect
payments being made to DVA clients. It understands the importance of reassuring
veterans that there is no reason for anxiety on their part with regard to the
correctness or accuracy of their payments.
4.44
However, the Committee is concerned at the discrepancies and errors
highlighted by the Audit Report and the lack of progress in implementing a
comprehensive and accurate electronic database for the Department. Despite DVA’s
reassurances, the Committee is concerned at the potential for a detrimental
flow-on effect with regard to client payments and service delivery.
4.45
The Committee recognises the importance of DVA to veterans and stresses
the need for the Department to continue to adapt to the needs of its changing
client base and deliver the same high level of support as it has in the past.
To facilitate delivery, the Committee urges DVA to implement the ANAO
recommendations to improve the data integrity of its income support records
whichever system is selected after the current review of its IT services.