- Conduct of performance statements audits
- As discussed in chapter one, performance statements audit is a new body of work for the ANAO to improve the accountability of government entities, which the Committee has strongly supported.
- This chapter will examine the ANAO’s approach to and conduct of performance statements audits discussing:
- ANAO’s audit schedule
- capability impacts
- the performance statements audit methodology
- impact of performance statements audits on entity reporting, and
- future of performance statements auditing.
ANAO’s performance statements audit schedule
3.3The ANAO plans to increase its performance statements audits in a phased approach, auditing the same entities annually with additional entities added to increase the number of audits conducted. As a result, once entities are audited, they will continue to be subject to annual audits.
3.4The ANAO conducts these performance statements audits in three phases:
- planning phase where ANAO develops its audit strategy and issues its engagement letter
- interim phase where ANAO assess the appropriateness of measures and internal controls, and issues an interim management letter, and
- final phase where ANAO makes a final assessment of the appropriateness of measures and IT controls and assess the completeness and accuracy of the annual performance statements based on evidence, and issues an Auditor’s Report and Final Management Letter.
- The ANAO reported that like financial statements audits, annual performance statements auditing will:
- assist entities to improve the quality and reliability of their annual performance statements; and
- provide the Parliament with reasonable assurance that the performance statements contain accurate and reliable information about the entity’s performance in achieving its purposes.
- However, audited entities have raised concerns with both the timing of the audits and their frequency.
- The Department of Education (Education) stated that the ANAO’s audit process did not align well with the department’s existing planning and performance cycle and introduced timing pressures. The department stated this made it difficult to improve its performance information or resolve findings within the next performance cycle. Education stated it was important to consider the timing of audits and the interplay with the performance cycle:
We see value in the audits raising issues around the completeness of our measures, the structure of the measures, but find that it takes us 12–18 months to actually cycle that through the process and get them published.
3.8The Department of Veteran’s Affairs (DVA) also expressed concerns with the timing of performance statements audits, noting that:
Annual [performance statements] audits do not fit well with the planning and performance cycle and potentially misses opportunities for improvement. The planning and performance cycle requires entities’ performance information for the upcoming reporting year to be finalised in May in order to be published in Portfolio Budget Statements. This is well before the current reporting years’ audit is finalised (and in some cases, before the interim management letter has been received). This does not give entities adequate time to absorb and appropriately address the feedback, and could lead to entities taking a compliance focus.
3.9The Department of the Treasury (Treasury) stated that its performance team were overwhelmed at times by supporting the ANAO audit and completing its normal business. Treasury stated that auditing entities less frequently would allow entities to focus on addressing ANAO findings and maturing their performance framework between audits.
3.10The ANAO stated annual audits incentivises entities to meaningfully improve performance reporting in a way that less regular audits do not:
Past experience has shown that periodic, rather than annual, external review of performance information is unlikely to drive the desired level of improvement in performance measurement and reporting. Annual audits provide an incentive in the system for entities to prepare annual performance statements to the standard necessary to meet the Parliament’s purposes.
3.11The ANAO also stated that annual rather than periodic performance statements audits could keep audit costs down as there are additional start-up costs for both ANAO and the audited entities. ANAO stated the challenges of conducting performance statements audits less frequently include needing to obtain a more through understanding of the entity and changes since the previous audit, and managing senior management commitment to good performance reporting.
3.12ANAO stated that one of its areas of focus is bringing forward the commencement date for its performance statements audits to ‘enable earlier insights to be provided to entities to assist them improve the quality of performance information included in the following year’s Portfolio Budget Statements and corporate plan.’
3.13ANAO advised the Committee that it aims to start performance statements audits earlier and begin the interim process looking at the entities performance measures as early as possible. ANAO stated that this would allow it to start looking at measures and systems close to the end of the calendar year to allow findings from this section to feed into entities’ corporate plan processes, including entity’s Portfolio Budget Statements.
3.14The Auditor-General expressed the view that beginning the process earlier would address some of the entities concerns in this respect and allow them to implement changes in their corporate plan in response to ANAO findings within a one-year cycle, rather than two years.
3.15As discussed in chapter one, the ANAO can currently only conduct performance statements audits upon request from the Finance Minister, or other relevant Minister, under section 40 of the Public Governance, Performance and Accountability Act 2014 (PGPA Act). The ANAO cannot commence its work on the next cycle of performance statements audits until this approval has been received. This approval is not required for the ANAO to conduct all performance or financial statements audits.
3.16The Committee has previously recommended this requirement be amended on two separate occasions. In 2017 the Committee recommended amending the PGPA Act to allow mandatory audits of annual performance statements. Additionally in 2022, the Committee recommended that the Auditor‑General Act 1997 be amended to allow the Auditor-General to initiate audits of annual performance statements without external approval or direction. Neither of these changes have been implemented. The Committee has not received a government response to its 2022 report on its Review of the Auditor-General Act 1997.
3.17The ANAO noted that it received the Minister for Finance’s request for the 2023–24 program of performance statements audits on 18 July 2023. This approval allowed the audits to begin earlier than in previous years, consistent with the ANAO’s comments. The Finance Minister did not request the 2021–22 annual performance statements audits be undertaken until October 2021. For the 2022–23 performance statements audits, this request was not received until January 2023.
Capability impacts
3.18A number of audited entities raised concerns that the annual performance statements audits put increased pressure on the limited staff prepared to support performance statements. This issue was often linked to the scheduling of audits discussed above.
3.19DSS stated that entities need a substantial uplift to prepare for performance statements audits and suggested that the Committee consider auditing entities periodically based on risk to minimise the diversion of resources from programs and policy to supporting the audit.
3.20DVA submitted that supporting a performance statements audit process was resource intensive, and that capable staff within this area were already a limited resource. DVA stated its limited capacity was stretched over attempting to improve performance reporting, deliver the annual performance statements and support the ANAO audit, often at the same time. This links to the evidence above discussing the audit schedule. DVA also stated that ‘there is no professional stream for performance specialists, whilst the skill set required is niche and subject to market competition.’
3.21The Department of Education (Education) submitted that there was a significant workload created by the ANAO audit, advising that the ANAO asked it to respond to approximately 600 audit questions, conduct approximately 40 walkthroughs and respond to 35 audit issues. Education stated that this volume of work placed unexpected resourcing pressure on the department, but noted that efficiencies in these processes should be realised as ANAO’s audit approach matures.
3.22Treasury stated that it had a small performance team who were at times overwhelmed by ANAO’s information requests in addition to their usual priorities, such as developing the annual performance statements. Treasury also stated that unlike financial statements, performance reporting staff do not ‘grow up through a professional field.’ Treasury stated that as a result, finding staff and building capability across the Commonwealth can be difficult.
3.23The ANAO advised the Committee that the resource costs experienced by entities are largely due to a lack of maturity in processes:
The resource impact on entities, especially when being audited for the first time, is largely reflective of an under-investment by entities in building their performance reporting frameworks. This is despite legislative obligations requiring accountable authorities to prepare annual performance statements being in place for nearly a decade.
3.24Finance also submitted that it had observed a difference in the size and capability of teams dedicated to an entity’s financial reporting compared to those dedicated to non‑financial reporting.
3.25The ANAO also stated the core skills used for performance monitoring are largely the standard skills that you would expect of staff managing programs in the public sector. The ANAO noted specialist skills, such as measurement, may be needed, but that they are not unique to performance monitoring and are well established in entities such as the Australian Bureau of Statistics. The Auditor-General stated:
I'd argue that the skills that [audited entities are] talking about—about building good performance information—aren't a novel thing. There has certainly been underinvestment in them, and we've seen that not just in the performance information and reporting framework but in the evaluation activity in the public sector as well.
Performance statements audit methodology and guidance
3.26Under the Commonwealth Performance Framework discussed in chapter one, it is up to the entity to determine how best to report their performance, including what their performance measures and targets are, and the structure and format of performance statements. Within this context, the ANAO assesses whether an entity’s reporting is consistent with the requirements of the Commonwealth Performance Framework and if this accurately reports on the performance of the entity in relation to achieving its purposes.
3.27The ANAO’s approach to performance statements audit has many similarities to financial statements auditing practices. However, unlike financial statements preparation, performance reporting and developing meaningful performance information is an underdeveloped capability in the public sector. As a result of this, ANAO’s approach to these audits involves a higher degree of education than other audit products.
3.28The ANAO conducts its performance statements audits under the ANAO Auditing Standards which are based on the Standard on Assurance Engagements ASAE 3000 issued by the Auditing and Assurances Standards Board. The ANAO reported it was finalising a performance statements audit manual and would make this available on its public website. ANAO also noted that would continue to refine its audit methodology to ensure it remained fit‑for‑purpose.
3.29In the evidence received by the Committee multiple entities raised the lack of a performance statements audit methodology or transparency of this process and noted this impacted their planning and support for the audit.
3.30DSS recognised the challenge in creating a detailed audit methodology but noted ‘there would be benefit in a more detailed outline of the ANAO methodology for conducting performance statements audits and expectations in the early stages of engagement – well ahead of the agency being subject to the audit process.’ DSS stated this guidance would help entities prepare for the audit process before they are subject to it.
3.31DVA stated that despite being based on the financial statements approach, performance statements audits do not have the equivalent detailed standards to audit against. Instead, DVA stated that there are the high-level principles articulated in the PGPA Act. DVA noted this required an evolving approach to the audits and the requirements they are auditing against, which has created challenges for the department. DVA also stated that it ‘has welcomed the ongoing engagement with the ANAO as the process matures, including the provision of an audit strategy document for the 2022–23 audit.’
3.32Education stated that though Finance’s resource management guides are a useful tool, the application of the PGPA Rule can be subjective which can be challenging in performance statements audits. Education stated that it expects the ANAO’s performance statements audit manual to provide the transparency needed to understand the policies and guidance applied to these audits. It also commented that a common understanding of the PGPA Rule would also support better engagement with ANAO.
3.33Treasury submitted that ANAO did not provide documentation for the 2021–22 audit with sufficient detail on the audit methodology or process to help the department prepare. Treasury further stated in its submission that ANAO progressively provided information as it became available which created substantial uncertainty and it was concerned that ANAO had not clearly defined requirements of a standardised performance audit methodology before starting the audit. Treasury advised the Committee at the public hearing that at times it ‘feel[s] a little bit like building the plane and flying the plane.’
3.34Treasury additionally submitted that it would be helpful to have more information on the audit methodology and ANAO’s interpretation of the PGPA Rule and the Finance resource management guides before the audits commence, noting that ANAO provided an audit strategy in the planning stage of its 2022–23 audit process. For example, Treasury highlighted in its response to questions from the Committee that the ANAO advised that it conducts its completeness assessment of entities’ purposes, key activities, and performance measures against the entities administrative arrangements orders (AAO) and responsible minister’s charter letters. However, Treasury stated that the Finance Resource Management Guide indicates that entities should only consider AAOs when developing their purpose.
3.35Treasury also remarked in its submission that a financial statements compliance approach does not translate effectively to performance reporting. Treasury indicated there is a risk that entities may take a compliance-based approach to minimise the risk of audit qualifications, rather than providing transparent and innovative measures, highlighting the need for a fit-for-purpose audit methodology.
3.36Finance, as the steward of the Commonwealth Performance Framework, submitted that performance measurements and the audits of them are inherently complex. Finance expressed the view that settling ANAO’s performance statements audit methodology will help entities better understand the audit approach as well as how to use performance information to report on objectives, rather than purely a compliance approach. Finance also advised the Committee that it regularly engages with ANAO on performance statements audits and this engagement informs Finance’s guidance and advice to entities.
3.37The ANAO stated that transparency and level of detail in the audit methodology can be limited by the type of requirements the ANAO is auditing against:
The more templated the reporting framework is the more templated the auditing process is. Financial audit has lots of rules down to lots of detail about how things can be done, so you can template that down quite a lot in an audit process. Performance statements are relatively high level in terms of the rules and the guidance, so the methodology is less templated because you can't do it that way….The higher the level the rule is set then the greater the likelihood of disagreement of interpretation.
3.38In its response to written questions from the Committee, the ANAO stated that ‘there is a risk that a narrow focus on compliance when conducting performance statements audits could increase compliance without necessarily increasing the quality of performance statements.’ However, the ANAO stated that its audit approach ‘aims to improve the quality of performance reporting and provide assurance to the Parliament.’ The ANAO stated it will continue to refine its audit methodology to ensure it is fit-for-purpose and drives continuous improvement.
3.39The Auditor-General advised the Committee at the public hearing that the ANAO plans to publish its performance statements audit methodology ‘in the coming weeks’ and has begun releasing an audit strategy document to audited entities that sets out what the ANAO will likely look at. In its submission to the inquiry, Treasury noted that the ANAO provided an audit strategy as part of its 2022–23 audit process. Treasury further commented at the public hearing that this ‘was a really helpful step forward in the maturity of the process’ and provided Treasury with the ‘left and right of arc’ for the audit.
3.40The ANAO also noted in its report that performance reporting practices in the public sector are still evolving, and to ensure that the quality of performance reporting continues to improve it will continue to mature its audit methodology.
Guidance on performance statements requirements
3.41In addition to the above comments on ANAO’s audit methodology, the Committee received evidence from entities on the sufficiency of current guidance on the Commonwealth Reporting Framework.
3.42Finance advised the Committee that it ‘considers that its guidance and support is appropriate.’
3.43Finance also stated in its submission that it supports entities to meet the intent of the Commonwealth Performance Framework in its stewardship role:
In a principles-based environment Finance takes a stewardship approach to supporting entities meet the requirements and intent of the PGPA framework. This involves the development and maintenance of a range of support tools and activities.
3.44Finance advised in its submission that this support included developing and updating resource management guides as guidance material, as well as holding regular communities of practice forums, entity-specific Commonwealth performance Framework 101 sessions, distributing a PGPA newsletter and conducting an annual PGPA entity survey.
3.45Finance also advised the Committee that it regularly engages with ANAO on performance statements audits and that this engagement informs Finance’s guidance and advice to entities.
3.46DSS submitted that the PGPA Act was intentionally flexible to allow for accountable authorities to make decisions appropriate to each entity’s operating context. However, it noted that clearer guidance on the intention of the PGPA Act and examples of better practice would better support entities planning for performance statements.
3.47Education also indicated that the sector lacks an example of an organisation with a mature performance reporting framework to use as a guide of what better practice looks like.
Impact of performance statements audits
3.48The ANAO advised the Committee that it has observed an increased focus from entities on their performance measurement and reporting, including improved governance processes. The ANAO stated:
This demonstrates that audits of annual performance statements can positively influence the quality and reliability of performance reporting to the Parliament. Moreover, our sector engagement shows that some entities not yet subject to audit are investing to improve the quality of information presented in performance statements.
3.49The ANAO reported that audited entities sought to resolve findings from both the 2020–21 performance statements pilot and also during the reporting period where possible. Additionally, the ANAO found that the three entities audited in the pilot program (AGD, DSS and DVA) resolved the findings that led to qualifications for 15of the 16 (94 per cent) measures.
3.50The Committee also received evidence from both entities involved in the performance statements audits and Finance in its role as the steward of the Commonwealth Performance Framework, on the positive impact of the performance statements audit work both on entities directly and the sector more broadly.
3.51Finance submitted that the ANAO’s findings demonstrated that entities subject to the audits were improving their performance reporting, but that more broadly entities are sharing and applying lessons learnt from the audits to continuously improve their performance reporting.
3.52DSS submitted that its performance reporting has improved due to the ANAO’s audits stating:
The assurance audits have supported greater accuracy in performance reporting which in turn helps transparency. The department continues to mature its approach to performance measurement through the lessons learned each year.
3.53DSS also noted that its constructive relationship with ANAO during the performance statements audits allowed emerging issues to be addressed and supported transparent reporting.
3.54Education submitted that it continues to improve its quality of performance reporting and that in response to the ANAO’s recommendation, it had improved its performance statements preparation processes and is developing an enterprise level performance framework.
3.55DVA noted it had been involved in performance statements audits since the pilot program and stated the benefits of this:
While there have been challenges, the audits have helped to improve the transparency, robustness and completeness of DVA’s annual performance statements. DVA welcomes the opportunity to continue working with the ANAO as the processes mature.
Future of performance statements audits
3.56The ANAO has been provided funding to increase the number of performance statements audits it conducts from six in 2021–22, to 24 in 2025–26.
3.57As discussed earlier, ANAO advised the Committee that it intends to publish its performance statements audit methodology, and will continue to mature this methodology to ensure that the quality of performance reporting continues to improve. The ANAO reported in the audit that an area of particular focus is ensuring that entities are not limiting performance information to avoid potential audit findings and noted it will ensure that:
The methodology incentivises entities to positively embrace performance reporting requirements and prepare performance statements that report performance effectively rather than a focus on complying with minimum reporting requirements or meeting the minimum standard they think will satisfy the auditor.
3.58To support this continued development, ANAO has established a Performance Statements Expert Advisory Panel (the Panel) to:
- provide advice on how the audits can contribute to improving performance information
- provide views on the appropriateness of the current audit methodology, and
- improve ANAO’s understanding of strategic and operational issues related to performance statements audits.
- The Panel includes representatives from the ANAO, Finance, Treasury, the Parliamentary Budget Office, a state or territory Auditor-General, and several entities’ audit committee chairs.
- Treasury indicated at the public hearing that it supported and commended ANAO’s development of the Panel and that it would help identify and address issues with the performance statements audit methodology.
Committee comment
3.61Performance statements auditing is the most significant change to public sector auditing that we have seen in decades. It’s an important change, bringing greater reliability and transparency to how government entities are performing to both the public and the Parliament. However, it is also a departure for ANAO from its well‑established standards and methodologies, particularly for financial statements audits. This puts the ANAO in the new position of establishing, rather than refining, an audit methodology.
3.62ANAO’s scheduling and timing for the performance statements audits was an issue that was brought up by many entities, many of which stated that annual audits were too frequent and that the current cycle didn’t align with existing performing reporting cycles.
3.63The Committee has considered these concerns, but has also noted ANAO’s comments that annual audits provide better results and prevent additional set-up costs for both the entity and ANAO. The Committee agrees with the ANAO that annual performance statements audits are appropriate to minimise costs and to better drive improvements in performance monitoring and reporting.
3.64However, the concerns raised by entities related to the timing of these audits are significant. A key factor impacting the schedule which ANAO conducts the performance statements audits in has been the requirement for the Finance Minister to request that ANAO conduct these activities. This is an issue that the Committee has considered for many years and has twice recommended be removed as a requirement.
3.65The need to request approval from the Finance Minister is unique to this audit work and creates unnecessary delays and inefficiencies. Although the ANAO received an early request for the upcoming 2023–24 cycle, this timeliness cannot be guaranteed going forward and can compound the issues raised by entities in relation to the audit and performance reporting cycles not being aligned.
3.66The Committee reiterates its prior recommendation in Report 491 that the Auditor-General Act 1997 be amended so that annual performance statements audits can be initiated by the Auditor-General without the need for approval, within the next 12-months.
3.67The Finance Ministers’ approval to begin is not the only factor that influences the timing of these audits. How ANAO schedules its work once it can begin the audit is another factor. Entities emphasised the importance of the audit schedule better aligning with the performance reporting schedule, noting that entities can receive findings too late to action in the current cycle. It is not reasonable to potentially give an entity the same finding in two reporting cycles, because the result was not provided to them early enough for them to change. Entities need to be given enough time to action ANAO’s findings, especially in relation to the appropriateness of performance measures before they are assessed again.
3.68Considering this, the Committee agrees that the ANAO needs to begin these audits as early as possible in the cycle to allow findings from the interim stage to contribute to an entity’s portfolio budget statements. The Committee considers that, if this is actioned, it will address many of the entities’ concerns related to the cycle of audits. The Committee looks forward to seeing the impact of ANAO’s efforts to begin the audit cycle earlier in its examination of ANAO’s performance statements audits going forward.
3.69The Committee recommends that the Australian National Audit Office and the Department of Finance work together to develop and publish a revised annual performance statements audit schedule that allows the Australian National Audit Office to begin its work earlier in the cycle so findings from the interim stage contribute to an entity’s annual portfolio budget statements and report back to the Committee on progress within six months.
3.70A number of entities also raised the impact of these audits on limited pool of staff in the performance monitoring and reporting space. The Committee understands finding and retaining high quality staff in the public sector is a challenge. The growing interest in the public sector in performance reporting as a result of the ANAO’s performance statements auditing also puts increased pressure on entity staff with the relevant expertise or experience. However, the Committee agrees with ANAO’s comments that the core skills used in performance reporting are program management and measurement skills, rather than bespoke performance reporting expertise.
3.71The Committee also understands that there can be a resource impact when entities are first audited but notes the Auditor-General’s view that this is often due to a previous lack of investment in meeting the Commonwealth Performance Framework.
3.72The ANAO has communicated since it began the pilot program that the methodology will change and evolve as it improves its processes. The Committee welcomed evidence from multiple entities that despite facing challenges and issues in the performance statements process, as any new body of work is likely to generate, entities’ engagement with the ANAO has been positive, collaborative and constructive. The Committee commends both the ANAO for forming these productive relationships, and the audited entities for meaningfully engaging with ANAO in this new process.
3.73The Committee looks forward to the ANAO publishing its performance statements audit methodology and agrees with entities that this, along with the ANAO’s new process of developing and sharing audit strategies with entities will help reduce confusion and allow entities to better prepare for audits.
3.74The Committee understands that further guidance in this area would be welcomed and will be interested to see how Finance’s role in providing this on the requirements of the Commonwealth Performance Framework evolves as ANAO’s performance statements audit work matures.
3.75The Committee recommends that updated advice be provided to it within six months from:
- the Australian National Audit Office regarding the publication of its performance statements audit methodology, the flagged audit strategy documents and initial feedback from agencies on these documents, and
- Department of Finance regarding any further guidance across the system that has been issued or is planned in relation to performance statements auditing.
- When inquiring into a body of work that is maturing as it is being applied it is often easy to focus on the challenges rather than the successes. The Committee was heartened to see the benefits of the performance statements audits were identified by both the ANAO and the audited entities. The improvements in the audited entities and across the sector shows the importance of these audits, and that this work to increase transparency needs to be continue and grow.
- The Committee supports ANAO’s commitment to continuing to mature its approach to performance statements audits through its Performance Statements Expert Advisory Panel. As shown in its previous inquiries and engagement on this topic, the Committee strongly supports ANAO’s work in performance statements audits and is encouraged by their progress and the impact this has, and will continue to have in the public sector in the future.
Mr Julian Hill MP
Chair