Chapter 3 The Administration of Major Programs - Australian Greenhouse Office

Chapter 3 The Administration of Major Programs - Australian Greenhouse Office

Audit Report No. 34, 2003-2004

Introduction
Risk assessment
Measuring outcomes
Selection and appraisal of projects
Management and monitoring of agreements
Performance reporting

Introduction

Background

3.1

Climate change is recognised as a major issue for all nations. New and stronger evidence that humans are having an influence on the global climate through greenhouse gas emissions was presented by the Intergovernmental Panel on Climate Change in 2001.1

3.2

In 1997, and subsequently in 1999, the Australian Government introduced two major spending packages with a total value of almost $1 billion. These packages, Safeguarding the Future (1997) and Measures for a Better Environment (1999), were designed to address the challenges posed by the issue of climate change and to meet Australia’s domestic and international climate change commitments. The Australian Government has agreed to ‘develop and invest in domestic programs to meet the target of limiting greenhouse gas emissions to 108 percent of 1990 emissions over the period of 2008–2012’ – the Kyoto Protocol target.

3.3

Since its inception in 1998, the Australian Greenhouse Office (AGO) has been responsible for the implementation of these two major packages and subsequent government greenhouse policies. The agency’s mission is to lead Australia’s greenhouse action to achieve effective and sustainable results. AGO seeks, amongst other things, to facilitate projects that maximise cost effective greenhouse gas abatement and reduce growth in greenhouse gas emissions.

3.4

The ANAO audit was conducted between May and August 2003. Since then, the administrative arrangements for the AGO have changed. Following the 2004 Federal Election, the AGO was moved to become part of the Department of Environment and Heritage (DEH). As such it ceased to operate as an Executive Agency and is now a division within DEH.

The audit

3.5

The objective of the audit was to examine and report on the administrative efficiency and effectiveness of significant programs administered through AGO. The audit examined seven programs across both the 1997 and 1999 packages, which accounted for 87 percent of total program cost estimates.

3.6

Since the ANAO’s audit report was tabled, the government has announced a new suite of climate change responses as part of its energy white paper, Securing Australia’s Energy Future (released June 2004), and also in the 2004-05 Federal Budget. While some existing AGO programs remained untouched, others were refined and some new programs were announced, such as the Solar Cities Programme. This new package is known as the government’s Climate Change Strategy. Total funding under the Climate Change Strategy (including existing funding from the 1997 and 1999 programs) now totals $1.8 billion.2

3.7

The programs reviewed in the ANAO’s audit are outlined in the table below.

 

Table 3.1 AGO programs examined in ANAO Audit Report no. 34

Program

Announced funds ($ million)

1997 – Safeguarding the Future Package

 

Greenhouse Challenge Program (Challenge)–a voluntary industry program to reduce greenhouse gas emissions, drive continuous improvement and enhance knowledge and understanding of cost effective ways of managing greenhouse gas emissions.

27.1

Renewable Energy Equity Fund (REEF)–an investment program to encourage the commercialisation of research and development in renewable energy technologies by addressing capital and management constraints.

21.0 A

Renewable Energy Commercialisation Program (RECP)–a grant program to support innovative renewable energy equipment, technologies, systems or processes that have strong commercial application and the prospect of significant abatement of greenhouse gas emissions over the longer term.

29.6

1999 – Measures for a Better Environment Package

 

Greenhouse Gas Abatement Program (GGAP)–a grant program to support activities likely to result in substantial emissions reductions or substantial sink enhancement, particularly in the first Kyoto commitment period 2008–2012.

400.0

Renewable Remote Power Generation Program (RRPGP)–a grant program to increase the uptake of renewable energy technologies in remote areas, assist in developing the renewable energy industry, help meet the energy needs of indigenous communities and lead to long-term greenhouse gas reductions.

264.0 B

Photovoltaic Rebate Program (PVRP)–a grant program to encourage the long-term use of photovoltaic technology, increase renewable energy in Australia, reduce greenhouse gas emissions, assist in the development of the photovoltaic industry and increase public awareness of renewable energy.

31.0 C

Alternative Fuels Conversion Program (AFCP)–a grant program to reduce greenhouse gas emissions and significantly improve urban air quality by facilitating heavier commercial road vehicle and public transport buses to operate on compressed natural gas (CNG) or liquefied petroleum gas (LPG).

75.0 D

Extension of RECP–as per 1997 package with additional funding for industry development component.

26.0 E

Total Value of Programs Evaluated

873.70

A Subsequent revised estimate of $19.5 million.

B Subsequent revised estimate of $179.9 million.

C Subsequent revised estimate of $34.6 million.

D Subsequent revised estimate of $71.4 million.

E The audit only examined the $20 million extension of the RECP not the $6 million allocated to the industry development component.

 

Source ANAO Audit Report No. 34 2003-04, p. 28.

Audit findings

3.8

The ANAO found that the administration of greenhouse programs had been characterised by substantial challenges.

3.9

The ANAO reported that administrative processes could have been better focused at the planning stage on comprehensive risk assessment, as well as in designing programs with more measurable objectives and targets. The absence of these factors made it difficult to measure results against program objectives and exposed some programs to risks that could have been better identified and treated in the early stages of the programs.

3.10

The ANAO found that catch-up administrative improvements had been put in place to overcome initial shortcomings in planning. Project appraisal and selection was generally rigorous and based on merit. AGO had put in place sound and well drafted agreements to manage residual risk at the program level. Monitoring and evaluation were also given sufficient priority. Linking payments to milestones assisted in the efficient management of funding allocations.

3.11

Nevertheless, ANAO concluded that substantial risks remained—particularly in terms of the timely achievement of program objectives. Areas for further improvement included refining performance measurement, including the use of intermediate measures and/or assessments to gauge progress towards longer term objectives. The ANAO also found that a more consistent approach to project appraisal and selection would also assist in improving the transparency of decision-making. Attention also needed to be given to the timeframes of negotiations over funding agreements. Finally, the ANAO reported that improvements to performance reporting were necessary to enable Parliament to come to a more informed view on the progress and effectiveness of AGO in implementing programs of national significance.

ANAO recommendations

3.12

The ANAO made five recommendations:

 

Table 3.2 ANAO recommendations, Audit Report no. 34, 2003-04

1.

In order to maximise value for money from grant expenditure and minimise the potential for any adverse impacts on program effectiveness, the ANAO recommends that, prior to consideration of any future funding assistance programs, the AGO conduct a comprehensive program risk assessment. If this timing is not achievable in practice, then the ANAO recommends it should occur as early as possible and certainly, before the commitment of any substantial resources.

 

2.

In order to assist in measuring and/or assessing program results, the ANAO recommends that prior to implementation of any future funding assistance programs, the AGO consider incorporating clearly defined and measurable intermediate outcomes and operational targets (where possible) to underpin program objectives.

 

3.

In order to improve the measurement and the consistency of performance reporting across programs, the ANAO recommends that the AGO give high priority to the completion of an integrated performance information system for measurement of greenhouse gas abatement.

 

4.

In order to improve the rigour and transparency of the appraisal and selection process, the ANAO recommends that the AGO seek Ministerial approval to apply, where appropriate, across competitive programs:

5.

In order to enhance public reporting through the use of performance information to improve the quality and consistency of reports, the ANAO recommends that AGO annual reports include:

 

3.13

The AGO agreed with each of the recommendations and reported to the Committee on its progress in meeting each recommendation.

The Committee’s review

3.14

The Committee held a public hearing on 14 February 2005 to review the progress made against the audit’s recommendations. Witnesses from the ANAO, AGO and Commonwealth Scientific & Industrial Research Organisation (CSIRO) attended the hearing.

3.15

The Committee took evidence on the following major issues:

Risk assessment

3.16

The ANAO’s publication Administration of Grants Better Practice Guide notes that effective planning is the cornerstone of an economic, efficient and effective program. The ANAO’s review of the 1997 and 1999 greenhouse funding packages found ‘substantial shortcomings in the initial risk assessments’.3 While this lack of risk assessment did not have significant ‘downstream’ implications for most of the programs examined by the ANAO, it had important consequences for two programs – the Photovoltaic Rebate Program (PVRP) and the Alternative Fuels Conservation Program (AFCP).

Photovoltaic Rebate Program

3.17

Photovoltaic systems convert sunlight into electricity. Under the Photovoltaic Rebate Program (PVRP), domestic householders and small community organisations could apply for a rebate of $8,250 against the costs of installation of a photovoltaic system.4

3.18

The ANAO found that the announcement of the PVRP created an immediate expectation in the market for the pending subsidy. However, the AGO was not ready to distribute subsidies, because appropriations and delivery agreements were not in place. As a result, there was an initial slump in sales of photovoltaic units prior to the introduction of the program.5

3.19

When the program became operational in January 2000, the pent up demand then exceeded supply. The AGO ran out of funds for the PVRP program, and was forced to reallocate resources from another greenhouse program to cover liabilities. This was undertaken with Ministerial approval. In order to slow down demand, the rebate amount was also reduced to $4 per watt, and a cap of $4,000 per domestic household was put on the rebate.6

3.20

The ANAO reported that the AGO had responded to the program challenges within a reasonable timeframe. However, ANAO stated:

…these findings highlight the importance of undertaking an early risk assessment. In this case, a risk assessment may have assisted in identifying the level for demand for the program and the need for strengthened controls on expenditure.7

3.21

The Committee notes that while the PVRP program was due to be phased out, the 2005-06 budget provided for an extra two years of PVRP, at a cost of $5.7 million per year. The extra funding is intended to provide a ‘bridge’ between the PVRP program and the new Solar Cities program announced in the Energy White Paper, which is due to commence in June 2006.8 The bridging program will help to allay solar energy industry fears of a consumer slump leading up to the introduction of the Solar Cities program, as was experienced prior to the introduction of the PVRP in 1999/2000.

Alternative Fuel Conversion Program

3.22

The ANAO also found that poor risk assessment affected the operation of the Alternative Fuel Conversion Program (AFCP). Under its original design, the AFCP aimed to reduce greenhouse gases and improve urban air quality by facilitating heavier commercial road vehicles and public transport buses to operate compressed natural gas (CNG) and liquefied petroleum gas (LPG). The program was initially funded for $75 million from 2000 – 2008, and was intended to provide a rebate of up to 50 per cent of the cost price of converting an existing commercial vehicle to CNG or LPG, or upgrading to a new CNG or LPG engine.

3.23

However, the ANAO found that although the AGO had consulted with industry in designing the scheme, the assessment proved to be optimistic and unrealistic, given the low uptake of available grants.

3.24

A key factor in the low uptake (particularly for CNG conversions or upgrades) was an absence of refuelling infrastructure to support CNG, and low consumer and industry confidence in CNG and LPG for heavy vehicles.

3.25

The AGO undertook a review of the program in November 2001, 15 months after the program commenced. This review was praised by the ANAO. However the ANAO did conclude that had AGO undertaken a more thorough risk assessment in the early stages of the program, some of the later issues could have been avoided:

In particular, the need to have a robust consultation and analysis to test the accuracy of sectoral industry advice is a key lesson learned from this program.9

3.26

The AGO stated that its review of the AFCP program had resulted in better targeting of the available funds. AGO reported that the redesigned AFCP program:

…focuses on pilots with significant fleet operators – for example, PO, Coates, Murray Goulburn and a couple more. We have agreements with those companies to actually work with them to develop a business case and to pilot the technology to facilitate import of engines, and the revamped program is actually working very well.

So we actually consider AFCP to be case of best practice where we monitored, we identified slow uptake, we undertook a review, we put the findings of the review and proposed redesign to our minister and that was then taken to the budget and the program was redesigned. We are now effectively progressing the development of those technologies.10

AGO response

3.27

In response to the ANAO’s recommendation regarding risk assessment, the AGO developed a Risk Management Policy. According to the AGO’s 2003-04 Annual Report, the Risk Management Policy recognises that risk management is a key component of effective corporate governance. In response to Committee questioning on risk assessment, the AGO advised that it has conducted comprehensive program risk assessment for all new climate change measures announced in the 2004-05 Federal Budget and the Australian Government’s energy white paper, Securing Australia’s Energy Future.11

Measuring outcomes

3.28

As noted by the ANAO, measurable and precise objectives provide a solid foundation for effective performance management and accountability.12 The ANAO found that the key objectives for most AGO programs were broad and not easily measurable. However, there were several good examples, such as the Greenhouse Gas Abatement Program (G Gap).

3.29

The G Gap program aims to reduce Australia’s net greenhouse gas emissions by supporting activities that are likely to result in substantial emissions reductions or activities to offset greenhouse emissions, particularly in the period 2008-2012. The G Gap program provides assistance to private sector projects which will lead to greenhouse savings or offsets.13 At the time of the ANAO’s audit, two rounds of the G Gap program had been completed, with a third and final round now nearing finalisation (at April 2005).14

3.30

The ANAO praised the G Gap program for including operational targets which underpinned broad objectives. ANAO commented that this was good practice and particularly useful in providing an indication of progress towards objectives.15

3.31

Other programs with operational targets included Greenhouse Challenge and the AFPC. However, for other programs, ANAO found that it was difficult to gauge to what extent programs could be achieved, particularly where there were long lead times involved. For example, for the PVRP there was no measurement of:

3.32

The AGO agreed with the ANAO’s recommendation to incorporate clearly defined and measurable intermediate outcomes and operational targets, where possible. However, in the ANAO’s report and also before the Committee, the AGO noted the difficulties in doing this:

For a large number of these projects – for example, with the development of renewable energy technologies – it is not possible to measure any greenhouse outcome or any other interim measure except where a milestone, be it the delivery of a boiler or the construction of a component of plant, has actually been met. It is not until the commissioning or the end of a project when different technologies are linked together that you can actually start to measure the greenhouse outcomes.16

3.33

However, AGO reported that the ANAO’s recommendation had been taken into account in measures announced in the 2003-05 budget, and in measures in the Energy White Paper.17

Selection and appraisal of projects

3.34

Appraisal and selection processes for grants must be transparent and lead to the selection of projects which represent value for money against program objectives and outcomes. The ANAO praised the AGO’s appraisal system for the programs covered by the audit. ANAO found that the AGO brought in specialist advice where needed.18

3.35

The ANAO noted one problem in project appraisals, where a grant of nearly $1 million was awarded for a project that was near completion. Although the funding was not critical to the implementation of the project, AGO advised the Minister that the grant should still go ahead, as there was nothing in the program guidelines to preclude funding for projects which are already fully committed.19 The AGO re-worded its guidelines for subsequent funding rounds.

3.36

The ANAO stated that the appraisal process for all future projects should contain a criteria related to a project’s need for financial assistance to proceed. If there is no need for financial assistance, the ANAO argued,

…funds paid in such circumstances provide no added value and represent an opportunity cost to the Australian Government.20

3.37

The Committee agrees with the ANAO’s argument and calls for the AGO and other agencies to ensure that they support only those projects which genuinely require government assistance to proceed.

3.38

The ANAO found a problem in documentation of decisions in selecting projects for the Greenhouse Gas Abatement Program (GGAP). While the AGO grouped projects into risk categories when presenting them to the Minister for decision, unlike other programs, the AGO provided no recommendations for Ministerial decision makers. This was specifically because of a Ministerial request to not provide recommendations on GGAP projects.21

3.39

The ANAO found that Ministerial decisions for G Gap funding allocations could have been better documented. Greater detail in the reasons for decisions would have assisted in explaining why some projects received funding given the level of risks raised in the appraisal brief. The ANAO recommended that the AGO apply an order of merit rating scheme for projects, and recommendations on selection that highlight projects that are most likely to achieve project objectives.

3.40

The Committee asked the AGO whether it knew the reasons why the Ministerial Council had requested that selection briefs for GGAP programs not provide recommendations on projects or a suggested order of merit. The AGO replied that the minutes of the Ministerial Council meeting at which decision-making procedures were set down (16 February 2000) do not record an explanation for the Council’s decision.22

3.41

The AGO advised the Committee that the ANAO’s recommendation, regarding advice to Ministers on proposed project funding, was undertaken for the third and final round of the GGAP program.23

3.42

The AGO also advised that, at the time of the Committee’s hearing, the guidelines for administration of the new greenhouse abatement programs arising out of the Energy White Paper were still being signed off by Ministers. The Committee would like to ensure that the guidelines for these programs reflect the ANAO’s recommendations on assessment and selection procedures for project funding.

3.43

Recommendation 4

The Committee recommends that the Australian Greenhouse Office provide to the Committee a copy of the guidelines for administration of the new greenhouse abatement programs announced in the Energy White Paper and the 2004-05 Federal Budget; and an outline of how the new guidelines meet the ANAO’s recommendations regarding assessment and selection procedures for project funding.

Management and monitoring of agreements

3.44

The ANAO found that, in general, the agreements between the AGO and program participants were well designed and appropriate for the specific design of each program. However the ANAO noted a number of specific examples where agreements could be strengthened. These included:

3.45

The Committee believes the ANAO’s points are important to ensure the effective administration of project funds and that taxpayers are receiving maximum value for money. Accordingly, the Committee makes the following recommendation:

3.46

Recommendation 5

The Committee recommends that the Australian Greenhouse Office include the following elements in all future agreements for project funding:

3.47

The ANAO analysed the performance monitoring of each AGO program included in its audit. The ANAO found that ‘performance monitoring to date has been thorough and given the necessary priority. It is too early to tell if programs such as GGAP, RECP, REEF will achieve their objectives in the longer term.’27

Performance reporting

3.48

The ANAO found that there was room for improvement in the AGO’s annual reporting. The AGO’s annual reports were largely focused on inputs and activities, rather than reporting against targets for programs and reporting of trends and changes over time. For example, the AFCP had a target of 800 buses and 4,000 commercial trucks to be converted to CNG or LPG gas in each of the four years of the program. The AGO’s 2002-03 annual report stated that 568 buses had converted or been purchased, and that this represented a 150 per cent increase since the program’s inception. However, the AGO did not report against the original target of 800, which ANAO felt ‘could give the reader a misleading impression about the achievements of the program’. The actual performance on the number of trucks converted (or purchased) is not reported.28

3.49

The ANAO also found that the AGO’s annual reports had a lack of year-by-year comparisons on performance. The ANAO felt these comparisons were particularly important given the long lead times involved in achieving results of between three and 13 years.

3.50

The ANAO noted that the suite of programs managed by the AGO have a target of abating 67 Mt of CO2 each year. The ANAO would like the AGO’s annual report to include a progress report towards the target from all the programs funded by AGO.

3.51

The AGO agreed with the ANAO’s recommendation to enhance public reporting, and told the Committee that its 2003-04 annual report included the following improvements:

3.52

At the hearing, the AGO also told the Committee that it had improved its performance reporting via new performance standards in the 2004-05 Portfolio Budget Statements (PBS), which will be reported against in the 2005-06 Department of Environment and Heritage annual report. For example, the AGO’s OutPut Group 2 – Emissions Management, includes the Renewable Remote Power Generation Programme, Photovoltaic Rebate Programme, Renewable Energy Commercialisation Programme, Greenhouse Gas Abatement Programme and others which were not reviewed by the ANAO’s audit. These programs will now be subject to performance reporting against criteria such as those outlined in the table below.

 

Table 3.3 Australian Greenhouse Office Portfolio Budget Statement 2004-05 – Emissions Management

AGO Output Group 1.2 – Emissions Management

Objective: To lead action on Australia’s commitment to the 108% target and a lower greenhouse signature in the longer term

QUALITY

Reporting systems are appropriately targeted and high quality

 

Implementation of consistent measurement of abatement across programmes

 

Risks to programme delivery identified and managed

 

Effectiveness of support for greenhouse response within sectors

 

Measurable behaviour change within sectors

 

Effectiveness in relevant interjurisdictional processes on issues for which the AGO has lead responsibility

 

Effectiveness of financial administration

 

 

QUANTITY

Reported abatement activity including emissions reductions and/or energy savings

 

Estimated cost (Government funds) of greenhouse abatement ($ per tonne)

 

Investment dollars (or contributory funding) leveraged from other parties by projects and programmes

 

Extent of engagement of key stakeholders

 

Extent of support for long-term low emission technology uptake.

Source Australian Greenhouse Office, Portfolio Budget Statement 2004-05, pp. 188 – 189.

3.53

The Committee looks forward to more detailed performance reporting from the AGO in the 2005-06 Annual Report from DEH.



Footnotes

1

Australian Greenhouse Office, internet site: http://www.greenhouse.gov.au/ago/index.html, accessed May 2005. Back

2

Australian Greenhouse Office, internet site: http://www.greenhouse.gov.au/ago/index.html, accessed May 2005. Back

3 ANAO Audit Report no. 34, 2003-04, The Administration of Major Programs (Australian Greenhouse Office), Commonwealth of Australia, March 2004, p. 32. Back
4

Minister for Environment and Heritage, Media Release 8 February 2000, available at: http://www.deh.gov.au/minister/env/2000/mr8feb00.html, accessed May 2005. Back

5

ANAO Audit Report no. 34, 2003-04, p. 33. Back

6 ANAO Audit Report no. 34, 2003-04, p. 34. Back
7

ANAO Audit Report no. 34, 2003-04, p. 34. Back

8 Department of Environment and Heritage, Portfolio Budget Statements 2005-06, Fact Sheet, available at: http://www.deh.gov.au/about/budget/2005/factsheet-solar.html, accessed May 2005. Back
9

ANAO Audit Report 34, 2003-04, p. 35. Back

10

Australian Greenhouse Office (AGO), Transcript of Evidence, 14 February 2005, p. 6. Back

11

Australian Greenhouse Office, Annual Report 2003-04, pp. 66-67; and Australian Greenhouse Office, Submission no. 5.1, p. 1. Back

12

ANAO Audit Report no. 34, 2003-04, p. 36. Back

13

Australian Greenhouse Office, Annual Report 2003-04, p. 24. Back

14 Australian Greenhouse Office Annual Report 2003-04, p. 25. Back
15

ANAO Audit Report no. 34, 2003-04, p. 37. Back

16

AGO, Transcript of Evidence, 14 February 2005, p. 4. Back

17

AGO, Submission no. 5.1, p. 1. Back

18

ANAO Audit Report no. 34, 2003-04, p. 50. Back

19

ANAO Audit Report no. 34, 2003-04, p. 50. Back

20

ANAO Audit Report no. 34, 2003-04, p. 50. Back

21

ANAO Audit Report no. 34, 2003-04, p. 55. Back

22

AGO, Submission no. 5.1, p. 3. Back

23

AGO, Submission no. 5.1, p. 2. Back

24 ANAO Audit Report no. 34, 2003-04, p. 60. Back
25 ANAO Audit Report no. 34, 2003-04, p. 61. Back
26 ANAO Audit Report no. 34, 2003-04, p. 65. Back
27

ANAO Audit Report no. 34, 2003-04, p. 78. Back

28

ANAO Audit Report no. 34, 2003-04, p. 81. Back

29 AGO, Submission no. 5.1, p. 3. Back

Print Chapter 3 (PDF 169KB) < - Report Home < - Chapter 2 : Chapter 4 - >