CHAPTER 3 – THE OUTCOMES AND OUTPUTS FRAMEWORK

THE FORMAT OF THE PORTFOLIO BUDGET STATEMENTS - SECOND REPORT
TABLE OF CONTENTS

CHAPTER 3 – THE OUTCOMES AND OUTPUTS FRAMEWORK

Introduction

In 1985 with the introduction of Program Management and Budgeting (PMB), the first steps were taken to move the focus of attention of parliamentary scrutineers from inputs to results of expenditure. The committee's predecessor reported twice in that year on the change in the first agencies to trial PMB, and was cautiously optimistic, noting however the need for reconciliations between the Appropriation Bills and the programs. [1]

When PMB was well established and its intricacies somewhat understood by agencies and senators alike, the explanatory documentation served more or less adequately as budget scrutiny documents and agendas for estimates hearings. `Programs' or `sub-programs' often equated to agencies (such as the Office of the Commonwealth Ombudsman, the Navy, the Federal Court or the National Gallery of Australia ) or distinct chunks of departmental activity (such as `tourism' or `consumer affairs'). What PMB did not provide was a full price for these activities, encompassing corporate overheads, allowances for superannuation liabilities, depreciation and so forth. Nor did it disclose in a transparent fashion what might be termed `sensitive' expenditure, which could be, and frequently was, buried undisclosed in a broader category of expenditure or moved from agency to agency. And while it could not be said that the format precluded an examination of the outcomes of expenditure, as any reading of the interchanges, for example, between former Senator Peter Baume and officers of the then Department of Health on the subject of the then government's drug offensive will show, the committee accepts that PMB did not lend itself particularly well to such an examination.

So a new outcomes and outputs framework has been introduced, with the aim of sharpening the focus on the use of government resources and the outputs produced. Instead of programs, sub-programs and components, we have moved, broadly speaking, to outcomes, outputs, or output groups and outputs.

Key components of the new framework are as follows:

Advisory documentation from DOFA indicated that no set number of outcomes were required but that ministers and their agencies could select the number and level of outcomes that were most relevant and practical for their management and reporting needs. Similarly, outputs were to be defined in a way that was most meaningful for key stakeholders. [3] This approach recognises what the committee and others have pointed out over the years, that one size does not necessarily fit all. It further recognises the reality of the devolution of control from central agencies to individual agency CEOs under the Financial Management and Accountability Act 1997. But the results have nevertheless been quite dramatically different from portfolio to portfolio and it appears that not all PBS structures meet the needs – or preferences - of some parliamentary stakeholders.

In the remainder of this chapter the committee will consider separately outcomes, outputs, administered items, the overall outcomes/outputs frameworks, how they relate to organisational structures, how stable they might be, and then will draw certain conclusions from the evidence in the PBS to date.

Outcomes

The first issue to be determined by portfolios for the new reporting framework was the selection of outcomes. Outcomes are defined by DOFA as `the results, impacts and consequences of actions by the Commonwealth on the Australian community'. [4] They are ministerially endorsed and funding via the Appropriation Bills is to outcomes. Agencies must report in their annual reports on actual outcomes against the desired outcomes specified in the PBS.

From the committee's point of view, examples of good, clear outcomes from the 1999-2000 PBS included the following:

These outcomes share a number of desirable characteristics. They are aspirational; they are likely to have wide general support; they are succinct; they do not attempt to explain how the outcome is to be achieved; and they are not hedged about with terms and conditions. The committee accepts, however, that there is scope for both narrow, short-term outcomes as well as high-level aspirational ones.

It has been argued that many of the `outcomes' formulated for these first outcomes/outputs reporting frameworks could as realistically be termed `objectives'. Some are very general indeed and it is hard to see how accountability could possibly be enhanced in any reporting against them. Also, the limitations of the portfolio structure ensure that there are no cross-portfolio outcomes. Communications, health and transport activities directed to regional Australia, for example, are all dealt with separately in the individual PBS. Significant cross-portfolio issues are drawn together in ministerial statements, but translating from those statements to the PBS has caused problems. [5]

No set number of outcomes per portfolio or per department or agency was required in the new framework. By the committee's calculation, outcome numbers in fact ranged from one to ten, and averaged at three for departments. The level of appropriation they represented varied per outcome from a high of $17 509 082 000 for the single outcome for Defence, `The prevention or defeat of armed force against Australia or its interests' to a low of $271 000 for the single outcome of the Criminology Research Council, `Criminological research which informs the Commonwealth and the states'. Sixteen outcomes enjoyed appropriations of over $1 000 000 000, while five had appropriations of less than one million dollars. The committee is not necessarily convinced that materiality is of paramount importance in outcome definition, but it does note the extreme range and will consider the matter further in subsequent reports.

As noted previously, outcomes could be formulated at portfolio level or at departmental or agency level. Portfolio structures, themselves sometimes accidents of history, play a part in outcomes gestation. The Finance and Administration portfolio, comprising inter alia such strange bedfellows as the Office of Asset Sales and IT Outsourcing, the Australian Electoral Commission and Commonwealth Superannuation Administration as well as the department, wisely elected not to devise a single portfolio outcome. By contrast, the Agriculture, Fisheries and Forestry portfolio deemed that all of its general government sector agencies actually contributed to a generic outcome, `more sustainable, competitive and profitable Australian agricultural, food, fisheries and forestry industries'. The Health and Aged Care portfolio developed ten outcomes, shared by the department and all portfolio agencies. On the other hand, many of the agencies within the Attorney-General's portfolio could be said to contribute to one or other of the departmental outcomes, and yet they have discrete individual outcomes.

As mentioned above, and as a cursory glance at the outcomes defined in the 1999-2000 PBS will show, they are for the most part not likely to be `achieved' in the short term. They represent continua along which progress can be charted in successive PBS and annual reports. In many cases, public sector agencies will have little to no control over some of the factors which might affect the outcome. Agencies should be able to define and measure a number of practical contributions to those outcomes, however, in the form of outputs.

Outputs

Outputs are defined as `the goods and services produced by agencies on behalf of government for external organisations or individuals ... Agency outputs ... are funded by government, throught the Budget process, on the basis of agreed output prices. Outputs are then produced using departmental expenses.' They are to be clearly linked to intended outcomes. And they are distinct from administered items, such as benefit payments, grants to other levels of government and transfer payments, which are managed but not controlled by agencies.

The success or failure of an outcomes/outputs reporting framework depends in part on the choice of outputs and whether as a whole they contribute to an adequate measure of the outcome to which they relate. There is a risk, as was pointed out to the committee, that only measurable outputs will be defined and reported on.

Depending on circumstances, outputs can be stand-alone or grouped for management and reporting purposes. DOFA-provided guidance suggested that, in deciding whether to group outputs, issues to consider included the homogeneity of outputs within a group, the adequacy of the information for performance monitoring, materiality, measurability and accountability. [6] In describing his department's choice of five output groups, Peter Shergold of the Department of Employment, Workplace Relations and Small Business stressed that the aim had been to allow clear distinctions to be made between key roles and functions of the department. [7]

By the committee's rough calculation, slightly more than half the outcomes were supported by direct outputs rather than output groups. Direct outputs per outcome ranged from one to eleven; output groups per outcome ranged from one to nine; while the greatest number of outputs (via output groups) per outcome was 58, for outcome 1 of the Department of Environment and Heritage, `The environment, especially those aspects that are matters of national environmental significance, is protected and conserved'. Interestingly, the Environment and Heritage portfolio was commended in estimates hearings for its detailed approach. [8] There was little to no correlation between the price of outputs, where specified, and breadth or narrowness of output definition.

Steve Gurr of the Victorian Treasury described that state's experience in output definition:

In its report on the Victorian 1998-99 budget estimates, that Parliament's Public Accounts and Estimates Committee reflected the process of consolidation of outputs across most departments and concluded there was still room for improvement in output specifications to ensure that they reflected appropriate and meaningful outputs. [10]

At a federal level, the 1999-2000 PBS revealed some outputs which were very highly aggregated and others which were quite specific. Sometimes the choice for agencies appeared logical and straightforward; in other cases, agencies went to some lengths to cater for the known predelictions of scrutinising senators. As Martin Dolan of AFFA explained,

Administered items

For financial management purposes, a distinction has been drawn between `departmental' items, which an agency controls, such as salaries and allowances, and `administered' items, such as benefit payments and grants to other levels of government, which the agency administers but does not control. Accountability for admininistered items differs as agencies have a more limited influence over them.

The definition of control is crucial to the distinction between administered and departmental items, but it has proved difficult for agencies to apply and even more difficult for senators to understand. Because the end result affects the nature of an agency's responsibility for the item in question, the reasons for the judgment one way or another must be robust and transparent.

The estimates hearings illustrated many cases in which senators did not appreciate the distinction between departmental and administered items, and the resultant accountability consequences. Workplace reform secret ballots and the Tasmanian Freight Equalisation Scheme were but two cases of confusion. The committee believes that with time, the distinction will become clearer and no particular additional measures need to be taken in the PBS to accentuate the distinction. If, however, decisions are taken to redefine a given item, its changed status should be highlighted in the overview.

Outcomes/outputs framework

In the move to an outcomes and outputs reporting framework, no particular outline was mandated. This is consistent with the devolution of responsibility for such matters to agency heads, but has inevitably resulted in a most inconsistent picture when viewed across the board. While broad guidelines were provided by DOFA, portfolios were given the freedom to chose a framework which suited their particular circumstances and many acknowledged to the committee their appreciation of this flexibility. [12] Whatever the merits of an outcomes/outputs framework in clarifying what should be produced, at what price and how efficiently – and the committee considers these and other issues associated with the new framework in the following chapter – the basic issue for senators using the PBS in estimates hearings remains whether the framework is meaningful to them. And just as the program structures before them were criticised, so too were the initial outcomes/outputs frameworks.

Portfolio or agency outcomes?

The first obvious difference in the chosen frameworks was the selection of portfolio versus agency outcomes. As indicated previously, some portfolios – Agriculture, Fisheries and Forestry, Health and Aged Care, Treasury, and Transport and Regional Services – had one or more portfolio outcomes, while the remainder chose to define their outcomes at departmental and agency level. One agency – ATSIC – developed sub-outcomes. Portfolio outcomes were always an option [13] and their use is not of itself a problem but it does constitute one more nail in the coffin of PBS consistency.

The Transport and Regional Services portfolio's rationale for a single portfolio outcome was as follows:

The Health and Aged Care portfolio adopted ten across-portfolio outcomes. This approach was explained by the departmental secretary, Andrew Podger, as `a best attempt at classifying our activities according to objectives' and one which was an improvement on the previous arrangement. [15] But did the Senate scrutiny committee think so? The Community Affairs Legislation Committee, which examined the estimates of two portfolios – Health and Aged Care, and Family and Community Services - with quite differing approaches to structuring outcomes and outputs, was critical of the differing formats. [16] Apart from its 10-outcome structure for the whole portfolio, Health and Aged Care adopted seven categories of output for use across all outcomes where applicable, while the Family and Community Services portfolio produced separate outcomes for Centrelink and the Australian Institute of Family Studies and integrated outcomes for the Department, the Child Support Agency, the Social Security Appeals Tribunal and CRS Australia. Officers representing the Department of Family and Community Services at the committee's public hearing explained that, as a new combination of agencies following the changes to the administrative arrangements in October 1998, they had to define from scratch what they wanted to achieve, then determine their outcomes/outputs structure and organise the department around it. [17]

Level of specificity of outcomes/outputs

In its submission to the committee, DIMA described the development of its broad outcomes and outputs framework, of two outcomes and seven outputs, noting however that many other agencies had taken a much more specific and detailed approach, even identifying policy advice and ministerial services as separate outputs. The committee agrees with DIMA that these types of services are, generally, processes in delivering outputs and not of themselves outputs. It is not unduly concerned at the lack of consistency in level of specificity in the outcomes/outputs frameworks devised. Over time, and with experience, it is probable that a greater level of consistency will evolve without being mandated. Even if it does not, the committee takes the view that portfolios have the right to be flexible in defining their outcomes and outputs, provided that they remain responsive in estimates hearings to requests for differing levels of aggregations.

This committee has wrestled with the issue of whether the reporting framwork really matters a great deal per se and whether, with more familiarity with it, we and our senatorial colleagues might find the present variety less problematic. A number of not necessarily distinct issues are involved here: ministerial preference; the internal diversity of each portfolio; the stability of the administrative arrangements; and even senatorial notions of accountability. Does the outcome structure matter? After all, it merely replaces a program structure which was variously divided on agency or on thematic lines, and at differing reporting levels. And just as the previous variations irritated some scrutinising senators, [18] so did the new, aided and abetted by `the shock of the new'.

Failure to disclose discrete entities

Where the framework clearly does matter to scrutinising senators is where the output structure is framed so as to obscure the impact, or even existence, of quite discrete entities. The exchanges in this committee's estimates hearings over the Office of the Status of Women (OSW) are a case in point. To Senator Crossin's comment, `I have to tell you that I feel like I was playing “Where's Wally”. I could not actually find the words OSW', the head of the OSW replied that the department had made the decision a couple of years ago to treat its financial statements by group and that the OSW was part of the social policy group. [19] To the committee's mind, the lack of homogeneity of that group, which also encompasses the Office of Indigeneous Policy, is striking and should perhaps be avoided or at the very least, specific performance indicators should be produced.

Senator Forshaw also articulated this problem with the Agriculture, Fisheries and Forestry PBS:

To which the officers responded by describing the change of philosophy in a budget that focuses on outcomes and outputs. The committee discusses the question of output pricing, and cost allocation to outputs, in Chapter 4.

In the committee's view, the individual contributions of discrete administrative bodies such as AQIS, whether or not they contribute to a more aggregated output, should at the very least be disclosed by way of performance indicators. To argue that this would subvert the change in emphasis to outcomes is unhelpful, particularly for statutory bodies which are creatures of Parliament and in which Parliament can be reasonably expected to wish to take a close and continuing interest.

Enabling services as separate outputs?

Several senators were puzzled by the absence of what used to be termed `corporate services' in the new structures. The logic of appropriations to outcomes dictates that enabling services such as personnel management and IT provision are attributed to outputs which in turn contribute to outcomes. The enabling services are not an end in themselves and, in the committee's view, need not be separately identifiable in the PBS. This is not to say, however, that agencies should not be responsive to questioning about enabling services in estimates, merely that information about them in the PBS could be at the discretion of agencies.

Relationship of the outcomes/outputs framework with the organisational structure

Experience in other jurisdictions has shown that where the outcomes and outputs structure reflects the organisational structure, the system works more smoothly. Steve Gurr, representing the Victorian Treasury, indicated that in that state, they were working with departments to encourage them to better reflect the the output structure in their organisational structure, so that management decision-making could be aligned with outputs. [21]

The committee has noted how the Finance and Administration portfolio, having of course been in the best position to see the changes coming, revised its own organisational structure in line with its present outcomes and outputs well in advance of the first accrual budget so that it had an apparently relatively seamless transition to the new framework. As Steve Bartos of DOFA explained,

The concept of outputs or sub-outputs to which contributions come from many workgroups or even agencies came as a surprise to many senators. In considering the Department of Employment, Workplace Relations and Small Business output 2.1.1, policy advice and advocacy, for example, the following exchange took place:

Similarly in the estimates for the Transport and Regional Services portfolio, Senator Mackay, noting that the outputs in the PBS did not reflect the departmental structure, was told:

The committee believes that where possible – and it concedes that it is sometimes neither possible nor desirable – the preferred framework should reflect the organisational structure of the agency. It is of the view that where agency outputs do not reflect the organisational structure, a clear description in the PBS of the method used to allocate costs to outputs is required.

Stability of the outcomes/outputs structure

In estimates hearings, in the committee's public hearing and in submissions to the committee, one theme which emerged loudly and clearly was the issue of the stability of the present outputs and outcomes structure. Mr Gurr described the Victorian experience:

The Foreign Affairs, Defence and Trade Legislation Committee explored this question in some detail in its examination of the estimates and concluded that it would be concerned if major changes were to take place as `it would make it difficult to keep track of expenditure from year to year'. [26] Joanna Hewitt of the Department of Foreign Affairs and Trade assured the committee that that department was hoping that the present structure would give it a stable platform from which to work and report over time and one which would give meaningful quantitative data over time. She added that organisational change was being considered to align parts of the department that contribute to a particular output. [27]

Greg Harper presented a different picture from the perspective of the Transport and Regional Services portfolio, which had developed its PBS on an outputs basis for 1998-99 as well. In that year, there were nine output classes, organisationally based and Mr Harper acknowledged that `we may have shaped the outputs which we were to deliver to the organisation which then existed'. [28] However, following the administrative arrangement changes, they had recast their output structure along broad functional lines.

Other agencies indicated that they were still engaged in bedding down their outcomes/outputs structure following the experience of the first year. The Australian Federal Police, with five outcomes, 19 output groups and about 40 outputs, was considering rationalising for 2000-2001; the Australian Customs Service in contrast deliberately set out to have a simpler structure, with one outcome and five outputs, but with the view that `there would be a period of some years before the output-outcome structure would really shake down'. [29]

This pattern was repeated across many estimates hearings, corroborating the experience reported by the ACT and Victorian representatives that the frameworks and associated issues could take up to five years to stabilise.

Administrative arrangement changes always disrupt reporting frameworks. There is a reasonable expectation on the part of senators that when a function is moved from one portfolio to another, it is slotted transparently into the new framework and at a similar reporting level from whence it came. Should changes be required, including changes to performance information, they should be clearly disclosed in both the former and the receiving PBS and justified in the latter.

Steve Gurr, the director of budget reform in the Victorian Treasury, in describing that state's first accrual budget, commented:

This was clearly evident in Senate estimates hearings as well. A given output would be called upon, a question asked, and the officers at the table often had little idea as to which of their colleagues was actually responsible for the area in question, resulting in a considerable musical chairs exercise while the appropriate person was found. This may be a passing problem and with time and experience, it may be overcome. Or it may require in some cases redefinition of outputs, and another settling down period.

Conclusions

On the evidence available to it, the committee believes that an amount of `bedding down' of the new reporting framework is required before a definitive view on its success or otherwise can be formed. Certainly it has the potential to provide more complete information on what has been achieved and at what cost, and this should assist managers and governments to take more appropriate future action. But unless that information is comparable year on year within the one portfolio and, ideally, comparable for like issues across portfolios, it may well be that key stakeholders such as senators may not be appreciative.

The committee notes that DOFA has indicated its responsiveness to the need for consistency and comparability of the financial reporting framework, and will be taking steps to facilitate the development of best practice in a continuing program of improvement and refinement.

The committee notes and applauds DOFA's stated intention of conducting an analysis of portfolio outcomes/outputs frameworks and recommends that DOFA publish and annually update for as long as necessary a best practice outcomes/outputs frameworks guide on its web site.

Footnotes

[1] See, for example, Senate Standing Committee on Finance and Government Operations, Second Report on Changes in the Presentation of the Appropriations and Departmental Explanatory Notes, 1985.

[2] Department of Finance and Administration, Specifying Outcomes and Outputs, 1998, p. 8.

[3] ibid., p. 4.

[4] ibid., p. 13.

[5] See, for example, the discussion of women's issues in Senate Finance and Public Administration Legislation Committee, Hansard, 2 June 1999, p. 272.

[6] DOFA, Specifying Outcomes and Outputs, 1998, p. 31.

[7] Senate Employment, Workplace Relations, Small Business and Education Legislation Committee, Hansard, 7 June 1999, pp. 4-5.

[8] Senate Finance and Public Administration Legislation Committee, Hansard, 31 May 1999, p. 41.

[9] Senate Finance and Public Administration Legislation Committee, Hansard, 17 June 1999, p. 17.

[10] Victorian Public Accounts and Estimates Committee, Report on the 1998-99 Budget Estimates, Melbourne: Government Printer, 1998, p. x.

[11] Senate Rural and Regional Affairs and Transport Legislation Committee, Hansard, 2 June 1999, p. 247.

[12] See, for example, Department of Immigration and Multicultural Affairs, in Senate Finance and Public Administration Legislation Committee, PBS Submissions, p. 53.

[13] See, for example, DOFA, Specifying Outcomes and Outputs, 1998, p. 13.

[14] Senate Finance and Public Administration Committee, Hansard, 17 June 1999, p. 65.

[15] Senate Community Affairs Legislation Committee, Hansard, 31 May 1999, p. 10.

[16] Senate Community Affairs Legislation Committee, Estimates Report, June 1999, p. 2.

[17] Senate Finance and Public Administration Legislation Committee, Hansard, 17 June 1999, pp. 53, 76.

[18] Senate Finance and Public Administration Legislation Committee, Format of the Portfolio Budget Statements, 1997, p. 16.

[19] Senate Finance and Public Administration Legislation Committee, Hansard, 2 June 1999, p. 68.

[20] Senate Rural and Regional Affairs and Transport Legislation Committee, Hansard, 2 June 1999, p. 245.

[21] Senate Finance and Public Administration Legislation Committee, Hansard, 17 June 1999, p. 20.

[22] Senate Finance and Public Administration Legislation Committee, Hansard, 17 June 1999, p. 77.

[23] Senate Employment, Workplace Relations, Small Business and Education Legislation Committee, Hansard, 7 June 1999, pp. 10-11.

[24] Senate Rural and Regional Affairs and Transport Legislation Committee, Hansard, 31 May 1999, p. 35.

[25] Senate Finance and Public Administration Legislation Committee, Hansard, 17 June 1999, p. 17.

[26] Senate Foreign Affairs, Defence and Trade Legislation Committee, Estimates Report, June 1999, p. 2.

[27] Senate Foreign Affairs, Defence and Trade Legislation Committee, Hansard, 9 June 1999, p. 300.

[28] Senate Finance and Public Administration Legislation Committee, Hansard, 17 June 1999, p. 65.

[29] ibid., p. 73.

[30] Senate Finance and Public Administration Legislation Committee, Hansard, 17 June 1999, p. 15.