Chapter 2 Extension of budget and timeline for the CSIRO Australian Square Kilometre Array Pathfinder (ASKAP) radio telescope project

2.1                   The Australian Square Kilometre Array (SKA) Pathfinder Radio Telescope (ASKAP), Western Australia project was referred to the Committee on 25 June 2008. CSIRO was the proponent agency.[1]

2.2                   The project proposed to establish the world’s most effective survey radio telescope intended for international research in cosmology, transient radio sources, pulsar astronomy and the structure and magnetic field of our galaxy.

2.3                   The works were proposed to be located in Western Australia, primarily at the Murchison Radio-Astronomy Observatory (MRO) at Boolardy Station, with a support facility in Geraldton.[2]

2.4                   This extension of budget and timeline seeks to raise the project budget to $188 million. There is no completion date as funding has not been secured for the entire budget.

Original referral (2008)

2.5                   The scope of the original project was for up to 36 parabolic antennas, each with phased array feed receivers (PAFs) at the MRO site, with associated support facilities at Boolardy Station and in Geraldton. The original project budget was $111 million.[3] However, this costing was only for 30 antennas, not the full scope of 36 antennas.[4]

2.6                   Construction of the project was scheduled to be completed by November 2012.[5]

2.7                   The Committee reported on the project in Report 9/2008, tabled on 1 December 2008. In this report, the Committee accepted the need for the project, the suitability of the scope, and the adequacy of the costings provided. The Committee recommended expediency for the project.[6]

2.8                   At the time, the Committee was satisfied with the evidence provided regarding adequate costings for construction in a remote location, and adequate contingency planning.[7]

Extension of scope (2010)

2.9                   In 2010, CSIRO requested an extension of scope to the ASKAP project and the Pawsey High Performance Computing Centre for SKA Science.

2.10               This extension was related to sustainable energy rather than to the telescope components of the project.[8]

2.11               The Committee held a public hearing on this matter on 24 June 2010.

2.12               The Committee agreed to the extension of scope in Report 3/2010, tabled on 12 July 2010.

2.13               Following this extension of scope, the budget for the ASKAP project increased by $27.5 million, from $111 million to $138.5 million.

Extension of budget and timeline (2012)

2.14               In September 2012, CSIRO notified the Committee of extensions to the project budget and timeline.

2.15               CSIRO provided a private briefing to the Committee in November 2012.

2.16               The Committee conducted public and in-camera hearings with CSIRO on 31 January 2013 in Sydney. The public hearing was advertised on the Committee’s website and in a media release. The transcript of the public hearing is available on the Committee’s website.[9]

2.17               The need for the works remains valid. The scope of the works remains unchanged, and the costs now incorporate the full scope of the project (all 36 antennas). At the time of the public hearing, all 36 antennas had been completed, and four antennas have PAFs.[10]

2.18               Regarding changes to budget, CSIRO presented the remaining works in two phases:

• Phase 1: PAFs on 18 antennas and single pixel feed receivers on the remaining 18 antennas
  
  
• Phase 2: PAFs on all 36 antennas.[11]

2.19               Other project works are included in each of the phases, with the majority of the scope of the original project included in Phase 1.

2.20               The cost of the works above the original project budget of $111 million (and the extension in 2010 to $138.5 million) was presented as follows:

• Phase 1, an additional $25.8 million, to a total of $164.3 million[12]
  
  
• Phase 2, an additional $23.7 million, to a total of $188 million.[13]

2.21               CSIRO advised that funding for Phase 1 has been secured.[14]

2.22               Funding for Phase 2 has yet to be identified and secured.[15] This funding is unlikely to be secured all at once. Phase 2 would not be commenced until some funding is secured.[16]

2.23               Regarding changes to timeline, Phase 1 will be completed by July 2014. The timeline for completion of Phase 2 is dependent on CSIRO’s ability to secure funding.[17]

Project issues

2.24               The key issues relate to project costings:

• construction contingency and cost overruns on the original project
  
  
• no contingency for PAFs in the original project
  
  
• uncertainties surrounding the Phase 2 cost estimate.

2.25               For the public record, CSIRO’s Group Executive for Information Sciences and Chief Finance Officer apologised to the Committee for CSIRO’s costing errors in the project:

… I am always embarrassed to come back for more money.[18]

… there were past deficiencies. In the 2008 submission, clearly we did not recognise, portray, the financial risk. I think when we met previously I used the words, ‘it was imprudent’, and it was; it was imprudent not to come forward with a costing that acknowledged the degree of risk we were going into. We said it then and I am very happy to put it on the record that the CSIRO has examined the processes under which it undertakes conceptual costings, and there is no risk that it would happen again. We are doing everything in our internal processes—we have done so in terms of the double checking that is undertaken, in terms of the discussion with our colleagues from our science side to have them understand more of the risk and therefore the obligation of articulating the risk so that we can put appropriate financial parameters around that risk. We can never probably cover everything all the time, but the points you are making are well made, we do feel them, and we are at the point of needing to come back to the committee for an approach that says, ‘We are outside the budget’. […] we do acknowledge the error of the 2008 costing submission. As I say, internally we have taken steps to ensure that this does not happen in anything, whether or not it comes before the Public Works Committee or whether it is any other type of project that we undertake.[19]

Construction contingency and cost overruns on the original project

2.26               CSIRO stated that the infrastructure costs for the original project (included in Phase 1) came in slightly over budget, but on schedule:

The infrastructure costs which are largely in place came in more or less on budget—just about a million over on quite a substantial amount.[20]

2.27               The contingency for the infrastructure portion of the project was 20 per cent.[21]

2.28               CSIRO stated that the infrastructure contingency and the escalation anticipating cost increases in the initial submission to the Committee in 2008 proved insufficient. CSIRO stated that this was due to unanticipated increases in construction in Western Australia.[22]

2.29               One increase was due to a requirement for a higher standard of camp accommodation. CSIRO attributed this to the mining boom in Western Australia:

In the initial estimates that came through from the quantity surveyor, which is part of us putting together the estimate of the cost for the work, they were a Perth based company that do quantity surveying and they estimated at that point that they saw a camp of a particular style as meeting the WA standards for this sort of work. In the time between that and when we then bid the contract infrastructure out we basically moved into even steeper part of the boom curve and the requirements for the camp. The contractors who were submitting tenders for that infrastructure work all submitted tenders that had a substantially more impressive camp associated with housing their staff. The argument that was put to us was that there was the very real shortage of labour in the Western Australian construction market and to attract the labour that was required to build these facilities you had to provide a competitive camp facility—competitive with the other big construction projects in Western Australia, such as the mining industry.[23]

2.30               CSIRO also underestimated the expense of providing radio-quiet and energy efficient building infrastructure:

… the radio-quiet part is a moderately unique aspect and requirement of the facility. There are very few facilities in the world that require the levels of stringent radio quietness that our buildings do. In fact, the whole reason we are going to Western Australia—to this remote region—is radio telescopes everywhere in the world are now having a real problem with the amount of noise we generate with our day-to-day living. So the requirement for that radio suppression has increased significantly. It has gone beyond what were well-known and well-understood technologies. We have had to work with the contractors to work out how to implement very well-known industrial processes and try to apply them to this unique field.

With the energy efficient component in particular, as well, we were very keen—hence the 2010 submission—to make sure we came up with a design that was extremely energy efficient for that environment so as to lower our long-term carbon footprint, use of diesel fuel and that sort of thing. The fundamental issue there was that the combination of having that energy efficiency with the RFI quiet building was quite complex.[24]

2.31               CSIRO indicated that the requirement for radio-quiet infrastructure was more complex than anticipated, and it took a significant amount of time and effort to finalise this with contractors. There was also a delay between commencing this process and contracts being approved, which resulted in additional costs.[25] Further time delays resulting in cost overruns were evident in other areas of the project.[26]

Committee comment

2.32               The Committee notes that the cost of infrastructure is higher in remote locations and for more technical building requirements. However, these costs are not new or completely unknown. CSIRO should have been more conservative in developing its costings.

2.33               The Committee is of the view that CSIRO should have identified and quantified potential areas of cost increases, and should have included a higher contingency to allow for such situations. If there was a high possibility of unknown factors or potential risks then the project cost estimates should have reflected this.

2.34               Further, the Committee is stunned that the requirements for radio-quiet infrastructure were underestimated. The need for radio-quiet was a key reason for locating the project in outback WA. The Committee expects CSIRO to be able to appropriately cost and deliver its core business requirements.

No contingency for PAFs in the original project

2.35               In the initial project, CSIRO did not apply any contingency for the design and deployment of the PAFs.[27] CSIRO accepted that this was a significant error and that a contingency of up to 50 per cent would have been appropriate.[28]

2.36               Further, CSIRO acknowledged that it had made errors during the development of the cost estimate for the PAFs, and did not adequately assess the project risks:

At the conceptual phase of the project, CSIRO should have been engaging with the committee and indicating that we felt it was prudent to put anything up to a 50 per cent contingency at concept stage on those costings. We should also have been having this discussion which, in some sense, we are now having in hindsight—which is much more about the risk of the design, the risk behind the costings. That discussion which should have been held with the committee did not occur.

… Had we done a contingency between the 50 to 100, […] we would be sitting here within budget. So your point about the inadequacy of the contingency is correct.[29]

2.37               CSIRO indicated that it has addressed these shortcomings by instituting new processes for compliance for other current and future projects.[30]

2.38               CSIRO explained that PAFs were a new technology in 2008:

… our knowledge of these phased array feeds was minimal. No-one had ever built one at all of this design. So we were very much experimenting with coming up with how you would build it, how it would work. So we have learnt a great deal. We have retired a lot of the risk associated with that R&D project.[31]

2.39               CSIRO stated that it is now confident that it can reliably cost the construction and deployment of each of the remaining PAFs:

We are now in a position where the infrastructure is in place, the antennas are in place. The mark 1 phased array feed was designed and has been deployed on four of the antennas to do testing. In that design they found there were two or three deficiencies in that system to achieve the science. One was the frequency range it could operate at and one was the temperature of the system when operating, which probably impacted on the frequency range at the same time. As a result, they redesigned. They went to a redesign of that phased array feed, which is to set a field of view that looks into space, receive the signals and feed them back. They reduced the mass considerably. They reduced the electronics considerably. In the time that has gone on since then, they are now comfortable that the new design works, they know the design and they can proceed with that design for the future antennas.[32]

2.40               Further to this, CSIRO indicated that:

… having now come through the design and development, we do, if you like, have a proven blueprint, a proven design. We have experience and now know who and how to manufacture. We have, therefore, the known costs of what today it takes to develop and install a phased array feed.[33]

2.41               While CSIRO agreed that its costing for the research, design, development and deployment of the PAFs was inadequate, it reminded the Committee that research carries inherent risk, and the result is of an extremely high standard:

… I would say that the design that we have got on the table—this phased array feed—is world leading, full stop. The rest of the world will follow rather than lead on this. Australia really took a big step forward in terms of the technologies that are going to be available to radio astronomy, and took that lead as a conscious decision. With all research programs, the fact that it is called research means there is uncertainty. If you knew the answer it would not be research. In this case, research is not just about doing the new science; the research has been in the technology that you require to provide the capability to do that science. This project has been a research project in its own right, designing these phased array feeds. Australia took the decision to take the lead. I have a feeling that one or two countries were asked to be involved and did not come in. But now that we have got to the root of the problem and got the problem solved, I think you will see the phased array feed being the flagship not just in astronomy but in other fields as well, because the basic design really is fundamentally new and different and world leading.[34]

Committee comment

2.42               The current Committee does not wish to reflect on the deliberations of the previous Committee at the time of the original inquiry. The current Committee recognises that there may have been deficiencies in some information presented to the previous Committee.

2.43               The Committee concedes that there has been less consistency or clarity of costs details in some projects in the past. There have been differences in the quality of information provided to the Committee, particularly prior to the revision of the Committee’s Manual of Practice and Procedure in 2010.

2.44               The Committee is very concerned that the original costings for this project may not have been sufficiently detailed for scrutiny purposes. The Committee found it extraordinary that there was no contingency at all for the research, design, development, installation and testing of PAFs.

2.45               The Committee is of the view that CSIRO should have included a significant contingency on this element of the project, commensurate with the high risk it involved. The possibility or existence of risks does not excuse poor costings.

2.46               Further, significant concerns would be raised if a project was brought forward for scrutiny with a contingency of 50 to 100 per cent for one element of the project.

2.47               Alternatively, CSIRO might have chosen to develop the PAFs to an appropriate level prior to planning and commencing this project. This would have reduced some of the risk to the project. However, the Committee does recognise the constraints that required CSIRO to develop the PAFs and progress the project concurrently.

2.48               The Committee is pleased that CSIRO has instituted new processes for developing cost estimates. However, the Committee cannot fully understand how such a significant risk assessment oversight occurred.

Uncertainties surrounding the Phase 2 cost estimate

2.49               CSIRO stated that it is satisfied with Phase 1 costings, however the estimate for Phase 2 could change, depending on when and how the required funding is secured.

2.50               As CSIRO now has a proven design for the PAFs, it is able to provide an estimate for Phase 2, which includes a 20 per cent contingency:

… based on the fact that we feel we may be putting the remaining 18 on in a piecemeal fashion and obviously we are going to get diseconomies in that process. So it is very much based now on costs that we are much more certain about.[35]

2.51               CSIRO explained why the $23.7 million figure is still only an estimate:

Very broadly, the estimate is, as I indicated, based around the time and in a piecemeal fashion. If you like, we do know the cost of a PAF and how much it takes to build it. But the actual installation and, as my colleague has indicated, getting it on board, getting the positioning correct, and testing also requires a team of people. Should it be that we do not manage to back phase 2 immediately onto the end of phase 1, we are in the situation of potentially standing people down with people having to come back in. Therefore, that could change some of the contracts and things that would go around it. In some sense that is why I have indicated we have put a 20 per cent contingency on that final element, because we are in an unknown area with the timing.[36]

2.52               CSIRO also stated that it was confident that the contingency for Phase 2 was adequate.[37]

Committee comment

2.53               The Committee considers 20 per cent to be a high contingency for a project with known costs and cost certainties. However, the Committee understands that the cost of Phase 2 may change as funding has not been secured.

2.54               The Committee notes that CSIRO will not proceed with Phase 2 unless all or part of the required funding is secured.

Final Committee comment

2.55               In 2008, the Public Works Committee of the 42^nd Parliament made an assessment of the ASKAP project, based on the information it was given at the time.

2.56               The present Committee is disappointed that the costings that CSIRO developed for the original project had significant deficiencies. The Committee is appalled that there was no contingency applied to the PAFs portion of the project.

2.57               The Committee is not satisfied with CSIRO’s statement that infrastructure cost increases could not have been anticipated. The Committee is also disappointed that CSIRO did not adequately estimate the cost of providing radio-quiet infrastructure.

2.58               The Committee acknowledges that the cost estimate for Phase 2 is dependent on funding being secured in the near future. It is unclear if or when CSIRO will secure the necessary funding.

2.59               The Committee reiterates that CSIRO must understand that it is responsible for assessing risks in its projects and incorporating appropriate contingency allowances for those risks.

2.60               It appears that CSIRO employees, particularly scientists, may not fully understand how to identify, assess and quantify project risks. The Committee acknowledges that scientists must be involved in the development of projects, however they should be trained in the appropriate assessment of project risks.

2.61               In this project, CSIRO subject matter experts were attempting to take on project management roles. The Committee has seen this in other inquiries and strongly advises agencies to engage experienced project management experts to deliver projects.

2.62               CSIRO should have balanced the scientific and project management aspects of the works between internal and external individuals and organisations with financial and project management experience. The Committee expects CSIRO to apply this approach to future projects.

2.63               While the Committee accepts that risks are an integral part of research and development projects, this is not an acceptable reason to fail to identify, quantify, mitigate and include contingency for project risks.

2.64               This significant oversight, leads the Committee to question CSIRO’s ability to provide reliable cost estimates for projects. In particular, the Committee is concerned that seemingly obvious factors were not adequately accounted for.

2.65               The Committee acknowledges that CSIRO has implemented new internal processes to prevent such an error from occurring in future projects. The Committee expects future cost estimates to be subject to a rigorous costing process. The Committee also expects future projects to include greater involvement of project management experts.

2.66               With regard to this requested extension of budget and timeline, CSIRO’s written correspondence and submissions lacked clarity and gave greater weight to the scientific aspects of the project than the delivery and cost aspects.

2.67               The Committee expects CSIRO to improve its ability to write concisely, clearly present the most relevant information, and provide an appropriate balance of scientific and project management information.

2.68               The Committee was satisfied that CSIRO answered the Committee’s questions and acknowledged its errors on the public record. However, given the history of the project, the Committee is not satisfied that the remainder of the project will meet the $188 million budget.

2.69               Further, as the Committee was not presented with an initial cost estimate for the full scope of the project (all 36 antennas), it is unable to determine how much of the final budget of $188 million is attributable to the increases in scope and how much is due to cost overruns.

2.70               However, the Committee acknowledges and approves this change of budget and timeline for the project.

2.71               CSIRO should be aware that this extension does not reflect well on its financial reputation and that the Committee will continue to scrutinise the project.

2.72               Accordingly, as the project progresses, the Committee expects CSIRO to provide the following:

• a post-implementation report at the end of Phase 1
  
  
• notification of additional funding as it is secured
  
  
• notification of any changes to the Phase 2 cost estimate or budget as they occur
  
  
• a post-implementation report at the end of Phase 2.