The first edition was produced in October 2024 as a collaboration between the Digital Transformation Agency (DTA) and the University of Sydney based on contemporary events and research findings. The intent of this version is that it is used as guidance and facilitates broader dissemination and feedback. The second edition incorporates input gathered during a consultation period and reflects updated insights and perspectives.
This series engages researchers from academia on issues influencing the performance of digital projects. These projects play a vital role in allowing Australia to seize the opportunities presented by new technologies.
The DTA works across the Australian Government to support the successful design and delivery of digital projects. The work of the DTA includes managing the assurance system which drives good decision-making and seeks to create the conditions each project needs to succeed.
This research series is part of the DTA’s commitment to ensure the Australian Government achieves nothing less than excellence in digital project design and delivery.
Dr Julien Pollack is an Associate Professor with the John Grill Institute for Project Leadership and the School of Project Management at the University of Sydney.
He started as a project manager in IT, organisational change, and manufacturing projects, before moving into research. He now explores multiple aspects of project management, with the broad aim of helping to transform the discipline into one that addresses the needs of complex and uncertain environments.
This includes investigation of project teams and their productivity, project management methodology, and projects with ill-defined objectives and outcomes. His research in these areas is regularly published in the leading international project management journals and research conferences.
Dr Natalie Smith is an Associate Proffessor of practice with the John Grill Institute for Project Leadership. Her expertise is in the governance of digital transformation. Natalie’s PhD was on digital transformation governance, and she has since completed a research fellowship on trust in Artificial Intelligence (AI) in the Australian public sector. Her current research is on designing fit-for-purpose assurance for digital projects.
Natalie is also a non-executive director in not-for-profit Health and Community Service organisations and government. She is a member of the National AI Thinktank for Responsible AI. Previously, Natalie was a partner in Deloitte’s Risk Advisory practice, providing project assurance and supporting organisations delivering digital transformations.
Dr Wei-Ting Hong is a postdoctoral research associate at the John Grill Institute for Project Leadership, the University of Sydney. His research covers digital transformation in project delivery, Natural Language Processing (NLP), prompt engineering and public transport management.
He is particularly interested in how artificial intelligence (AI) can benefit project models, concentrating on the potential of NLP and Large Language Models (LLMs). His PhD was on leveraging the power of NLP to enhance rail safety and organisational learning behaviour. Wei-Ting is also a committee member of Railway Technical Society of Australasia (RTSA), a Technical Society of Engineering Australia (EA).
The Digital Transformation Agency (DTA) engages across the digital project lifecycle to support, advise and coordinate the government’s digital and ICT-enabled investments via the six states of the Investment Oversight Framework (IOF).
State 4 (Assurance) of the IOF aims to provide assurance to government that approved digital projects are on-track to deliver expected outcomes and benefits – including by ensuring projects plan for and implement fit-for-purpose assurance activities to support good decision-making throughout delivery.
The DTA engaged the John Grill Institute for Project Leadership at The University of Sydney to contribute to a research series aimed at eliciting best practice guidance to maximise the rate of digital project success.
As the first topic in this series, this paper and the guidance within supports independent assurers tasked with assurance activities that review and assign Delivery Confidence Assessment (DCA) ratings for digital projects.
The primary purpose is to help improve the consistency and understanding of DCA ratings for digital projects.
Specifically, this publication is intended to:
The primary audience for this guidance consists of:
Assurance research series: 01
Assurance Research Series: 01
Delivery Confidence Assessment (DCA) ratings result from independent assurance activities that agencies conduct to assess the confidence level of the project delivering successfully.
Assurance activities are defined as independent and objective assessments and evaluations undertaken by people and entities separate to the delivery team and the Senior Responsible Officer (SRO). While not every assurance activity will produce a DCA rating, every DCA rating will have an associated assurance activity that informs the rating.
The Assurance Framework for Digital and ICT Investments requires that regular assurance activities are conducted to produce a DCA. These ratings are crucial to provide an indication of an investment’s overall trajectory to deliver on intended outcomes and benefits.
The DTA draws heavily on assurance information to inform and focus its oversight and engagement across the portfolio of in-flight digital and ICT investments.
DCAs form a key input to the DTA’s advice and reporting to government. Information from DCAs helps shape government improvements to the ways in which digital projects are managed. They help to ensure that at-risk projects trigger escalation processes and help to direct remediation planning to at-risk projects so they can be brought back within accepted success criteria.
Consistency in how DCAs are defined is critical to the effectiveness of key decisions by project governance forums and to the DTA’s oversight and advice.
Assurance activities that require the inclusion of a DCA rating in reports provided to the DTA must use the agreed ratings below. They should also instil confidence the DCA rating being awarded is unbiased, rigorous and evidence based.
Successful delivery of the investment to time, cost, quality standards and benefits realisation appears highly likely and there are no major outstanding issues that at this stage appear to threaten delivery significantly
OffSuccessful delivery of the investment to time, cost, quality standards and benefits realisation appears probable however constant attention will be needed to ensure risks do not become major issues threatening delivery.
OffSuccessful delivery of the investment against budget, schedule, scope and benefits, appears feasible but significant issues already exist, requiring management attention. These appear resolvable at this stage and, if addressed promptly, should not present a cost/schedule overrun or loss/delay of benefits.
OffSuccessful delivery of the investment requires urgent action to address major risks or issues in a number of key areas. Changes to budget, schedule, scope or benefits may be necessary if the investment is to be delivered successfully.
Off
Incorporate feedback: Offer users the ability to provide feedback, report issues and suggest service improvements. Promptly act on feedback and provide a timely, transparent response describing how it’s being actioned.
Raise awareness of your service: Plan an ongoing awareness campaign and deploy it across a variety of channels to reach your users. Consider training your frontline staff so they can inform, suggest or demonstrate the service to people.
OffSuccessful delivery of the investment requires changes to budget, schedule, scope or benefits. There are major issues with investment definition, schedule, budget, quality and/or benefits delivery, which don't appear to be manageable or resolvable without such changes being made.
OffSuccessful delivery of the investment to time, cost, quality standards and benefits realisation appears highly likely and there are no major outstanding issues that at this stage appear to threaten delivery significantly
Successful delivery of the investment to time, cost, quality standards and benefits realisation appears probable however constant attention will be needed to ensure risks do not become major issues threatening delivery.
Successful delivery of the investment against budget, schedule, scope and benefits, appears feasible but significant issues already exist, requiring management attention. These appear resolvable at this stage and, if addressed promptly, should not present a cost/schedule overrun or loss/delay of benefits.
Successful delivery of the investment requires urgent action to address major risks or issues in a number of key areas. Changes to budget, schedule, scope or benefits may be necessary if the investment is to be delivered successfully.
Successful delivery of the investment requires changes to budget, schedule, scope or benefits. There are major issues with investment definition, schedule, budget, quality and/or benefits delivery, which don't appear to be manageable or resolvable without such changes being made.
Assurance research series: 01
The review found the SSA model aligns with key related policies and strategies, with opportunities to further evolve how these are applied in practice. Given the ever-evolving policy landscape, it is important to consider SSA alignment as an ongoing activity to ensure they are kept up to date as new policies are introduced or changed.
The alignment of the SSAs to existing and emerging policy, strategies and key legislation was considered across four key policy dimensions: procurement and contracts, cyber and security, digital and investment, and the broader ecosystem.
In particular, the review identified the following improvement opportunities:
Test
Off2.1 Single seller arrangements (SSAs) are pre-negotiated, whole of Australian Government technology agreements with specified sellers, established to leverage the government's collective buying power and ensure consistency with legal terms and conditions. They are not a procurement pathway.
2.2 The policy and legislative environment is constantly changing. Similar to other long term contractual arrangements (e.g. Digital Marketplace Panel 2), an SSA may become out of step with policy changes over the life of the contract. For example, the recent introduction of the Supplier Code of Conduct on 1 July 2024 required updates to be made to the SSAs. This necessitates the periodic review of the SSAs throughout their life to ensure ongoing alignment to the policy and legislative environment as it changes.
2.3 To manage this, the Digital Transformation Agency (DTA) routinely undertakes procurement policy compliance assessments for the SSAs. Further, the SSAs include clauses that allow updates to be made to the SSAs as policy and legislative environment changes.
2.4 When undertaking any procurement activity, there are a range of policies any digital procurement needs to consider, as depicted in the chapter overview above. Some policies have specific values that trigger different requirements for digital procurements are triggered, which is depicted in the figure below.
The figure depicts 11 policy thresholds, from $0 to $100 million, as follows:
$0:
$10,000:
$80,000:
$80,000 to $200,000:
$1 million:
$2 million:
$4 million:
$7.5 million:
$10 million:
$20 million:
$100 million: