The first edition was produced in October 2024 as a collaboration between the Digital Transformation Agency (DTA) and the University of Sydney based on contemporary events and research findings. The intent of this version is that it is used as guidance and facilitates broader dissemination and feedback. The second edition incorporates input gathered during a consultation period and reflects updated insights and perspectives.

Digital project success and researcher profiles

Digital project success

This series engages researchers from academia on issues influencing the performance of digital projects. These projects play a vital role in allowing Australia to seize the opportunities presented by new technologies. 

The DTA works across the Australian Government to support the successful design and delivery of digital projects. The work of the DTA includes managing the assurance system which drives good decision-making and seeks to create the conditions each project needs to succeed. 

This research series is part of the DTA’s commitment to ensure the Australian Government achieves nothing less than excellence in digital project design and delivery.

Researcher profiles

Dr Julien Pollack - Associate Professor

Dr Julien Pollack is an Associate Professor with the John Grill Institute for Project Leadership and the School of Project Management at the University of Sydney. 

He started as a project manager in IT, organisational change, and manufacturing projects, before moving into research. He now explores multiple aspects of project management, with the broad aim of helping to transform the discipline into one that addresses the needs of complex and uncertain environments. 

This includes investigation of project teams and their productivity, project management methodology, and projects with ill-defined objectives and outcomes. His research in these areas is regularly published in the leading international project management journals and research conferences. 

Dr Natalie Smith - Associate Professor

Dr Natalie Smith is an Associate Proffessor of practice with the John Grill Institute for Project Leadership. Her expertise is in the governance of digital transformation. Natalie’s PhD was on digital transformation governance, and she has since completed a research fellowship on trust in Artificial Intelligence (AI) in the Australian public sector. Her current research is on designing fit-for-purpose assurance for digital projects. 

Natalie is also a non-executive director in not-for-profit Health and Community Service organisations and government. She is a member of the National AI Thinktank for Responsible AI. Previously, Natalie was a partner in Deloitte’s Risk Advisory practice, providing project assurance and supporting organisations delivering digital transformations. 

Dr Wei-Ting Hong - Postdoctoral Research Associate

Dr Wei-Ting Hong is a postdoctoral research associate at the John Grill Institute for Project Leadership, the University of Sydney. His research covers digital transformation in project delivery, Natural Language Processing (NLP), prompt engineering and public transport management. 

He is particularly interested in how artificial intelligence (AI) can benefit project models, concentrating on the potential of NLP and Large Language Models (LLMs). His PhD was on leveraging the power of NLP to enhance rail safety and organisational learning behaviour. Wei-Ting is also a committee member of Railway Technical Society of Australasia (RTSA), a Technical Society of Engineering Australia (EA).

Background and purpose

The Digital Transformation Agency (DTA) engages across the digital project lifecycle to support, advise and coordinate the government’s digital and ICT-enabled investments via the six states of the Investment Oversight Framework (IOF)

State 4 (Assurance) of the IOF aims to provide assurance to government that approved digital projects are on-track to deliver expected outcomes and benefits – including by ensuring projects plan for and implement fit-for-purpose assurance activities to support good decision-making throughout delivery. 

The DTA engaged the John Grill Institute for Project Leadership at The University of Sydney to contribute to a research series aimed at eliciting best practice guidance to maximise the rate of digital project success. 

As the first topic in this series, this paper and the guidance within supports independent assurers tasked with assurance activities that review and assign Delivery Confidence Assessment (DCA) ratings for digital projects.

The primary purpose is to help improve the consistency and understanding of DCA ratings for digital projects. 

Specifically, this publication is intended to:

  • Address the unique challenges and issues faced by digital projects.
  • Outline inputs and focus areas that assurance reviewers could use to determine DCA ratings.
  • Provide general tolerance levels for each rating category.
  • Uplift capability and understanding of DCA ratings for the people who use them, including Senior Responsible Owners (SROs) and project boards. 

The primary audience for this guidance consists of: 

  • Departments and agencies (agencies) and Independent Assurance Providers (IAPs) engaged by agencies.
  • Senior Responsible Officials (SROs) within agencies who are accountable for digital projects.
  • Governance boards (also called steering committees) overseeing digital projects.
  • Central agencies of the government (such as the DTA) who rely on DCAs to form advice to government.

Assurance research series: 01

Assurance Research Series: 01

What are Delivery Confidence Assessment (DCA) ratings and why are they important? 

Delivery Confidence Assessment (DCA) ratings result from independent assurance activities that agencies conduct to assess the confidence level of the project delivering successfully. 

Assurance activities are defined as independent and objective assessments and evaluations undertaken by people and entities separate to the delivery team and the Senior Responsible Officer (SRO). While not every assurance activity will produce a DCA rating, every DCA rating will have an associated assurance activity that informs the rating.

The Assurance Framework for Digital and ICT Investments requires that regular assurance activities are conducted to produce a DCA. These ratings are crucial to provide an indication of an investment’s overall trajectory to deliver on intended outcomes and benefits. 

The DTA draws heavily on assurance information to inform and focus its oversight and engagement across the portfolio of in-flight digital and ICT investments.

DCAs form a key input to the DTA’s advice and reporting to government. Information from DCAs helps shape government improvements to the ways in which digital projects are managed. They help to ensure that at-risk projects trigger escalation processes and help to direct remediation planning to at-risk projects so they can be brought back within accepted success criteria.

Consistency in how DCAs are defined is critical to the effectiveness of key decisions by project governance forums and to the DTA’s oversight and advice. 

Assurance activities that require the inclusion of a DCA rating in reports provided to the DTA must use the agreed ratings below. They should also instil confidence the DCA rating being awarded is unbiased, rigorous and evidence based.

High

Successful delivery of the investment to time, cost, quality standards and benefits realisation appears highly likely and there are no major outstanding issues that at this stage appear to threaten delivery significantly

Off
Medium High

Successful delivery of the investment to time, cost, quality standards and benefits realisation appears probable however constant attention will be needed to ensure risks do not become major issues threatening delivery.

Off
Medium

Successful delivery of the investment against budget, schedule, scope and benefits, appears feasible but significant issues already exist, requiring management attention. These appear resolvable at this stage and, if addressed promptly, should not present a cost/schedule overrun or loss/delay of benefits.

Off
Medium Low

Successful delivery of the investment requires urgent action to address major risks or issues in a number of key areas. Changes to budget, schedule, scope or benefits may be necessary if the investment is to be delivered successfully.

Off
Implement a feedback mechanism

 

Incorporate feedback: Offer users the ability to provide feedback, report issues and suggest service improvements. Promptly act on feedback and provide a timely, transparent response describing how it’s being actioned.

Raise awareness of your service: Plan an ongoing awareness campaign and deploy it across a variety of channels to reach your users. Consider training your frontline staff so they can inform, suggest or demonstrate the service to people.

Off
Low

Successful delivery of the investment requires changes to budget, schedule, scope or benefits. There are major issues with investment definition, schedule, budget, quality and/or benefits delivery, which don't appear to be manageable or resolvable without such changes being made.

Off
  • Successful delivery of the investment to time, cost, quality standards and benefits realisation appears highly likely and there are no major outstanding issues that at this stage appear to threaten delivery significantly

  • Successful delivery of the investment to time, cost, quality standards and benefits realisation appears probable however constant attention will be needed to ensure risks do not become major issues threatening delivery.

  • Successful delivery of the investment against budget, schedule, scope and benefits, appears feasible but significant issues already exist, requiring management attention. These appear resolvable at this stage and, if addressed promptly, should not present a cost/schedule overrun or loss/delay of benefits.

  • Successful delivery of the investment requires urgent action to address major risks or issues in a number of key areas. Changes to budget, schedule, scope or benefits may be necessary if the investment is to be delivered successfully.

  • Successful delivery of the investment requires changes to budget, schedule, scope or benefits. There are major issues with investment definition, schedule, budget, quality and/or benefits delivery, which don't appear to be manageable or resolvable without such changes being made.

DCA rating

Assurance research series: 01

Alignment to policy, strategies and legislation

Chapter Overview

The review found the SSA model aligns with key related policies and strategies, with opportunities to further evolve how these are applied in practice. Given the ever-evolving policy landscape, it is important to consider SSA alignment as an ongoing activity to ensure they are kept up to date as new policies are introduced or changed.

The alignment of the SSAs to existing and emerging policy, strategies and key legislation was considered across four key policy dimensions: procurement and contracts, cyber and security, digital and investment, and the broader ecosystem.

 

Details the rules, policies, strategies, etc that align with Procurement and contracts, Digital and investment, Cyber and security, and Broader ecosystem.  Further details is provided in the accordion directly below this image.
Figure 11 Policy, strategy and ecosystem dimensions
description of Figure 9 Policy, strategy and ecosystem dimensions Off

Procurement and contracts

In particular, the review identified the following improvement opportunities:

  • Further value can be obtained in existing policies by negotiating adoption expectations in the head agreement, rather than devolving this responsibility to the buyer’s contract (e.g. CAIP Plans).
  • Consider how to mitigate unintended market consequences in the existing Contract Limits and Review Policy to address unintended market outcomes.
  • Define data and digital sovereignty with consideration of localisation requirements, and assess the technology landscape to identify technology necessitating stronger cyber and security requirements.
  • Distinguish more clearly whole of Australian Government digital contracts from Coordinated Procurements within the Commonwealth Procurement Rules (CPRs).
     
Test

Test

Off

Procurement and contracts

2.1 Single seller arrangements (SSAs) are pre-negotiated, whole of Australian Government technology agreements with specified sellers, established to leverage the government's collective buying power and ensure consistency with legal terms and conditions. They are not a procurement pathway. 

2.2 The policy and legislative environment is constantly changing. Similar to other long term contractual arrangements (e.g. Digital Marketplace Panel 2), an SSA may become out of step with policy changes over the life of the contract. For example, the recent introduction of the Supplier Code of Conduct on 1 July 2024 required updates to be made to the SSAs. This necessitates the periodic review of the SSAs throughout their life to ensure ongoing alignment to the policy and legislative environment as it changes.

2.3 To manage this, the Digital Transformation Agency (DTA) routinely undertakes procurement policy compliance assessments for the SSAs. Further, the SSAs include clauses that allow updates to be made to the SSAs as policy and legislative environment changes. 

2.4 When undertaking any procurement activity, there are a range of policies any digital procurement needs to consider, as depicted in the chapter overview above. Some policies have specific values that trigger different requirements for digital procurements are triggered, which is depicted in the figure below.

The figure depicts 11 policy thresholds, from $0 to $100 million. Refer to the accordion for Figure 12 for a long description.
Figure 12 Policy thresholds
Description of Figure 12

The figure depicts 11 policy thresholds, from $0 to $100 million, as follows:

$0:

  • All policies without a financial threshold are applied as default.
  • Pay on Time policy, invoices are to be paid within 20 days or 5 days if Pan-European Public Procurement On-Line (PEPPOL) enabled

$10,000:

  • Contracts to be published within 42 days of arrangement being entered

$80,000:

  • Tenders must be put out to open market on AusTender, subject to Limited Tender rules
  • Completion of the Fair Criteria Checklist and the Consider First Assessment Tool
  • Suppliers must provide a Certificate of Compliance with the Workplace Gender Equality Act

$80,000 to $200,000:

  • Mandatory Set Aside requiring Indigenous enterprises be engaged first on procurement opportunity

$1 million:

  • Vendors to provide Supplier Environmental Sustainability Plans for ICT goods
  • ICT contracts need to consider economic benefit to Australia

$2 million:

  • Consultancy contract reporting

$4 million:

  • Statement of Tax Record must be submitted as part of any Open Tender response and be maintained throughout the contract
  • Payment Times contractual clauses to be included requiring suppliers to pay sub-contractors up to $1m (GST inc) within 20 calendar days or pay interest

$7.5 million:

  • Indigenous Procurement Plan required, including Mandatory Minimum Requirements for contracts / panels to be awarded to Indigenous enterprises where contracts are wholly delivered in Australia

$10 million:

  • Skills guarantee requires targets for ICT projects for gender equality, apprentices and cadets to be set

$20 million:

  • Australian Industry Participation Plan required

$100 million:

  • Modern Slavery Reporting by suppliers who have revenues over $100m
  • Digital contracts cannot exceed $100m in value (and three years), unless an exemption has been granted.
Off

Connect with the digital community

Share, build or learn digital experience and skills with training and events, and collaborate with peers across government.