Links
- Understand your environment - VPSC
- Technology | Department of Industry Science and Resources
- Information management for current, emerging and critical technologies | naa.gov.au
- Cost Benefit Analysis | The Office of Impact Analysis (pmc.gov.au)
- Guidelines: Cost-Benefit Analysis | NSW Treasury
- further-guidance-04-cost-benefit-analysis-guide.pdf (statedevelopment.qld.gov.au)
Scenarios | Department of the Prime Minister and Cabinet (DPMC)
Off
Establish a baseline for the service
- Understand the current state: For existing services, determine the current state by identifying and reviewing existing metrics. For new services, establish a baseline for the problem identified in Criterion 1. Both are a yardstick to measure progress.
- Use benchmarks to gauge performance: Compare the service to similar services or existing standards to identify areas of improvement. Seek out best practices of similar and well-performing services to consider if they can be adopted.
Identify the right performance indicators
- Select meaningful metrics: Collect metrics that accurately capture the service’s ability to deliver the outcomes that users expect. These might include adherence to design standards and privacy legislation, site/app performance, security benchmarks or tasks completed by users.
Measure, report and improve according to strategies
- Measure against the Data and Digital Government Strategy: Make sure the service meets the Data and Digital Government Strategy. Consider how information collected and reported could improve the service in line with the Strategy’s implementation plan.
- Apply benefits management: All digital and ICT-enabled investment proposals must define their purpose, outcomes and methods for measuring, monitoring and optimising them. Find out more in the Benefits Management Policy.
Guidance to monitor your service
Exemption policy
Apply criterion 9 of the Service Standard: Monitor your service
To meet criteria 9 of the Digital Service Standard, refer to the Digital Performance Standard. The Performance standard is an extension of criteria 9 of the Service Standard.
On-
To meet criteria 9 of the Digital Service Standard, refer to the Digital Performance Standard. The Performance standard is an extension of criteria 9 of the Service Standard.
Improve the service across its life
- Make improvements: Increase people’s use of the service by continuously optimising performance, enhancing security, introducing relevant features, addressing bugs and increasing compatibility. Use metrics identified in Criterion 9 (‘Monitor your service’) to reveal the biggest opportunities for impact and ground improvements in evidence. Provide adequate training and materials for staff to support change.
Schedule regular assessments
- Undertake assessments: Define the goals and scope of the assessment then observe performance and experience over time. Performance metrics might include load times, responsiveness or bottlenecks. Experience metrics might include entry/exit points, dwell time or task abandonment. Ongoing monitoring should be part of business-as-usual processes and a detailed review part of regular service evaluation.
Communicate service upgrades
- Communicate the change: Develop a communication plan for how, when and through which channels to share updates and findings with users. When writing content, show how users’ feedback informed the actions that have been taken. Highlight key achievements or milestones reached and use real-life stories to demonstrate how users shaped change.
Guidance to keep it relevant
Make improvements as required
Keep abreast of what changes and improvements could be made incrementally, such as addressing bugs and introducing relevant features to your service. Use methods or tools such as:
- Establishing a robust system for reporting and tracking bugs. This could involve using issue tracking software to log bugs and assign them to appropriate team members.
- Establishing a routine (e.g., monthly, quarterly) to assess the overall performance of the service. This can involve reviewing user engagement, system performance, and the effectiveness of new features.
- Developing thorough testing protocols, including unit testing, integration testing, and user acceptance testing (UAT). Consider automated testing where possible to streamline this process.
Introduce relevant features
Update your service by introducing relevant features, where there is a benefit for users. Use methods or tools such as:
- Conducting surveys, interviews, and usability tests to understand user needs and pain points. This research will inform feature development that aligns with user expectations and enhances the overall service.
- Staying up to date on new or popular features that are well-received, and consider how similar or improved features could be beneficial for your service.
Ensure that feedback can be provided
Ensure that feedback can be provided so that you can make changes in line with what users want to see, and have asked for. Use methods or tools such as:
- Creating multiple channels for users to provide feedback, such as surveys, feedback forms, or user forums. This encourages users to share their thoughts and experiences.
- Ensuring that user support teams are responsive to feedback and inquiries. Acknowledge received feedback and communicate any actions taken in response.
- Using user feedback to continuously refine and improve the service. Highlight changes made based on user suggestions to foster a sense of community and investment among users.
Compliance & reporting
Document your findings
Document your findings and recommendations to apply criterion 10:
- Implement version control to keep track of changes made over time.
- Document all user feedback received through various channels. This can be compiled in a dedicated feedback log.
- Track actions taken in response to user feedback, documenting any changes made based on this input. Include user testimonials or case studies where applicable.
- Make sure the data is collected and documented in a centralised knowledge repository.