188体育

As a service owner, you should know how well your service is working for users. To do this, you鈥檒l need to design metrics that have a clear meaning and collect data that tells you how your service is performing against them.

You must collect and publish data for the 4 mandatory key performance indicators (KPIs). You鈥檒l also need your own KPIs to fully understand whether your service is working for users and to communicate its performance to your organisation.

Have a performance analyst on your team from the start of discovery and make them part of the process of building your service. Work out your measurements as you build the service, not afterwards.

You should always combine your metrics with user research and iterate your service based on insight from both to make sure it鈥檚 always improving.

Base your metrics on a sound understanding of your service鈥檚 purpose

Well designed, meaningful metrics tell you whether your service is doing what it should. Start designing your metrics by clearly defining the purpose of your service.

You should consider:

• the user needs your service is designed to meet
• any policy outcomes your service is intended to produce, for example a reduction in fraud or costs

Aim to sum up why your service exists in a single line. For example:

188体育 Pay will make it easier and more efficient for government to process payments, saving time and effort for service teams across government.

You might find it useful to with your team to achieve this.

Define goals, or 鈥榖enefits鈥�, for your service

Derive a small number of 鈥榖enefits鈥� for your service. A benefit should be a short description of a certain outcome that will contribute to your service fulfilling its purpose. Base your benefits on your understanding of your users鈥� needs.

Develop hypotheses based on your benefits

Articulate how something you are going to do will help you achieve one of your service鈥檚 benefits and why.

Use the format:

Doing/changing [something]鈥�

Will [benefit]鈥�

Because [assumption/logic/strategy]鈥�

Aim to get to the core assumptions about how your service will add value. This will help you make your measurements as relevant as possible.

For example:

If we provide an online form, it will speed up the application process because digital applications are quicker than paper ones.

Hypotheses will help you , because they let you measure the effect of a specific course of action you鈥檝e chosen to follow.

Decide what to measure based on your hypotheses

Your hypotheses should take you to a point where you have a set of things to measure that are easily measurable. At the start of developing your service you do not have to think about specifically how you鈥檒l measure them, but you should have a clear understanding of what each measurement means.

For example:

We will measure how long it takes a user to make an application using the online form we build.

As your service evolves, you鈥檒l need to identify which metrics will be the KPIs you share to show how your service is performing. Choose 3 or 4 metrics that would answer the question: 鈥榠s this service working?鈥�

You鈥檒l still probably need other metrics within your team to understand your service鈥檚 performance in more detail.

Find data sources for your measurements

After you鈥檝e chosen what to measure, you鈥檒l need to get access to all the data sources you鈥檒l need to make your measurements. Data sources could include:

• digital (web) analytics
• user feedback
• site performance
• call centre data
• financial information (for example, how much your hosting costs)

Digital analytics will tell you how people are interacting with the online parts of your service. You should make sure:

• you choose a digital analytics tool that will give you what your service needs
• your performance analyst is involved in building your service - it鈥檚 much easier to implement data collection and solve problems as you build than to go back and do it afterwards

Do not just use digital analytics. Combining data from a range of sources will give you a richer picture of how your service is performing.

There are different ways to access each data source and you鈥檒l be able to use different tools to do so. Your performance analyst can help you consider practical questions such as:

• how you鈥檒l gather user feedback - for example, you might need to run a user satisfaction survey or collect data from your call centres
• what software you鈥檒l need to install if you鈥檙e going to monitor service uptime and performance

Collect and analyse data

Once you鈥檝e identified all your data sources, your team鈥檚 performance analyst will:

• make sure your data is collected accurately
• check for statistical significance
• test for errors or inconsistencies in the data
• do appropriate analysis

Analysis needs to be timely and help you take action. Effective analysis should tell you:

• what happened
• why it happened
• what you need to do about it

Give context to your measurements

To be useful, metrics need to have context. You should provide context for your metrics by:

• using segments in your analysis
• measuring your service鈥檚 performance over time
• comparing your service鈥檚 performance with that of similar services

Segmenting data

Make sure you use segments in your analysis. Segments group your users, for example by:

• device type - whether they used a phone or a desktop computer to access your service
• new or repeat user - whether they had used your service before

Using segments correctly will help you identify and understand trends, because they help you understand how your users are different. For example, new users may behave very differently to returning users, making an overall average of user behaviour inaccurate.

Measuring performance over time

Establish a 鈥榖aseline鈥� of how your service performs currently, across all channels. Judge changes to the service鈥檚 performance against this.

If you鈥檙e building a new service, you can use the performance of existing or legacy systems to inform your baseline.

Measure performance continuously as much as possible, rather than taking 鈥榮napshots鈥�. This will give you a 鈥榯rend line鈥� against which peaks and dips in performance can be measured, making it easier to see the effects of:

• changes to the service
• communications initiatives
• seasonal variation

Set targets for what you expect each KPI to be at a specific time in the future if your plans to improve your service are successful. For example, in 6 months鈥� time you might expect to see a 10% reduction in the time it takes for a user to make an application because of your work to simplify the process.

Comparing performance with similar services

There are likely to be services similar to yours in terms of:

• number of users
• complexity

Comparing your service鈥檚 performance with other, similar services will let you know whether your service is performing significantly better or worse than expected.

You can get .

Using dashboards, reports, alerts and visualisations to share findings

Find a way to share findings that suits what you鈥檙e trying to do.

Presenting data visually in a prominent place or message is a good way to maximise the likelihood that the information will be acted upon and improvements made to your service.

Dashboards are objective-focused and will help inform decisions, often with the help of real-time data. Read about

Alerts are used to inform the users about a change or an event, often using attention-grabbing delivery mechanisms.

Reports provide regular, scheduled snapshots of data and tend to require extra context and time to digest.

You should try to automate the creation of standard reports, for example so your analyst can spend their time on more valuable work.

Monitor, iterate and improve

Iterate your metrics as your service changes - you鈥檒l need to know different things as your service changes over time.

Further reading

You might find the following blog posts useful: