IT wants to deliver quality service, which requires accurately measuring and presenting your service performance. Choosing the right approach ensures that you’ll be able to quickly focus executives and managers on the information they need to make agile decisions.

If you’ve been working with IT metrics for a while, you’ve seen many standard KPIs that all use the same measurement, approach: Averages.

Mean Time to Repair, Mean Time to Resolve, Mean Time between Failures – all are calculated using averages. But this isn’t the only way you could measure these outcomes, so why do it this way? Why not use, for example, Medians?

You might be doing this because this is the way you’ve always done it, or because your tool doesn’t give you any options. But let’s supposed you’re not satisfied with doing things the way they’ve always been done – why do we use averages instead of medians?

 So, what is the difference between Averages and Medians?

Overall, both averages and medians are looking at what falls in the middle of a range of numbers 

If you cast your mind back to school, you’ll remember that an average is calculated by adding up all the values and dividing by the number of values.

For example, to average 1, 2, 3, 3, 4, you would add the numbers up (1 + 2 + 3 + 3 + 4 = 13) and then divide by the number of values (5) to get a result of 2.6.

On the other hand, to get the median of 1, 2, 3, 3, 4, you would put the number of values in order and take the middle value to get a result of 3.

Even from that little example, you can see that the results are similar but not the same.

The reason is that averages tend to be more sensitive to outliers. Medians, meanwhile, are limited to either the numbers in your set, or (if your set of data is odd) halfway between two numbers. Let’s take a look at two examples that illustrate the impact.

Here’s one example, based on the number of times incidents have been reassigned:

You can see a clear divergence in the reassignment average and median.

A major reason for the difference is the 12 incidents that were reassigned 100 times. Although it’s only a few edge cases (compared with over 200 that were only reassigned once), it has a big impact on the average. Take a look at the same report, but with those 12 incidents excluded:

You can see that by dropping the outlier, the average moves a lot without the median moving much – and now the average looks a lot more like the median!

Okay so they’re not the same – why does an IT team care?

Clearly if these two measures give you different results, you want to think twice about which one you pick!

Let’s start by looking at the differences and what that might mean in the data.

Use Medians to Limit the Impact of Outliers

When you have a big set of data that may have a lot of outliers, you can use a median to get a steadier idea of the data. For example, Service Level Agreements may often have a lot of data, and there may be a few SLAs that stay open for a long time, disrupting your results.

Suppose you have an organization that is measuring resolution of service delivery. Our target is to have the typical service resolved within 14 days.

Here’s a chart with the average and median resolution times (the dark dotted line across the center is our target):

You can see that the Average resolution time has a lot more spikes and troughs where a few outages may have deeply impacted the numbers. The Median is a much more consistent measure of what the typical person is experiencing.

What does that tell you? It tells you that your typical resolution experience is actually on-track! If you tackle the few big outliers, you could make a big impact in your number! On the other hand, if the median was above the target, you’d want to make decisions about how to speed delivery time in your typical process.

(Good news though: the colored, dotted lines into the future show that both measures are trending to improve).

Use Averages Where Fine Distinctions are Useful

On the other hand, sometimes the sensitivity of an average is useful. One common example, which also speaks to service quality, is service desk survey results – typically ranking service on a scale of 1-5 or 1-10.

Take the following set of survey results:

Here, because the values are whole numbers, the median is going to tend to be a whole number – and in this case, it’s pretty likely to be 3. This is a case where you want the outliers to have a bit of an effect, and you know it won’t be disproportionate because your range is limited from 1 to 5!

The Right Measurement for the Task at Hand!

As you can see, the choice of which measurement can really impact what you see when you analyze these important service quality metrics. Both averages and medians have their place, so make sure to think carefully about which approach provides your executives and managers with the most accurate answers to the question they’re asking.

Last week, I had the pleasure of attending one of my customer’s internal strategy planning meetings.

Leaders from a few different IT groups had come together to talk about how to streamline their operations over the coming year, sharing successes and strategies.

Explore Analytics was being featured as one of the solutions that had freed up time and brought agility to decision-making.

In order to demonstrate the value and efficiency Explore Analytics delivered, we focused on just one report that Explore Analytics replaced.

The True Cost of Relying on Excel

The following chart shows the Explore Analytics report that was previously done using Excel:

The spreadsheet replaced by the report above was an important decision-making report using data dumped out of SAP.

The report shows the average length of time that a scheduled job within SAP takes to run, and it’s an important indicator on whether or not SAP is running into performance issues.

Before Explore Analytics, the customer needed to dump the relevant information into an Excel spreadsheet to generate the report manually.

Because this was being done in Excel, it had to be done by hand every day.

I asked his manager, “What happens when he’s on vacation?” and the answer was: “We don’t get the report.”

This report was a crucial report — in the sample above, you can see that the job time spikes later in the week, indicating a serious problem that needed to be addressed.

With a manual report, run once a day, this insight could come delayed or not at all, depending on the availability of the human manually running the report.

Are You Held Hostage to Excel Spreadsheets?

This is just one Excel report. This customer now has nearly 1500 reports and counting; many of which were also laborious manual reports needed for crucial insights.

The reason this report was the subject of discussion of the process improvement summit was because this report was not an outlier.

Rather, this report is typical of how problems get solved in IT — with an Excel stop-gap report that sadly becomes a routine, manual process.

Every organization I come across is suffering from the same problem — time being wasted by manual reports and Excel workarounds to fill in gaps in existing reporting.

Ask your team. Are any of them spending time on excel spreadsheets for reports? You may be shocked to find out how much time your team is spending on these manual efforts.

Get Better Visibility

Explore Analytics provides a better alternative, for a few reasons:

• Automate your reports; Reports only need to be built once, so manual effort is never needed again to get the same insight. Spend your time elsewhere!
• Get your reports live; In the example above, the manual version of the report would only be accessible once daily. If problems began at the beginning of the day, they wouldn’t be seen until late in the afternoon. Live reports give you insights when you need them, not just when people have time to build them.
• Connect to data and build reports easily; Many organizations think they have the reporting tools to solve these problems. But when they require advanced integrations, knowledge of coding or SQL, and other time and labor intensive efforts, they are throwing too many barriers up to letting your team build reports. Instead, your team will fall back on Excel, thinking they’re saving time. With Explore Analytics, you don’t need to waste time on the technical side — get it done in minutes!

Take a few minutes this week to ask your team and see — are they spending time building reports manually? Are your key reports based on one-time or scheduled dumps of data? Is it dragging your team down?

If so, give Explore Analytics a try, and free up your team today!