Taking Measurement of Your Measurements

Wise things I have learned over the years.  Never eat cotton candy in the rain, never shine a flashlight into African trees where monkeys may be sleeping, and one that is more relevant to all you readers: if you are not measuring what matters then you measurements don’t matter.

Editor’s note: Rob Armstrong is an employee of Teradata. Teradata is a sponsor of The Smart Data Collective.

Wise things I have learned over the years.  Never eat cotton candy in the rain, never shine a flashlight into African trees where monkeys may be sleeping, and one that is more relevant to all you readers: if you are not measuring what matters then you measurements don’t matter.

Editor’s note: Rob Armstrong is an employee of Teradata. Teradata is a sponsor of The Smart Data Collective.

So what irked me this week?  As I was back on the travel circuit, it was time to once again hit the planes and fly around the country.  One of my flights was showing delayed but “it is a short flight so all connections are OK”.  After we pushed back we sat on the tarmac for another 30 minutes.  Finally took off and landed at our destination.  We landed, but we did not deplane (BIG DIFFERENCE).   Due to not having a gate available, we again sat on the tarmac for over 30 minutes.  I must admit I had a great view of my connecting flight push back and depart without me.

Why does this matter?  Because when I was looking at the flight status and such I kept seeing that my original plane had “landed”.  Unfortunately what matters to the passengers is not that we “land” but that we actually get off the plane.  This is a classic case of companies measuring the wrong metrics or providing the wrong details and information to people that need to make decisions and take actions against that information.

I will say upfront that I understand the airlines do not like delays anymore than the passengers do and there are many things, such as tarmac congestion, weather, and what not beyond the control of the airline.  I also understand that even with the 13 hour delay in getting home as I got to spend the night until the next flight out, I still made it home faster than if I had driven.  I get all that, what I don’t get is how the problems are ever going to be really addressed if the metrics companies use (and broadcast to others) do not reflect the reality of the situation.

With all that, the question is what makes a meaningful metric?  I think the most important attribute is that it must be focused on understanding the actions and outcomes that you not only run your business by but also the ones that impact your ultimate customers the most. 

For example one of the key metrics in a call center is “time spent on call”.  How long was the call center operator on each call.  Are they spending an inordinate amount of time and therefore not clearing the queue or are they quickly dispatching one call and working on the next?  Another example was that at the end of a call to a help line I was asked by the rep if “we helped you to the best of our ability”.  They scored high, unfortunately the best they could do did not resolve the problem.

A better metric is single call resolutions, that is, how often is the customer situation identified and resolved on the first call rather than having multiple attempts?  If the customer is properly served in a two minute call that is much better than four calls at thirty seconds each.  An even better measurement would be to get a combination of average time and one call resolution.  This is just a simple example that shows that many perspectives need to be considered for the whole picture.

The next attribute of a good metric is timely visibility to all who can influence the actions.  If you are tracking my average call time, then I hope I can see that at any time so I can adjust my actions and work processes accordingly.  If the metrics and measurements are only shared on a periodic basis (say daily or weekly) then any resolution to problems will only be delayed.

So timely is important, but let’s look at the last part of that attribute, to all who can influence actions.  No sense sharing metrics to people without the authority or intelligence necessary to make decisions and then take actions. 

One example is a company that shared critical operational process metrics with the executive teams via a dashboard.  When a problem was identified it took an executive (or subordinate) making many calls to spur corrective action.  Fortunately the company figured this out and now gives the necessary information directly to those involved in the process so immediate steps can be taken.  This company has radically changed how events are responded to and in many cases the problems are resolved before there is any noticeable impact to the end consumer.

Alright, question time:  What metrics are you measured by?  Are they readily available for you to review and gauge progress?  Do you understand the actions and processes that can change the measurements both positively and negatively?  If not, why not and how do you prioritize your work?