What to Do When You Don’t Know What You Don’t Know…

So once again there was a great coincidence from my travels.  Three times in the past week I was told that infamous line “We don’t know what we don’t know”.  While I understand the sentiment behind the comment, it is often just an empty throw away line to explain why inertia is ruling the day.

The real issue is that trying to understand “what you do not know” is complicated by two big factors.  The first is that that universe of the unknown is huge and what you really want to get at is what is unknown, but valuable to know. 

So once again there was a great coincidence from my travels.  Three times in the past week I was told that infamous line “We don’t know what we don’t know”.  While I understand the sentiment behind the comment, it is often just an empty throw away line to explain why inertia is ruling the day.

The real issue is that trying to understand “what you do not know” is complicated by two big factors.  The first is that that universe of the unknown is huge and what you really want to get at is what is unknown, but valuable to know. 

The second problem that complicates this venture is that even if I could tell you what you don’t know(that has value), you would not understand why you want to know it.

Rather than try to explain how one gets to “know what they don’t know”, I take a different tack and offer this strategy to anyone in the situation.  I ask if they know what they are suppose to know and go from there.  Some examples to illustrate the point.

Editor’s note: Rob Armstrong is an employee of Teradata. Teradata is a sponsor of The Smart Data Collective.

If you are in a call center dealing with customers then do you know all the transactions and interactions that customer had in the past few weeks?  Do you know the total value of the customer and the frequency at which they do business with you or have called regarding a bad experience?  Better yet, do you know all the bad experiences they had even if they did not call to discuss the problem?  Most likely the answer is that while that information may be available it is not served up to the call center employees.  In that case you do know what you don’t know, you simply are making it known.

Following in that line of investigation the next questions are about the timing of that knowledge.  You may “not know what you don’t know”, but you certainly do know things but you know them too late to do anything about it.  So what data and reports do you use to measure your effectiveness?  Why is that same data not available soon enough to act on out rather than report on it? 

For example, retailers have long figured this one out.  Rather than waiting to get a report that you are out of stock for an item, set triggers at a discrete item / location level.  This means that re-orders happens far enough in advance to avoid the outage altogether.  By looking at your process you will find that you do know what you don’t know, you just don’t know it soon enough.

Once you start getting data to the right people at the right time then you will be able to ask the real question, what is it that I am missing to make a better decision?  Sometimes this data is elsewhere in your organization but outside your scope, other times this analysis drives you to include external data into the equation.

At one airline they were trying to minimize missed flight connections and delays.  After getting the easy data about arrival and departure time and gate assignments to minimize time between connections they asked “what next”.  What else causes this pain?  This led to adding baggage, catering, crew schedules, and many other events into the warehouse to increase the analytic effectiveness.  Now you are starting to identify what you truly do not know, that has value,  so you can start knowing it!