Healthcare Has a Problem: Big Data & The Law of Seven

October 6, 2011
129 Views

In one of the most quoted papers on psychology ever written, George Armitage Miller claimed that he had been persecuted by an integer.  That integer was the number 7.  Over a period of seven years, he had found through his own research and through an extensive literature review th

In one of the most quoted papers on psychology ever written, George Armitage Miller claimed that he had been persecuted by an integer.  That integer was the number 7.  Over a period of seven years, he had found through his own research and through an extensive literature review that human beings were capable of processing seven (plus or minus two) variables at any one time.  For those of you more technically inclined, if human beings were computers, then seven would define the number of variables that we could store in our working memory.  In fact, more recent research has shown that the number might actually be closer to four, rather than seven.

Most of us experience this challenge every day.  Studies show that, in this increasingly wired and information intense world, we are interrupted every three to five minutes during the course of a work day, multitasking between email notifications, text messages, cellphone calls, chat requests and reviewing web sites.  We frequently end up getting distracted and temporarily forget what we were working on a few minutes before.  We are all information junkies and digital connectivity has radically increased the flow of data to feed our habit – yet our brain processing capacity hasn’t evolved significantly since the Stone Age.  No surprise that many folks feel overwhelmed and less productive and effective than they feel they could be.

So, what does this have to do with healthcare?  Actually, plenty!  Physicians and other caregivers are pattern matchers by training.  They rely on their extensive knowledge base, built painstakingly over many years of training, practice and ongoing education, to identify key signals in a patient’s presentation that will point to the most likely diagnosis, treatment options and ongoing day to day management.

Now consider this.   Assuming, for the time being, that your doctor is an unaided human functioning in a way that is not chemically or digitally enhanced, then depending on the amount of sleep and associated daily stress levels, they are capable of weighing approximately seven variables at any one time when dealing with something pretty important – your health!   Of course, that might not be so bad if the number of variables that your doctor had to try to juggle at any one time were typically less than seven.  In truth, quite frequently it doesn’t even take seven variables to be able to safely determine the most appropriate next steps in weighing evaluation or treatment options.  However, on occasion, and as will become increasingly frequent, there are many, many more variables to consider in managing the care of an individual patient as well as the care of large panels of patients.

Add some other emerging dynamics to this basic human limitation and we can see that we have a problem on our hands.

First, the doubling time of medical knowledge is currently estimated to be less than ten years and continues to decrease.  Second, the volume of medical literature that a busy doctor needs to read to stay current keeps increasing and all too frequently different research reports present apparently contradictory evidence – frequently, no single treatment approach is suitable for all cases.  Finally, as in all other aspects of our life, the amount of useful interpretable health data being collected has started to grow at an explosive rate.

When you consider these factors, combined with the pressure to maintain quality while seeing increasing volumes of patients, it’s easy to understand why many doctors today feel that there has to be a different and more satisfying way to deliver the high quality care that they believe is possible.

The truth is that doctors, like all humans, are excellent pattern matchers but highly fallible complex information processors.  They are also not “perfectible” – we all make occasional mistakes, some more serious than others.  We tend to overly simplify complex issues and our recent personal experience weighs more heavily than it should when good evidence exists to the contrary.

The bottom line is that healthcare has to change to accommodate these dynamics.  What does the future look like?  Over the next series of blog posts, I’ll examine how big “big data” in healthcare actually is and how fast it’s growing, its implications to the medical profession and to healthcare more broadly and some predictions about how health analytics are poised to help physicians deal with this dilemma in the same way that smart GPS systems delight even the most experienced motorist.