The New Predictive Profession: Odd Yet Newly Legitimate [BOOK REVIEW]

EricSiegel March 27, 2013 March 27, 2013

479 Views

• Share on Facebook
• Tweet on Twitter
• +

Review of Predictive Analytics – The Power to Predict Who Will Click, Buy, Lie, or Die, by Eric Siegel

Predictions have a problem.  They are viewed as either magic or “snake-oil” by most people.  It doesn’t help that many previous predictors in society like Nostradamus and Edgar Cayce were viewed somewhat askance at best, and as charlatans at worst.  Only recently has the making of predictions gained some legitimacy, and this is due to the recent rise of predictive analytics in many sectors of our society.  This rather odd profession has developed out of the science of Artificial Intelligence, which seeks to capture, however crudely, some of the intelligent predictive processing capability of human brains, and express it mathematically in computer programs.  Initially, predictive analytics (aka data mining) lived in the rarefied atmosphere of academics or highly-paid consultants.  The challenge for predictive analytics scientists and professionals is to recast the subject in a form that is “mapped” more closely to common perceptions of what we do in our brains and why we do it.  Eric Siegel has done it!

The human brain is a tricky thing to understand.  I was trained initially as a Biologist, which exposed me to the view that the human brain is the most complex non-linear pattern processing system in the universe.  Much of its function is devoted to prediction and decision-making, and much of it is done unconsciously. One of the biggest challenges confronting scientists of the brain is to explain its function in terms and expressions that everybody understands.  It is people with brains that do things.  Eric Siegel’s book helps to explain some very fascinating aspects of why people do what they do, in a very engaging way. 

He arrays the topics in the book around five “effects” of prediction: (1) the prediction effect; (2) the data effect; (3) the induction effect; (4) the ensemble effect; and (5) the persuasion effect.  To explain the nature and significance of these effects, what does he do?  He tells stories. Everybody likes stores. The most popular book ever written (the Bible) is basically a story book.  One of his extended stories is about John Elder, who is near and dear to my heart (many among his students and audiences think so to).  Eric couches a number of predictive analytics truisms in terms of how John Elder learned them, such as stumbling over the error of “leakage from the future”.  John loves to wax eloquently on that mistake in his presentations of “The Top 10 Data Mining Mistakes”.  Another extended story is about the Watson supercomputer that won over 2 human competitors on the TV show, “Jeopardy”.  Eric explains how Watson did it, using the ensemble effect.  An ensemble uses many mathematical techniques (algorithms) to predict an outcome, and then combines them to compose an overall prediction.  Watson does not just use ensembles, Eric explains that  its processing architecture consists of “an ensemble of ensembles of ensembles”.  That complexity would hurt my head, if Eric had not brought it down to earth in his explanation of what Watson is and how it works.

 The third extended story is about… (who else) Barack Obama.  Obama set up a team of data miners (as they were called then) in 2011, to be based in Chicago, and tasked with the challenge to leverage data mining technology to further his election campaign.  When I saw the many ads for these data mining professionals in several online job posts, I thought, “Watch out, Republicans; he’s going to eat your lunch”.  And, he did. Eric explains how Obama’s predictive analytics team predicted those “swing voters” who had the greatest likelihood of being influenced to vote for Obama.  Then, they used data from social media, like Twitter and Facebook, to predict which people were strong influencers of the swing voters; they targeted them, not the swing voters themselves (an example of the “Persuasion Effect”).  That approach is at the very cutting edge of predictive analytics today, largely because of Obama’s election campaign.  And Eric’s presentation of it makes you think, “Well, duh… of course”!

Eric Siegel has brought predictive analytics down from the intellectual stratosphere  where most scientists and engineers dwell, and expressed it in terms that anyone can understand, and vended it in the form of a bunch of stories.  This book should be your first predictive analytics book on your bookshelf, or to give to clients and friends when they ask, “So, what do you do”?  That is the question that Eric poses in the Preface to the book, and then he marshals his stories in the rest of the book to answer it.

In the University of California at Irvine Predictive Analytics Certification Program (where I teach), we require our own book (“Handbook of Statistical Analysis & Data Mining Applications”, R. Nisbet, John Elder, and Gary Miner, 2012).  NOW, we will require Eric Siegel’s book also, and direct students to read it first!  You should too.

Bob Nisbet, Ph.D.
Consulting Data Scientist
Instructor, UC-Irvine Predictive Analytics Certification Program

Bio:

Bob was trained initially in Ecology and Ecosystems Analysis.  He has over 30 years experience in complex systems analysis and modeling.  Currently, he teaches several courses in predictive analytics in the University of California – Irvine Predictive Analytics Certification Program.