Where Data Analysis is a Nightmare

There are the dream organizations that deploy data analysts wisely. Then there are the nightmares, such as the I.R.S. as portrayed in David Foster Wallace’s last novel, The Pale King, reviewed yesterday in the New York Times.

… In a universe of veiled and veiling numbers, the task of drawing the true [data] out into the light and holding them up for inspection, clear and remainder-less, really is a sacred one. … The problem, as I.R.S. recruits soon discover, is that neither moral nor heroic codes hold true anymore.

These recruits work with “excruciating difficulty … in an age of data saturation.”

The [instructor] presents “the world and reality as already essentially penetrated and formed, the real world’s constituent info generated . . . now a meaningful choice lay in herding, corralling and organizing that torrential flow of info.”

One character is the data psychic, Sylvanshine, who can “glean trivia about anyone simply by looking at him.” But, as if to prove that good data is far from the end of the story, he has a problem.

[He] is “weak or defective in the area of will.” Nor, due to endless digressions, can he complete anything. No one can; in “The Pale King,” nothing ever fully happens. That this is to a large extent a metaphor … becomes glaringly obvious when we hear one unnamed character describe the play he’s writing, in which a character sits at a desk, doing nothing; after the audience has left, he will do something — what that “something” is, though, the play’s author hasn’t worked out yet.

Let’s see, will an “easy to use,” “speed of thought” tool help? Is there a tool for Sylvanshine and the others?

No, at least not until the next update. But this is why business intelligence is fascinating. Under cover of a Macguffin — the tools and the data — we touch the heart — throbbing or dead — of the organization.