Self tracking is business intelligence

Back when secretaries were common, you could have had yours track your day in 15-minute increments. In his book The Effective Executive, Peter Drucker suggested this as a way to find out what you really did all day. The picture usually wasn’t so pretty.

Tracking your time then and now is personal, it’s messy, and it’s the essence of business intelligence: collecting data and reading it for guidance in business activities that matter. Is there anything that matters more to an organization than productivity of its people? For a small office or home-based business, this might be the best BI there is.

This gets no recognition in the BI industry that I can find, at least not in the conservative world of TDWI. At least not yet.

PI — for “private intelligence” — has different issues, starting with data collection. In BI, data comes from transactions, all recorded routinely. In PI, most of it has to come from a “secretary” or from our own, tedious notation.

I dabbled in it once. The insights were good, if painful, but mostly it was tedious. A few years ago, a confluence of personal events let me do something I’d always wanted to try: hole up for a few months in a . …

Back when secretaries were common, you could have had yours track your day in 15-minute increments. In his book The Effective Executive, Peter Drucker suggested this as a way to find out what you really did all day. The picture usually wasn’t so pretty.

Tracking your time then and now is personal, it’s messy, and it’s the essence of business intelligence: collecting data and reading it for guidance in business activities that matter. Is there anything that matters more to an organization than productivity of its people? For a small office or home-based business, this might be the best BI there is.

This gets no recognition in the BI industry that I can find, at least not in the conservative world of TDWI. At least not yet.

PI — for “private intelligence” — has different issues, starting with data collection. In BI, data comes from transactions, all recorded routinely. In PI, most of it has to come from a “secretary” or from our own, tedious notation.

I dabbled in it once. The insights were good, if painful, but mostly it was tedious. A few years ago, a confluence of personal events let me do something I’d always wanted to try: hole up for a few months in a Sicilian village I knew slightly. The food was good, I had relatives nearby, and the nearby church bells rang all day and all night, four times an hour. At the same time, I had a book to edit. To stay productive, I made a game out of the work, tracking my time to the minute in Filemaker.

I liked the local food and started to hate the book, an office manual that inadvertently revealed a con game. Even so, I threw myself at it every day. But no matter how hard I tried, no full day ever resulted in more than about two hours of actual, productive work. My “quick breaks” for walks and coffee with a friend actually took up more time.

I made a Filemaker database because I could find no off-the-shelf product that would do anything close. Each period, no matter how short, had a starting and ending times I entered with buttons, and a calculation field figured the duration. A drop-down menu offered my usual activites. I could make a report for any period.

I thought some product would do that better, but I could find nothing. Then the May 2 issue of the New York Times Magazine ran an article by Gary Wolf about this, “The Data Driven Life.” My Filemaker invention wasn’t too far from what others have used, and now new devices are coming along that could make all that seem so old hat. Some people are even sharing their data on the cloud.

But as in traditional BI, the technology just gets you in the door. The show has just begun.

Most people Wolf writes about do it for personal reasons. One wanted to know how his coffee consumption helped him focus, another tried to cure his sleep apnea, and still another noticed that flax seed oil, or just lots of butter, improved his cognitive performance.

As in good BI, the experiments often raised new questions. And sometimes the new questions are unexpected, as in Wolf’s own experience.

Often, pioneering trackers struggle with feelings of being both aided and tormented by the very systems they have built. I know what this is like. I used to track my work hours, and it was a miserable process. With my spreadsheet, I inadvertently transformed myself into the mean-spirited, small-minded boss I imagined I was escaping through self- employment. Taking advantage of the explosion of self-tracking services available on the Web, I started analyzing my workday at a finer level. Every time I moved to a new activity — picked up the phone, opened a Web browser, answered e-mail — I made a couple of clicks with my mouse, which recorded the change. After a few weeks I looked at the data and marveled. My day was a patchwork of distraction, interspersed with valuable, but too rare, periods of focus. In total, the amount of uninterrupted close attention I was able to muster in a given workday was less than three hours. After I got over the humiliation, I came to see how valuable this knowledge was. The efficiency lesson was that I could gain significant benefit by extending my day at my desk by only a few minutes, as long as these minutes were well spent. But a greater lesson was that by tracking hours at my desk I was making an unnecessary concession to a worthless stereotype. Does anybody really believe that long hours at a desk are a vocational ideal? I got nothing from my tracking system until I used it as a source of critical perspective, not on my performance but on my assumptions about what was important to track.

I wish Drucker were around to respond. Wolf’s insight sounds like important stuff for everyday knowledge workers, especially those who work alone. What’s more important to a knowledge worker than time?

These experiments are often haphazard and highly personal.

Generally, when we try to change, we simply thrash about: we improvise, guess, forget our results or change the conditions without even noticing the results. Errors are possible in self-tracking and self-experiment, of course. It is easy to mistake a transient effect for a permanent one, or miss some hidden factor that is influencing your data and confounding your conclusions. But once you start gathering data, recording the dates, toggling the conditions back and forth while keeping careful records of the outcome, you gain a tremendous advantage over the normal human practice of making no valid effort whatsoever.

Yes, just as analytics gives companies a tremendous advantage over those who make less effort.

Let the BI traditionalists pooh-pooh self-tracking. The very same people might have dismissed such things as visual analysis, agile development, and at one time even business intelligence itself. Sometimes it take a few pioneers and geeks, perhaps even a secretary, to prove a concept.

Link to original post