Decision Management in the New York Times

There was an interesting article in the New York Times this weekend called Smarter than you think focused on e-discovery and the use of computers rather than hordes of lawyers. Two things strike me as interesting about this article.

The first is that it is a classic example of the power of beginning with the power in mind. E-Discovery of this kind might, almost certainly will, involve text analytics as discussed in the article. But it might also involve network analysis (to see how is connected to whom to find fraud rings or collaborators for instance), predictive analytics to see how likely it is that something is true about a transaction or company, and rules defined by experts (in this case lawyers) or by the boundaries of the case. Beginning with the decision in mind will ensure that the right mix of things are applied rather than simply asserting “this is e-discovery therefore we will use text analytics” which runs a risk that data in a database or other approaches will be overlooked.

The other is that, while the focus of the article was on the reduction in staff needed, it fails to account for the massive growth in documents and data available in these cases. Without new tools, e-discovery could have been headed for what I call the “telephone operator event horizon” – the point at which e-discovery employed everyone because of the massive expansion in documents being processed (this is named after the famous prediction that expansion in telephone service would mean that half the population had to become a telephone operator – a prediction overtaken by the automated telephone exchange). Automation of previously expert decisions often shows this pattern:

Experts make decisions
The volume of these decisions being required begins to expand forcing a consideration of approaches to reducing the cost of the decision
The data needed for the decision is increasingly available in electronic form
This automation succeeds and in turn triggers an event larger increase in demand for the kind of decision being automated

Yes automation of decisions sometimes reduces the need for staff. Much more often it innovates and allows companies to apply the same staff to more problems by replacing boring, mechanical work with more interesting, more difficult work that is hard to automate or where automation is not desirable.