Founder compares his Wolfram-Alpha to Watson

Stephen Wolfram, founder of Mathematica computing software and the Wolfram-Alpha knowledge engine, takes a fascinating look at the future of search and knowledge–and where a computer like IBM’s Watson fits in. It doesn’t make sense to rehash his post. Better just to read it.

Stephen Wolfram, founder of Mathematica computing software and the Wolfram-Alpha knowledge engine, takes a fascinating look at the future of search and knowledge–and where a computer like IBM’s Watson fits in. It doesn’t make sense to rehash his post. Better just to read it.

The most interesting aspect to me is his thinking about the future organization of knowledge. No matter how it’s done, it requires human intelligence and input. The question is where the humans get involved. …quot;Somewhere you have to inject human expertise,…quot; Wolfram told a gathering at MIT in September. …quot;You can’t take humans out of the loop….quot;

In a search engine like Google, humans produce their content willy nilly on the Web, and its up to the technology to find order in it. Most of the brainpower works for the search engine (with the exception of the search-engine optimizing crowd, which tries to customize its Web pages so that the engines can find them.)

Watson, in its Jeopardy incarnation, studies a much smaller set of data–about 75 gigabytes. Some of the data is preprocessed, and a lot of thought has gone into which documents provide the machine with the best chance to nail Jeopardy clues. So if the data pouring into Google is like a jungle, Watson’s trove is closer to a game preserve. But still, most of the effort to build Watson went into the algorithms to analyze the data–and much less into preparing the data.

Wolfram believes in an expanding world of processed, …quot;computible…quot; knowledge. He has a team converting big data sets into a format that his knowledge machine can make sense of. In that sense, it’s a far cry from Google. And of course, unlike search engines, which simply point us toward answers, Wolfram-Alpha carries out computations on the data and provides answers.

Wolfram sees this curating process spreading across the realm of human knowledge, as people work to make their data comprehensible for computers. In recent decades, he said at MIT, people learned the value of turning paper documents into digital ones, which could then be shared across networks. In the next transition, he predicted, people will learn the value of making their documents …ldquo;computable….rdquo; This will mean formatting them so that a machine can read them, draw conclusions and answer questions based on the content. …ldquo;Anything that is not computable,…rdquo; he said, …ldquo;will seem marooned as data on paper does today….rdquo;

Incidentally, in the blog post, Wolfram discusses how his team fed 200,000 Jeopardy clues straight to search engines. The IBM team carried out the same experiment. Some 20-25% of the clues led to answers in the search results. According to Wolfram’s stats, close to 2/3 of the answers could be found in the first document by the search engines. (Of course, it’s human intelligence that ferrets out that answer from the document, and calculates its confidence in it. Watson has to do that work by itself. As I wrote today on IBM’s Smarter Planet blog, a key aspect of Watson’s intelligence is its confidence gauge in its answers.)