The excerpt in the Wall Street Journal sparked some doubts and concerns among readers. I responded to a few, and figured I’d blog them here.
Henry Grimmelsman wrote:
The excerpt in the Wall Street Journal sparked some doubts and concerns among readers. I responded to a few, and figured I’d blog them here.
Henry Grimmelsman wrote:
I think the computer will win. He who pays the piper calls the tune, and IBM is bankrolling it. I think the tiny details of how information is relayed to the computer will probably be on terms favorable to the computer. The way I remember the chess competition with Kasparov, Kasparov started dominating the computer early on (in a series of games), and the grandmasters and programmers working on the project got to reprogram it. I didn’t think this was fair. I thought the computer should have to play the whole series with its original program. But IBM sponsored it, and who knows how much Kasparov got for participating, so he was a gracious loser.
The most obvious advantage for the computer is that it only has to worry about taking on humans. Each human is taking on not only the computer, but another human. I’d like to see a contest between one of the human players, the IBM machine, and a machine designed by Google. I can’t help but wonder if a computer that somehow produced answers based on Google queries could beat what IBM programmed.
It’s still a pretty amazing project to build the computer, but I can’t imagine IBM turning it loose without knowing it would win.
My response:
Henry, One of the points of contention between IBM and Jeopardy during this process was whether IBM and Watson would have access to top Jeopardy talent for a series of test matches. The IBM team wanted to establish a scientific record of Watson’s performance against humans. And this would provide some backing (and bragging rights) in the event that Watson lost the televised showdown. As in most games, one single match in Jeopardy is unpredicatable. It often boils down to who lands on Daily Doubles. So IBM eventually provided tournament of champion-caliber players to take on Watson in 56 matches through the Fall. Watson won nearly 7 of 10 of these matches. Its greatest vulnerability was in Final Jeopardy, where the clues are usually more complex. One more point about a …quot;sure thing…quot; match for IBM. In order for it to be a sure thing, Watson would have to be such a dominant player that it would render the TV show tedious. Jeopardy was not interested in that.
As far as your point about the computer taking on two humans, instead of vice versa, that’s well taken. Going into the match, both Jennings and Rutter said they would have preferred facing two Watsons rather than one Watson and a supremely talented fellow human. That said, humans below the caliber of Jennings and Rutter did defeat Watson numerous times in recent months. The machine has its vulnerabilities.
I forgot to respond to his point about “details about how the information is relayed to the computer.” There haven’t been any complaints about that. The computer gets the electronic words at the same moment that the humans see the clue. In David Ferrucci’s words to human players: “As soon as it hits your retina, it hits Watson’s chips.” The real arguments centered on the buzzer–a crucial factor in Jeopardy. There was a battle over that, as I describe in the book, and Jeopardy eventually prevailed upon IBM to build “a finger” so that Watson could physically press the button, just like humans. That said, humans in their hurry to press the button occasionally buzz too early–and get penalized a precious quarter second. Watson, responding to the cue to buzz, never gets penalized.
Rich Gibbs wrote:
The “Interactive Graphic” contest is a bit unfair — it makes the game too easy, by using (essentially) a multiple-choice format. Often, a fair chunk of the difficulty in finding an answer, at least for a human contestant, is coming up with a list of eligible answers. Take, for example, a recent clue from the show, in “African Geography” (this is from memory): This West African country’s name appears in the name of an East African country. The answer is “Mali”, which appears in “Somalia.” Not that hard, but if you can’t remember the country names, you can’t even get started.
I agreed:
If Watson could play Jeopardy by selecting multiple choice answers, it would probably never miss one. The challenge for the machine (and for the rest of us) is to find a single response in a universe of information.