Editor’s note: Members of IBM Watson’s algorithms team Dr. Bill Murdock and Dr. Aditya Kalyanpur will write about Watson’s answers to some of the Jeopardy! clues throughout the three-day rebroadcast, September 12-14.
Wanted for general evil-ness; last seen at the Tower of Barad-Dur; it’s a giant eye, folks. Kinda hard to miss.
Aditya: Watson solved this question in a way that we imagine humans do. It used “Tower or Barad-Dur” to infer that the clue related to the “Lord of the Rings” universe. Then from passage evidence, determined that Sauron was an eye. These pieces of evidence needed to be combined to arrive at the correct answer.
Watson learned through this category to connect something seemingly random – an eye in this case – to a character from literature. This clue is also a good example of how Watson connects information that wouldn’t be in a traditional database set up. Consider: would a typical dictionary contain the fact that Sauron – a character from fiction – was an instance of an eye?
It’s a 4-letter term for a summit; the first 3 letters mean a type of simian.
Bill: The answer Watson gave is a four-letter term for a summit, only satisfying half the clue. Watson did not understand the constraint that the first three letters mean something else – a word puzzle that Watson had not seen before. So, while Watson does have an anagram algorithm, it does not have a strategy for splicing a word.
The system still had high confidence in its answer since, based on past experience, it had learned that satisfying some constraints of the type mentioned in the clue usually mean a correct answer. This clue was a good example of how Jeopardy!-specific puzzles are not seen in real world data and conversation.
Video: Gender Identification
It was the anatomical oddity of U.S. gymnast George Eyser, who won a gold medal on the parallel bars in 1904.
Aditya: There was strong passage evidence stating that George Eyser’s left leg was made out of wood, but Watson didn’t understand what part of these passages was the oddity. This is good example of the broad domain of things that Jeopardy! asks about – it’s difficult to anticipate every type of concept the clues might ask about. So, we do not attempt to.
Rather, Watson tries to classify new things on the fly based on what it has read. With this clue, Watson tried to get sufficient evidence that a wooden leg was not any more of an oddity then a leg of flesh and bone – keep in mind that this was the first time Watson tried to understand an oddity.
Watson knows that correct Jeopardy! responses are more often physical objects like “leg” than abstract concepts like “he had a wooden leg,” so Watson chose “leg.” It was close, but the judges, and Mr. Trebek, didn’t buy it.
Video: Betting Strategy
Clue: His victims include Charity Burbage, Mad Eye Moody & Severus Snape; he’d be easier to catch if you’d just name him!
Bill: Ok, “Harry Potter” was not close, but Lord Voldemort was not far behind with a confidence of 20 percent. Watson had narrowed it down to two characters with the stronger association with the characters mentioned in the clue, and Harry won that analysis (Watson did not read the seven Harry Potter books.)
We humans understood that the category of APBs meant that a villain must be in the clue responses – but not Watson. Without knowing this answer type, Watson’s other chance to get this correct was to find passage evidence supporting these characters being victims of Lord Voldemort. Though Watson found passages saying, for example that “Voldemort murdered Charity Burbage,” Watson was not able to infer that “murder” implied “victim” with sufficient confidence.
Video: Learning Across Categories