Editor’s note: Members of IBM Watson’s algorithms team Dr. Bill Murdock and Dr. Aditya Kalyanpur will write about Watson’s answers to some of the Jeopardy! clues throughout the three-day rebroadcast, September 12-14.
Its largest airport is named for a World War II hero; its second largest, for a World War II battle.
Bill: Watson’s Final Jeopardy! response of “Toronto” received more attention than any other clue or moment in the game. So, I’ll let Dave Ferrucci’s response explain it, here.
The team also answered questions about Toronto and other “under the hood” questions about Watson in a Reddit Q&A.
It’s just a bloody nose! You don’t have this hereditary disorder once endemic to European royalty.
Aditya: While Ken beat Watson to the buzzer on this clue, Watson did correctly and confidently answer all five clues in this medical category. Even though all of the questions in this category had distracting phrases, such as, “it’s just a bloody nose” – which would normally confuse a machine – Watson figured out the crucial parts of the clues.
Note that Watson doesn’t always beat humans to the buzz:
- Watson takes between one-and-a-half to six seconds to answer a clue, so Ken and Brad have an edge with short clues,
- Some clues take longer to process. This clue had two sentences, so while it was of medium length, it may have taken Watson longer to determine what part of which sentence was most important.
- And Watson does have to push a buzzer, which takes time, too – Ken’s anticipation can beat Watson’s reaction.
Video: Watson in Healthcare
A Goya stolen (but recovered) in 2006 belonged to a museum in this city (Ohio, not Spain).
Bill: Watson did not realize the clue was asking for a city in Ohio that shares its name with a city in Spain. It answered “Madrid” since it found passage evidence that the Prado Museum in Madrid has had suspicions raised about the authenticity of its Goya paintings.
Negation is a very tricky topic to do well. The meaning of “Ohio, not Spain” is complicated; specifically it is asserting that theft happened in a city in Ohio and not in a city in Spain — but it is also strongly implying that the answer should be the name of a city in Ohio that is also the name of a city in Spain. That is how they expect people to answer this one; by thinking of a city in Ohio that has the same name as a city in Spain.
In business applications, we expect negated text (like “not Spain”) to be more frequent and more important than it is in Jeopardy! So, we’re continuing to work on improving the subtleties in how Watson handles negation-like statements.
Note that Watson’s confidence in “Toledo” was a close second with 26 percent.
Check out day one analysis.