Final Jeopardy: Man vs. Machine and the Quest to Know Everything

Introduction

Watson paused. The closest thing it had to a face, a glowing
orb on a flat-panel screen, turned from forest green to
a dark shade of blue. Filaments of yellow and red streamed
steadily across it, like the paths of jets circumnavigating the
globe. This pattern represented a state of quiet anticipation
as the supercomputer awaited the next clue. It was a September
morning in 2010 at IBM Research, in the hills north of
New York City, and the computer, known as Watson, was annihilating
two humans, both champion players, in practice
rounds of Jeopardy! Within months, it would be playing the
game on national television in a million-dollar man vs. machine
match against two of Jeopardy ’s all-time greats.
 As Todd Crain, an actor and the host of these test games,
started to read the next clue, the filaments on Watson’s display
began to jag and tremble. Watson was thinking — or coming
as close to it as a computer could. The $1,600 clue, in the category
The Eyes Have It, read: “This facial ware made Israel’s
Moshe Dayan instantly recognizable worldwide.”
 The three players — two human and one electronic — could
read the words as soon as they appeared on the big Jeopardy
board. But they had to wait for Crain to read the entire clue
before buzzing. That was the rule. As the host pronounced
the last word, a light would signal that contestants could buzz.
The first to hit the button could win $1,600 with the right answer
— or lose the same amount with a wrong one. (In these
test matches, they played with funny money.)
 This pause for reading gave Watson three or four seconds
to hunt down the answer. The first step was to figure out what
the clue meant. One of its programs promptly picked apart
the grammar of the sentence, identifying the verbs, objects,
and key words. In another section, research focused on Moshe
Dayan. Was this a person? A place in Israel? Perhaps a holy
site? Names like John and Maria would signal a person. But
Moshe was more puzzling.
 During these seconds, Watson’s cognitive apparatus —
2,208 computer processors working in concert — mounted a
massive research operation through thousands of documents
around Moshe Dayan and his signature facial ware. After
a second or so, different programs, or algorithms, began to
suggest hundreds of possible answers. To us, many of them
would look like wild guesses. Some were phrases that Dayan
had uttered, others were references to his military campaigns
and facts about Israel. Still others cited various articles of his
clothing. At this point, the computer launched its second
stage of analysis, figuring out which response, if any, merited
its confidence. It proceeded to check and recheck facts, making
sure that Moshe Dayan was indeed a person, an Israeli,
and that the answer referred to something he wore on his face.
 A person looking at Watson’s frantic and repetitive labors
might conclude that the player was unsure of itself, laughably
short on common sense, and scandalously wasteful of com-
puting resources. This was all true. Watson barked up every
tree from every conceivable angle. The pattern on its screen
during this process, circles exploding into little stars, provided
only a hint of the industrial-scale computing at work. In a
room behind the podium, visible through a horizontal window,
Watson’s computers churned, and the fans cooling them
roared. This time, its three seconds of exertion paid off. Watson
came up with a response, sending a signal to a mechanical
device on the podium. It was the size of a large aspirin bottle
with a clear plastic covering. Inside was a Jeopardy buzzer.
About one one-hundredth of a second later, a metal finger inside
this contraption shot downward, pressing the button.
 Justin Bernbach, a thirty-eight-year-old airline lobbyist
from Brooklyn, stood to Watson’s left. He had pocketed
$155,000 while winning seven straight Jeopardy matches in
2009. Unlike Watson, Bernbach understood the sentence. He
knew precisely who Moshe Dayan was as soon as he saw the
clue, and he carried an image of the Israeli leader in his mind.
He gripped the buzzer in his fist and frantically pressed it four
or five times as the light came on.
 But Watson had arrived first.
 “Watson?” said Crain.
 The computer’s amiable male voice arranged the answer,
as Jeopardy demands, in the form of a question: “What is eye
patch?”
 “Very good,” Crain said. “An eye patch on his lefteye.
Choose again, Watson.”
 Bernbach slumped at his podium. This match with the
machine wasn’t going well.

It was going magnificently for David Ferrucci. As the chief scientist
of the team developing the Jeopardy computer, Ferrucci
was feeling vindicated. Only three years earlier, the suggestion
that a computer might match wits and word skills with human
champions in Jeopardy sparked opposition bordering on
ridicule in the halls of IBM Research. And the final goal of
the venture, a nationally televised match against two Jeopardy
legends, Ken Jennings and Brad Rutter, seemed risky to some,
a bit déclassé to others. Jeopardy, a television show, appeared
to lack the timeless cachet of chess, which IBM computers
had mastered a decade earlier.
 Nonetheless, Ferrucci and his team went ahead and built
their machine. Months earlier, it had fared well in a set of
test matches. But the games revealed flaws in the machine’s
logic and game strategy. It was a good player, but to beat Jennings
and Rutter, who would be jousting for a million-dollar
top prize, it would have to be great. So they had worked
long hours over the summer to revamp Watson. This September
event was the coming-out party for Watson 2.0. It was
the first of fifty-six test matches against a higher level of competitor:
people, like Justin Bernbach, who had won enough
matches to compete in Jeopardy ’s Tournament of Champions.
 In these early matches, Watson was having its way with
them. Ferrucci, monitoring the matches from a crowded observation
booth, was all smiles. Keen to promote its Jeopardy
phenom, IBM’s advertising agency, Ogilvy & Mather, had
hired a film crew to follow Ferrucci’s team and capture the
drama of this opening round of championship matches. The
observation room was packed with cameras. Microphones on
long booms recorded the back-and-forth of engineers as they
discussed algorithms and Watson’s response time, known as
latency. Ferrucci, wearing a mike on his lapel, gave a blow-byblow
commentary as Watson, on the other side of the glass,
strutted its new and smarter self.
 It was almost as if Watson, like a person giddy with hubris,
was primed for a fall. The computer certainly had its
weaknesses. Even when functioning smoothly, it would make
its share of wacky mistakes. Right before the lunch break,
one clue asked about “the inspiration for this title object in
a novel and a 1957 movie [which] actually spanned the Mae
Khlung.” Now, it would be reasonable for a computer to miss
“The Bridge over the River Kwai,” especially since the actual
river has a different name. Perhaps Watson had trouble understanding
the sentence, which was convoluted at best. But
how did the computer land on its outlandish response, “What
is Kafka?” Ferrucci didn’t know. Those things happened, and
Watson still won the two morning matches.
 It was after lunch that things deteriorated. Bernbach,
so frustrated in the morning, started to beat Watson to the
buzzer. Meanwhile, the computer was making risky bets and
flubbing entire categories of clues. Defeat, which had seemed
so remote in the morning, was now just one lost bet away. It
came in the fourth match. Watson was winning by $4,000
when it stumbled on this Final Jeopardy clue: “On Feb. 8,
2010, the headline in a major newspaper in this city read:
‘Amen! After 43 years, our prayers are answered.’ ” Watson
missed the reference to the previous day’s Super Bowl, won by
the New Orleans Saints. It bet $23,000 on Chicago. Bernbach
also botched the clue, guessing New York. But he bet less than
Watson, which made him the first person to defeat the revamped
machine. He pumped his fist.
 In the sixth and last match of the day, Watson trailed Bernbach,
$16,200 to $21,000. The computer landed on a Daily
Double in the category Colleges and Universities, which
meant it could bet everything it had on nailing the clue. A
$5,000 bet would have brought it into a tie with Bernbach. A
larger bet, while risky, could have catapulted the computer toward
victory. “I’ll take five,” Watson said.
 Five. Not $5,000, not $500. Five measly dollars of funny
money. The engineers in the observation booth were stunned.
But they kept quieter than usual; the cameras were rolling.
 Then Watson crashed. It occurred at some point between
placing that lowly bet and attempting to answer a clue about
the first Catholic college in Washington, D.C. Watson’s “front
end,” its voice and avatar, was waiting for its thousands of
processors, or “back end,” to deliver an answer. It received
nothing. Anticipating such a situation, the engineers had prepared
set phrases. “Sorry,” Watson said, reciting one of them,
“I’m stumped.” Its avatar displayed a dark blue circle with a
single filament orbiting mournfully in the Antarctic latitudes.
 What to do? Everyone had ideas. Maybe they should finish
the game with an older version of Watson. Or perhaps
they could hook it up to another up-to-date version of the
program at the company’s Hawthorne labs, six miles down
the road. But some worried that a remote connection would
slow Watson’s response time, causing it to lose more often on
the buzz. In the end, as often happens with computers, a reboot
brought the hulking Jeopardy machine back to life. But
Ferrucci and his team got an all-too-vivid reminder that their
Jeopardy player, even as it prepared for a debut on national
television, could go haywire or shut down at any moment.
When Watson was lifted to the podium, facing banks of cameras
and lights, it was anybody’s guess how it would perform.
Watson, it was clear, had a frighteningly broad repertoire.