Image for Elementary, My Dear Watson

IBM’s computer goes to (tax) law school, start-up Blue J Legal launches, and artificial intelligence is about to make legal research easier

By Alec Scott, LLB 1994 | Illustrations by Sonia Roy

From the Spring/Summer 2015 issue of Nexus

The IBM computer that easily defeated Jeopardy! champion Ken Jennings is moving on to less trivial pursuits or at least more potentially marketable ones. Watson has become a whiz at oncology diagnosis, helping doctors determine and treat what ails their cancer patients. It has a cookbook out, with recipes featuring supposedly palatable mixes of seldom-combined ingredients.

Because being a chef and doctor’s aide isn’t quite enough, Watson is now studying the law. With IBM’s blessing, some U of T professors and students are instructing Watson in tax law, in the hopes that it might, some day soon, become a trusty sidekick for lawyers, accountants and tax advisers. “We didn’t just want to teach Watson the black-and-white questions, the many straightforward rules of tax law,” says Ben Alarie, JD 2002, associate professor and the Faculty of Law’s former associate dean for the first-year program. He’s one of the interdisciplinary project’s leaders. “Watson would be wasted on that. What we wanted, instead, was to have it address grey areas.”

In the space of one hectic academic year, a team from the Department of Computer Science and the law school has been helping Watson get up to speed on the tricky distinction in tax law between employees and independent contractors. “I was skeptical coming in,” another team member, Assistant Professor Anthony Niblett, says. “It was like teaching a child at first—and we’re still learning how to teach it. But properly instructed, it assimilates rapidly. Now I would say I’m optimistic about the potential here, cautiously optimistic.”

Indeed, the progress to date has been promising enough that a start-up is coming out of this process, Blue J Legal—Blue for IBM and for U of T, ‘J’ for justice, the short-form for judges in reported decisions.

For the humans working with Watson, the ride to date has been headlong. The students and professors have had the rare chance to grapple with this early form of artificial intelligence (AI), to gauge Watson’s strengths and weaknesses. They’ve taught it about one area of tax law, but they’ve learned from it also, all while wrestling with one big existential question—will we humans become, at some point, superfluous?

Watson was built in part on lessons learned by IBM researchers in developing Deep Blue, the chess computer that famously lost to, then beat then-world champion Garry Kasparov in the 1990s. But Watson has gone further than its predecessor, learning to understand English, in all its foreigner-baffling complexity, with its byzantine sentence structures, its subordinate clauses, its synonyms and words with multiple, sometimes conflicting meanings. Language is “the holy grail,” Watson’s trainer-in-chief Dave Ferrucci is reported to have said, “The reflection of how we think about the world.” In preparation for Jeopardy! , it was taught through questions and answers—all of the game-show’s past questions were run through its system, as were many reliable reference works.

For all its quickness on the uptake, Alarie noticed that it lacked one thing that many humans have: “Intuition—this way of applying the breadth of our human experience to the case law."

Last summer, IBM approached 10 of North America’s top computer science faculties to propose that they help Watson master different areas of human lore as part of an effort to commercialize Watson’s skill-set. U of T was the sole Canadian school on the short-list, and Mario Grech, a director of the Department of Computer Science Innovation Lab, had student teams come up with drafts of AI products, complete with business models and business plans, all of them in the field of law. “With law such a language-driven field, we thought this would be a good test for it,” Grech says. He roped in Alarie to help judge his students’ end-of-term presentations in December.

The team pitching a legal research tool won out over those targeting immigration and family law, and Grech was able to help the students further polish their presentation in advance of a competition held, in January, against the other nine schools at an IBM office tower in New York. This office focuses on promoting Watson, and it showed the team a film of Watson helping a doctor to diagnose a child with the rare Kawasaki Disease. “Watson is already proving a great aid to doctors,” Alarie says. “It is promising for lawyers because it can add confidence to their decision-making.”

The U of T team ultimately placed second, but impressed executives at IBM enough that they were willing to allow the team continued access to Watson.

“We began feeding it, what they call a ‘corpus’ of information,” Alarie says. “In our case, a body of tax decisions, law review articles, statutes.” At first, according to Alarie, teaching Watson the law was slow going. He, Niblett, Professor Albert Yoon and joint JD/MBA student Ramin Wright would read the cases, feed them in, then ask Watson questions about the holdings, correcting wrong answers as they went along.

The method, as Alarie describes it, is surprisingly similar to the traditional Socratic method of instructing beginner law students. “Just as in first-year classes, we would change the fact-situation,” Niblett says. “What about if this set of facts were the case? How does that change the decision-making?” Although Watson doesn’t yet know how to accord more authority to higher courts than lower ones, it has already learned how to provide pretty reliable answers to the question of whether a certain worker is, for tax-law purposes, an employee or an independent contractor.

“With a law student,” Niblett says, “You have to be careful of the amount of reading you give. Not so with Watson. Once we learned a bit how to teach it, the speed with which it assimilated the information was extraordinary.” (Like Watson, Niblett has competed on Jeopardy! , bringing a respectable US$19,601 home from a sole victory scored on Christmas Day 2013. “I watched Watson play Jeopardy!—it was just beyond belief.”)

Alarie comments: “Watson has this bandwidth that we just don’t have. With a complicated multivariable test—like the one courts use in the employee-contractor issue—it can consider everything at once.” The trained Watson, like an adept articling student, tells you which case is closest to being on point, as well as others that are nearly there.

For all its quickness on the uptake, Alarie noticed that it lacked one thing that many humans have: “Intuition—this way of applying the breadth of our human experience to the case law. Often, with much less material, we can extrapolate; we can communicate an accurate sense of the law. We do more with less.” In the short term, the plan is to teach Watson to handle other thorny tax-law distinctions: for example, the difference between capital gains and income, and between current expenses and capital expenditures. In the long term, the team hopes to set the diligent Watson studying other legal areas.

Alarie sees some potential for Watson-like AI applications to increase the ability of people of average means to access affordable legal advice. He’s a longtime proponent of democratizing legal knowledge and, for years, he maintained a website annotating Canada’s tax statutes with the holdings of many pertinent cases. “It sometimes would get as many as 50,000 unique visitors a month,” says Wright, a former summer research assistant of Alarie’s. “The difference with Watson is that you can update its knowledge more easily: just feed in the new material.”

A recent assembly of AI experts in Puerto Rico predicted that, by 2050, computers will be able to do everything important better than we do, including skills we think of as particularly human ones—that they’ll, for instance, drive better, diagnose illnesses better, write better, cook better, even practice law better. Although impressed with Watson, the law school’s team is not entirely convinced.

“I think what happened with the chess example is instructive,” Wright says. “They found that the human player taking advice from the machine was the strongest combination, stronger than either of them alone.” Blue J’s tagline—“Making Professionals Better”—echoes this pro-cooperation sentiment, this idea that the two, human and machine, can work well together.

Alarie voices another argument for our continuing relevance: “Providing solid advice is not the only thing lawyers do. Having a good advocate by your side can bring immense psychological comfort. A strong, sympathetic lawyer can explain what’s happening to clients in terms they can understand. That’s simply not something Watson is equipped to do.”