A new legal aid. Advisor and mentor for young attorneys.

IBM ArgumentorPowered by the Debater Technology

Boaz Carmeli, IBM Research – Haifa

boazc@il.ibm.com

Agenda – things that I am intending to cover

� Short introduction and background: the Debater grand challenge

� Our technology focus: Pro-con analysis via Machine learning (deep learning) and NLP

� The Argumentor application: Pro-con analysis at the legal domain

� Argumentor: Short demo

� The university relations aspects: legal experts for requirements definition and legal text annotation

� Sumary and questions….

Background – The debater grand challenge

� The Debater Grand Challenge aims on developing technology that assist

humans to debate and reason...

� The Debater vision is that of an intelligent system able to take raw information and

digest and reason on that information, to understand the context, and to construct

arguments pro and con any subject…

� The Debater uses complex analytic pipeline a.k.a Argument Construction Engine (ACE)

3

Pro/Con Analysis� Goal:

– given two related sentences e.g.,

� A claim and an evidence or,

� A thesis and a fact

– determine for each pair:

� Whether the second supports the first (PRO) or contests it (CON)

� Example

– Claim:

� An evidence obtained from a search without warrant cannot be used in court

– Evidence:

� “The Fourth Amendment provides that "the right of the people to be secure in their

persons, houses, papers, and effects, against unreasonable searches and seizures,

shall not be violated.” (PRO)

� “It is frequently argued that in dealing with the rapidly unfolding and often dangerous

situations on city streets the police are in need of an escalating set of flexible responses,

graduated in relation to the amount of information they possess.“ (CON)

Deep Learning for Pro-Con Analysis

� Deep learning is a promising machine learning subfield that gains huge momentum lately

– Beats state-of-the-art algorithms in areas such as computer vison, speech recognition

and natural language processing

Sentence one Sentence two

Deep learning 2 i.e., Recurrent

Neural Net (RNN)

Deep learning 1 i.e., Recurrent

Neural Net (RNN)

Deep Learning layer 3 i.e., Classification

Pro-con results

Argumentor’s Target Professional

� Young attorney, less than 5 years of experience

� Usually takes a role of a Junior associate in mid-large law firms

� Chosen based on a market research by Watson group:

6

http://ace1.haifa.ibm.com:8080/arguMentor/?tryme

Argumentor Demo

Web client

supremecourtdatabase.org Supreme court decisions 1937-1975 (FLITE)

Keyword andConcept

extraction

Annotation tool

Annota

tionArgument detection

Debater claim detection service

ActiveLearningvia Exemplar

Clustering

Annotation management

Alchemy Langage

NLClassifierArgumentclustering

Argumentclassification

Solution Architecture

8

IBMArgumentor

Cognitive Elements� Argument Construction

– Deep natural language processing - Argumentor processes legal cases for

detecting arguments, based on technology from the Debater grand challenge

� Natural language input

– Argumentor receives a short case brief as its input

� Interactivity

– Argumentor integrates in current workflow of the legal professional

� Processes email as input

� Allows human feedback at each step

� Learns argument classification based on several examples from the user

� Helps the user focus on what is of her interest by interactive highlighting

� Supervised Machine Learning

– For this hackathon, we annotated sample data of real legal cases to teach the

computer to classify and evaluate arguments

9

University Relations Aspects

� Won a “Legal Argumentation Structuring and Gamification” UR country project

� Gave two lectures at the “Institute for Legal Implications of Emerging Technologies”

program at Interdisciplinary Center, (IDC) Herzliya, Israel

� Instructing a group of 4 students on ‘Legal and AI’ project

– Legal csaes annotation guidelines

– User requirements

� Search collaboration with Robert-Jan Sips and the Netherlands academy

– Application for the IBM University Programs to fund a PhD fellowship for crowdsourcing

and nichsourcing research in the legal domain

– Investigating annotation platforms such as Crowdflower and Watson Knowledge Studio

(WKS)

Summary

� Debater provides leading technology for argument construction

� Argumentor is a cognitive application that assists young attorney in her legal

research tasks

– Based on argument construction pipeline, namely claim detection and pro-con

analysis

� Collaboration with law schools and university provides a jump-start into this

complex domain

Hackathon Team

Thanks to:

IBM Research -Haifa

•Machine Learning Technologies group

•Medical Imaging Analytics

Watson Emerging Products Design

Watson Implementations

12

Backup

Backoffice work

� Downloaded 7,500 US Supreme Court cases from 1937-1975

� Used ACELab and GATE to annotate legal data

– Annotated by our legal professional on board – a lawyer team member from Watson

Implementations

� Created three exemplary use cases

� Conducted search for relevant cases for all three cases

� Annotated arguments from Supreme Court cases

– Created guidelines for annotation of legal argument

� Trained NLClassifier based on winning and losing arguments

� Downloaded structured data of US Supreme court cases http://supremecourtdatabase.org/

� Preprocessed all 7,500 cases by running the Debater pipeline offline due to performance

considerations

14

IBM Argumentor Flow - Input

� Attorney pastes an email or brief

that describes his case

� Argumentor uses AlchemyAPI to

extract keywords and concepts

� Argumentor reranks and filters

results to keep only relevant legal

concepts

� Attorney can add and remove

keywords

15

IBM Argumentor Flow – Argument Construction

� Argumentor uses Debater’s ACE

to construct arguments

� Argumentor augments arguments

data with case data

� Attorney can look at the relevant

cases Argumentor found

� Attorney can focus on arguments

using facets based on structured

data

16

IBM Argumentor Flow – global cases view

� Argumentor presents relevant

cases taxonomy in a treemap

� Argumentor presents wining and

losing parties statistics

� Attorney can interact with views so

Argumentor will focus on certain

arguments

17

IBM Argumentor Flow – cases view

� Argumentor presents relevant

color coded cases in a list

� Argumentor provides a case view

in a glance:

– Case relevancy

– Case characteristics

– Case taxonomy

� Attorney can interact with views so

Argumentor will focus on

arguments from a specific case

18

IBM Argumentor Flow – arguments pro/con analysis

� Argumentor presents arguments

sorted by relevancy

� Argumentor provides a quick link

between an argument and its case

� Attorney can place a few

arguments in folders - arguments

that support his client or against

him.

� Argumentor learns to classify the

rest of the arguments automatically

19

� https://wpncatalog.stage1.mybluemix.net/assets/assets_debater_claim_detection_service

� https://wpncatalog.stage1.mybluemix.net/assets/assets_activelearning_via_exemplarcluster

ing

20