Question Answering in the Business Environment

Yael Ravin

ravin@us.ibm.com

T. J. Watson Research Center, IBM

Knowledge Management• "Know what you know/don't know"• Involving both people and IT

– knowledge sharing culture (communities)– tools to enable access, processing, authoring and

dissemination

• Inside and outside the enterprise– B2E - increase productivity; training; support innovation – B2C - online shopping; online support

• Within and across industries– B2B - dynamic marketplaces.

KM is a big market

• IDC & Knowledge Management Magazine survey (01/01):

• about 30% of IT budgets was allocated for KM in 2001

• IDC WW KM Market forecast:1999 2003

Software $1.4B $5.4BKM Access software

$0.5B $3.3B

Infrastructure $0.9B $2.0BManagementServices

$1.3B $10.2B

KM Challenges

• Improve support for information seeking– search, question answering, dialog systems– classification and visualization

• Identify useful information– recommendation systems, link analysis, personal

profiles

• Reduce information overload– filtering, summarization

• Develop new ways to capture knowledge– phone, meetings, presentations

Portals: a KM path to market

Provide a secure single point of access to diverse information, business processes, and expertise personalized to a user's needs and responsibilities anywhere and at anytime

ApplicationsInformation Collaboration

Corporate PervasiveCommerce

A Simplified ‘one stop shopping’ Access to tools and information

One view of multiple data sources

A place to "Push" the latest information targeted to the practitioner

The latest documents crawled,

categorized and indexed every night

Claim DataProposalsRatings

Engineering Documents

U/W TemplatesLossesOSHASAFERNews

... A place to encourage collaboration and deeper team involvement

Communities:Mobile worker, Home Office, professional interest groups

Lotus Knowledge Discovery System

NLP/IR Technologies in Portals today

• Search engine• Document classification• Document clustering• Summarization• Information Extraction• Quality metrics• Subscription and notification based on profile

Integrate QA in KM and portals

• Work within different business domains and needs:– shopping, expertise locator, business intelligence, ...

• Integrate with other functionalities– a generic search engine

– a dialog system

• Operate with multiple sources of information:– databases, text, presentations

Online Shopping

Source: Forrester Report "Must Search Stink?" June 2000

58.3%Extremely Difficult

Difficult

Barely Acceptable

Easy

Extremely Easy

0 10 20 30 40 50

(N=350)Percent respondents

Finding Information by Search on the Network

Extremely

14%

Very

52%

Somewhat28%

Not very

6%

How Important is Search to your site?(eCommerce sites)

Source: Meta Group Multiclient Enterprise Portal Study (1999)

Business Value: Search-Related Revenue

• Precision/Recall vs. "Find Rate“

• Cycle:

Enter Search -> learn -> shop -> buy -> receive -> use

Some ThinkPad QuestionsWhat hard disks can I add to my ThinkPad 570?How do I add memory to my ThinkPad?Can I buy an external monitor for my laptop?What docking station works with my ThinkPad 570? What's a port replicator?Can I buy a port replicator for my ThinkPad 570?What batteries are available for my ThinkPad?What power adapters are compatible with my ThinkPad 570?What keyboards are available for my ThinkPad?How do I add Ethernet capability to my ThinkPad?

Expertise Location

• Current system is "Person to Topic Affinity" - based on documents in the topic and the people who authored, contributed, distributed, read and subscribed

• Opportunity for Question Answering

Question Answering for Resumes • Question Topics:

– Education (degrees, certificates)• programming languages, OS, applications, h/w, etc

– Knowledge/expertise in a technical domain – Knowledge of industry

• experience w/ customers, or business analysis

– Title/Role/Occupation • e.g., analyst, engineer, sales rep

– Previous customer engagements– Previous experience (including before IBM)– Geographic location and languages known

Sample Questions Who has professional experience with the Internet?Who has experience with Internet security?Who has participated in customer engagements involving

Internet security?Who has led healthcare related engagements?Who is a systems architect in the sales and distribution

industry?Who was project manager of an inventory management

project?Who has a degree in AI and experience with expert systems?

Approach• Search over metadata and structured data • Handle joints

– "systems analyst in publishing"

• Massage text:– multi-word NPs: Internet security, inventory management– missing subjects: "Developed Web apps..."– resume structure

• Domain ontology and knowledge of geography– "AI expert", "KM specialists in Europe"

• Extend semantic annotation for the domain – educational degrees, industries, ...

Conversational Interface for Shopping

Interaction

Conflict Resolution

Conversational Interface for Online Shopping

• Natural language dialog for finding IBM products on the Web

• Bridge the gap b/w consumer, business and content owner

• Based on NPs and keywords used by consumers to describe products

• Study compared w/ menu-driven app:– shortens interaction time by 33%, – reduces number of clicks by 63%

Virtual Assistant

• Address book– What is David's office number.

– Create and new contact for Jim Hanks.

• Miscellaneous– This is John Doe and my passcode is

123456

– What time is it?

– Hold on just a minute

– Repeat that

• Messaging– Do I have any messages from John?– Fax that to 914-555-1234.– Send Jennifer a note.– Reply to that.– List the first 4 messages.– Delete the message from Rich.

• Calendaring– What's on my calendar next Tuesday?– Am I free tomorrow afternoon from 2 to 3?– Please set up a one hour meeting this Friday

and invite Jane.– Delete today's meeting at 3 o'clock.

• Call management– Record a new voice greeting.– Hold all my calls until 1 pm.– Forward my calls to extension 5123.– Call Jim on his cell phone.– Conference Stella in on this call.

Virtual Assistant

• Intelligent speech understanding providing access to and control of business applications through broad class of devices.

• Unified messaging framework for e-mail, voice mail, and fax

• Create NL grammar by domain: e-mail, calendars, call management, general information

Issues for QA in business

• Integrate QA with other KM technologies• Prototype QA for applications and learn

what is useful• Develop measures:

– improvement attributable to QA– “good enough” for customers to use– cost of customization to new domains and

applications