Watson CaseSoftware Project ManagementAmbuj KumarAnkur GuptaPrithvi ChandraVikash Bagri

Background Challenges All in the name of

Jeopardy Evolution of

supercomputers A priori v/s post priori Comparison with Deep

Blue

QA Algorithms:1. NLP2. TREC3. PIQUANT4. Semantic Search (Ontology -

Swoogle)RDF/OWL

• Fuzziness of Wordplay- Sherlock’s Hat

• Incremental v/s Leap-Process Improvement-Disruptive innovation

• The problem of Legacy Systems-Scientific Paper: eg. of Google

• Team Size and Location-One Room

Ontology Example

Challenges from PM Perspective Querying Template No deterministic Project

Objective-Win Jeopardy!

Information Classification Accuracy & Quality of

Data-Confidence Estimation

Narrowing of Ideas(MECE) Benchmarking –

Humans/Machines-Winners’ Cloud/Bake Off

Text v/s Knowledge Search

Hardwired Steps:• Problem Analysis &

Requirement Gathering• Specification• Architectural Design• Implementation –

Development• Testing • Standard Software

Development Lifecycle Model v/s

Competing Algorithms

Human Mind v/s Machine Learning : Logarithmic to Exponential

Contrast with the traditional system

Traditional System The requirements are

usually pretty straightforward.

Traditional information system initiative involves mostly hard coding, with very minimal dynamic code generation.

The requirement and constraints are predetermined and factored into the development of the program.

Watson System• The requirement wasn’t

clear. The requirements would need to be determined by the program itself during its run-time.

• predominate hard coding cannot be used to address the Jeopardy problem. It has to be mostly dynamic

Coding and the logic has to be generated in real time.

Key Problem Areas

The “problem” that Watson project was commissioned to solve was “win at Jeopardy.”

Ferrucci was not addressing a deficiency in a current system or process. He and his team were instead creating a “problem” or puzzle to solve.

Second, this was a “green field” implementation. There weren’t any clues on how to break down “win at Jeopardy” in smaller process steps to define visible milestones and deliverables.

Due to the nature of the game, requirements would need to be found by Watson in real-time based on the clue data.

SDLC @ Watson

Watson was a non-traditional system which seek to go beyond the limits of traditional computing and software programs

Hence, It needed to look for new innovation methods for system development to meet to desired goal

It used several development approach simultaneously due to lack of any empirical evidence to suggest which SDLC process to adopt.

The Watson project a hybrid of Rapid Prototyping, Incremental development approach in parallel

The Watson Approach

The Watson system utilized a different approach to its methods of analyzing, referencing, and prioritizing user input.

The Watson system used an approach that included Parallelism a wide rage of question analytic methods confidence estimates balancing the strictness applied to knowledge sources

know as the DeepQA approach. Using an approach know as “parallelism” this Watson

would use a variety of external resources from varied resources to gather data and decompose the query and generate possible hypothetical answers.

Key learning and recommendations

The project first of its kind on such a large scale, hence it required a new approach

A more detailed project objective with key milestones and sub-processes should have been used to measure the progress for better planning

The team used rapid prototyping very late. They could have been used this method zearly in the development project to gain insights into the system

Thank You