New Trends in Intelligent Systems

Dr. Jay Liebowitz

Professor

Johns Hopkins University

Jliebow1@jhu.edu

“AI: Past, Present, and Future”, AI Magazine, 25th Anniversary Issue of AAAI, Vol. 26, No. 4, Winter 2005

“We are a scientific society devoted to the study of artificial intelligence”…Allen Newell, The First AAAI President’s Message, 1980

“As AI matures, its focus is shifting from inward-looking to outward-looking. Some of the new concerns of the field are social awareness, networking, cross-disciplinarity, globalization, and open access”…Alan Mackworth, Current AAAI President, July 2005

The Next 50 Years…

“The Semantic Web is to KR as the Web is to hypertext”…James Hendler, U. of Maryland

“AI has not yet succeeded in its most fundamental ambitions. Our systems are fragile when outside their carefully circumscribed domains”…Rod Brooks, MIT

“Reasoning programs still exhibit little common sense”…Patrick Winston, MIT

More Quotes

“Integrative research will be particularly challenging for research students. To do it, they must master a wide range of formal techniques and understand not just the mathematical details but also their place in overall accounts of intelligent behavior”…Haym Hirsh, Rutgers University

“Another reason for the slow progress is the fragmentation of AI”…Aaron Sloman, U. of Birmingham

Innovation, 2004 (Patent Applications Filed)—Financial Times, June 8, 2005, Thomson Scientific

1. Japan 342,726

2. US 167,183

3. South Korea 71,483

4. Germany 55,478

5. China 40,426

6. Russia 19,104

7. France 13,246

8. UK 12,245

9. Taiwan 8,684

10. Italy 4,869

11. Australia 4,142

12. Brazil 3,700

13. Canada 3,125

14. Sweden 2,272

15. Spain 2,260

…30. Ireland 300

Patents Filed by Sector in 2004 (Spain); Financial Times, Oct. 26, 2005, Thomson Scientific

48%: Chemicals, materials and instrumentation 14%: Telecom, IT, and electronics 13%: Food and agriculture 11%: Automotive and transport 10%: Pharmaceutical and medical 4%: Energy and power “Biotechnology: Spanish research highly rated in agro-industry,

medicine, and alternative fuels” “Spanish biotechnology is growing 4 times faster than the average of

the European 15” “Spain accounts for 4% of all biotech research published in the world” “Sluggish integration of IT solutions into daily life”

Integrative Research in Knowledge Management

PEOPLE

TECHNOLOGY

Building and Nurturing a Knowledge

Sharing Culture

Systematically Capturing and SharingCritical Knowledge

Creating aUnifiedKnowledge Network

PROCESS

Applying AI to KM:Expert Systems Technology

Knowledge elicitation techniques to acquire lessons learned (via structured/unstructured interviews, protocol analysis, etc.)

On-line pools of expertise (rule or case-based)

Knowledge representation techniques for developing an ontology

Intelligent Agent Technology

Intelligent multi-agent systems with learning capabilities to help users in responding to their questions

Searching and filtering tools User profiling and classification tools Agent-Oriented Knowledge Management

AAAI Symposium (Stanford University)

Data Mining and Knowledge Discovery Techniques

Inductively determine relationships/rules for further developing the KM system

Help deduce user profiles for better targeting the KM system

Help generate new cases

Neural Networks, Genetic Algorithms, etc.

Help weed out rules/cases Help look for inconsistencies within the

knowledge repository Help filter noisy data

--Develop “active” analysis and dissemination techniques for --Develop “active” analysis and dissemination techniques for knowledge sharing and searching via “intelligent” agent technology knowledge sharing and searching via “intelligent” agent technology (i.e., where “learning” takes place)(i.e., where “learning” takes place)

--Apply knowledge discovery techniques (e.g., data/text mining, --Apply knowledge discovery techniques (e.g., data/text mining, neural networks, etc.) for mining knowledge bases/repositoriesneural networks, etc.) for mining knowledge bases/repositories

--Improve query capabilities through natural language understanding --Improve query capabilities through natural language understanding techniquestechniques

--Develop metrics for measuring value-added benefits of knowledge --Develop metrics for measuring value-added benefits of knowledge managementmanagement

--Develop standardized methodologies for knowledge management --Develop standardized methodologies for knowledge management development and knowledge auditsdevelopment and knowledge audits

--Provide improved techniques for performing knowledge mapping --Provide improved techniques for performing knowledge mapping and building knowledge taxonomies/ontologiesand building knowledge taxonomies/ontologies

KM Research IssuesKM Research Issues

--Develop techniques for building collaborative knowledge bases--Develop techniques for building collaborative knowledge bases

--Develop improved tools for capturing knowledge from various media --Develop improved tools for capturing knowledge from various media (look at multimedia mining to induce relationships among images, (look at multimedia mining to induce relationships among images, videos, graphics, text, etc.)videos, graphics, text, etc.)

--Develop techniques for integrating databases to avoid stovepiping, --Develop techniques for integrating databases to avoid stovepiping, functional silosfunctional silos

--Build improved software tools for developing and nurturing --Build improved software tools for developing and nurturing communities of practicecommunities of practice

--Develop techniques for categorizing, synthesizing, and summarizing --Develop techniques for categorizing, synthesizing, and summarizing lessons learned (look at text summarization techniques)lessons learned (look at text summarization techniques)

--Explore ways to improve human-agent collaboration--Explore ways to improve human-agent collaboration

--Explore human language technologies for KM (input analysis, --Explore human language technologies for KM (input analysis, extraction, question-answer, translation, etc.)extraction, question-answer, translation, etc.)

KM Research Issues (cont.)KM Research Issues (cont.)

WBM 2005 Research Problem (James Simien, NPRST, April 2005)

How to provide IT support for the Navy’s future distributed business processes involving sailors and commands as outlined in the Navy’s Human Capital Strategy?

Distributed processes provide tremendous opportunity for increasing efficiencies across the enterprise.

– Proposed solution: Develop a Multi-Agent System incorporating software agents to

intelligently assist Users in performing tasks.

Major Focus in FY05 (Simien, 2005)

•Development of a formal methodology for knowledge acquisition and management for Navy’s business rules used in the assignment process (Liebowitz et al., 2005)

•Exploring use of genetic algorithms in Sailor job matching

•Development of agent bi-lateral negotiation for those assignment matches that occur outside of the general matching process

•Experimentation with multiple forms of distributed architecture to determine performance and scalability (Liebowitz et al., 2004; 2005)

Next Generation of Data Mining Applications (M. Kantardzic & J. Zurada, IEEE Press, 2005)

Current data warehouses in the terabyte range (FedEx, UPS, Wal-Mart, Royal Dutch/Shell Group, etc.)

Diversity of data (multimedia data) Diversity of algorithms (GAs, fuzzy sets, etc.) Diversity of infrastructures for data mining applications

(web-based services and grid architectures) Diversity of application domains (Internet-based web

mining, text mining, on-line images and video stream mining)

Emphasis on security and privacy aspects of data mining (protect data usually in a distributed environment)

Red Light Cameras and Motor Vehicle Accidents (Solomon, Nguyen, Liebowitz, Agresti, 2005; funded through GEICO Found.)

Objective– Employ data mining techniques to explore the

relationship between red light cameras and motor vehicle accidents

Data– FARS database– 2000 – 2003 in MD and Washington, D.C.– 16,840 entries

Strongest relationships are collisions with moving objects and angle front-to-side crashes.

The 3pm – 4pm hour and months later in the year.

Car collisions are more likely to happen on Fridays and Sundays.

Types of car crashes involved in running red lights are mostly rear-end crashes and angle front-to-side collisions.

High relative importance of gender.

Findings

New/Repackaged Growth Areas for AI

Business rule engines– The acquisition of RulesPower assets allows Fair

Isaac's customers a higher-performance business rule engine (BRE) option that leverages the RETE III algorithm (September 27, 2005; Gartner Group Report).

– Annual Business Rules Conference (November 2006 in Washington, D.C.)

Another Area for Growth

Strategic Intelligence: The Synergy of Knowledge Management, Business Intelligence, and Competitive Intelligence (see Liebowitz, J., Strategic Intelligence book, Auerbach Publishing/Taylor & Francis, NY, April 20, 2006)

Continued Growth in Discovery Informatics (Knowledge Discovery)

New curricula at the undergraduate level at College of Charleston (Discovery Informatics), Washington & Jefferson (Data Discovery), etc.

New Graduate Certificate in Competitive Intelligence (Johns Hopkins University; Jay Liebowitz, Program Director)

SCIP (Society of CI Professionals—www.scip.org)—CI analysts

Web and Text Mining

Steady Growth

Robotics and Computer Vision Natural Language and Speech Understanding Neural Networks, Genetic Algorithms, Self-

Organizing Maps Intelligent/Multi-Agents Fuzzy Logic

Papers Are Being WrittenWorldwide…

EXPERT SYSTEMS WITH APPLICATIONS is a refereed international journal whose focus is on exchanging information relating to expert and intelligent systems applied in industry, government, and universities worldwide.

Published by Elsevier; Entering Volumes 30 & 31 (2006)

Trends in Intelligent Scheduling Systems

Constraint-based Expert scheduling system shells/generic

constraint-based satisfaction problem solvers Object/Agent-oriented, hierarchical

architectures Hybrid intelligent system approaches

NASA Scheduling Environment

Two of the most pressing tasks in the future for NASA: Data capture/analysis and scheduling

GUESS (Generically Used Expert Scheduling System)

A generic intelligent scheduling tool to aid the human scheduler and to keep him/her in the loop

Programmed in Visual C++ and runs on an IBM PC Windows environment (about 9,500 lines of code)

2.5 year effort

Features of GUESS

OOPS feature of GUESS is that classes represent various abstractions of scheduling objects, such as events, constraints, resources, etc.

Resources--binary, depletable, group, etc. Constraints--before, after, during, notduring,

startswith, endswith, meta, etc. Repair-based scheduling

Major Scheduling Approaches in GUESS

Suggestion Tabulator: uses suggestions derived from the constraints

Hill climbing algorithm Genetic algorithm--used EOS, a C++ class

library for creating GAs Hopfield neural network algorithm

Neural Networks in Scheduling

The existing work demonstrated that scheduling problems can be attacked and appropriately solved by NNs

The majority of the artificial NNs proposed for scheduling were based on the Hopfield network (an optimizer)

Most of the neural networks developed for scheduling have been in manufacturing domains

Hopfield Network (NN Connections)

Each of the constraints on an event produces an error signal. The error signal is chosen to cause the event to move in the correct direction to produce a "satisfied" schedule. The errors on a given event induced by the constraints are summed together and then passed through a sigmoid function. The output of the sigmoid function f(x) is used to shift the begin and end times of the event to drive the schedule to a more satisfied state. Several different sigmoid functions were tried. The most promising was f(x) = tanh (x). This yielded the following equation for the neural network:

Equation Used for NN Connections

event theandevent ebetween therror constraint ),(

constant weighting

event shift the to timedelta

where,

),(tanh

jthitheec

k

ith

eeck

ji

i

j

jii

Table 1 - Comparison of Scheduling Methods using Computer Generated Cases

Number ofEvents

1000 2000 4000 6000

Number ofConstraints

5985 11992 23974 36202

Sele

cte

d

Sch

ed

uli

ng

Alg

ori

thm

Sp

an

(D

ay

s)

Sati

sfa

cti

on

(%)

Sp

an

(D

ay

s)

Sati

sfa

cti

on

(%)

Sp

an

(D

ay

s)

Sati

sfa

cti

on

(%)

Sp

an

(D

ay

s)

Sati

sfa

cti

on

(%)

Unscheduled 73 17.13 73 18.36 149 15.38 149 14.83SuggestionTabulator

73 28.29 73 27.49 149 23.04 149 22.11

Hill Climbing 73 78.10 73 76.85 149 75.86 149 76.85Neural Network 73 67.94 73 66.55 149 64.60 149 66.75

Table 2 - Comparison of Solution Times using Computer Generated Test Cases

Number of Events Hill Climbing (secs) Neural Network (secs)1000 21 82000 40 164000 106 416000 200 64

Different Types of Scheduling Applications Using GUESS

City of Rockville Baseball Scheduling Army strategic problem of scheduling arrival

of units in a deployed theater Army operational problem of scheduling Army

battalion training exercises College course timetabling at MC NASA satellite scheduling

Table 3 – Comparison of Scheduling Methods using Real World Test Cases

Army1 Baseball2 CAM3 MontgomeryCollege4

52 events333 constraints

165 events328 constraints

111 events195 constraints

11 events30 constraints

SchedulingApproaches

Sp

an

(D

ays)

Sati

sfa

cti

on

(%

)

Sp

an

(D

ays)

Sati

sfa

cti

on

(%

)

Sp

an

(D

ays)

Sati

sfa

cti

on

(%

)

Sp

an

(D

ays)

Sati

sfa

cti

on

(%

)

SuggestionTabulator

59 99.06 47 100.00 77 79.02 4 100.00

Hill Climbing 59 99.06 47 100.00 122 79.96 4 100.00GeneticAlgorithm

60 96.70 51 94.20 87 42.40 4 100.00

NeuralNetwork

59 98.61 47 100.00 368 47.05 4 100.00

1. Scheduling for Army battalion training exercises.2. Scheduling for the City of Rockville baseball games.3. Scheduling the arrival of military units to a deployed theatre.4. Scheduling Department of Computer and Information Sciences classes at Montgomery

College.

Lessons Learned

Don’t underestimate the amount of time required for the user interface design

Scheduling is a difficult (but pervasive) problem

Nothing goes according to schedule--so have efficient ways of handling rescheduling

Future Work

Develop database links for ease of inputting Classify different scheduling types and

models and incorporate them into GUESS Expand the number of scheduling methods

(OR+AI, etc.)

Questions to Ponder??

Will AI ever achieve natural/human intelligence?

Should we have called our field IA (Intelligence Amplification) versus AI, since most of the AI applications are still for decision support?

Have we found the “killer application” for AI yet?

Will AI survive as a field or discipline?

THE END

GRACIAS!!