approaches to solving complex real- world problems using...

Approaches to solving complex real-world problems using systemic Artificial Intelligence and Evolutionary Computing

Lars Hard, CTO, Expertmaker

2

Lecture overview

• Real world applications powered by AI

• Data-driven knowledge representation

• Rapid modeling and deployment of AI

• Systemic AI and integrative modeling process

• The importance of evolutionary computation simplifying complex problem solving

• Automatic optimization and adaptation of AI models

3

WHY AI?

• To increase precision and relevance

• To reduce information complexity

• To enable the use of many more information sources

• To become more personal

• To allow more adaptive user experience

• To function bettwer with incomplete information (example: detailed opt-ins)

• To provide new functionality (discovery)

• To make sensors more useful (context)

• To span over more advanced knowledge and experience (medical diagnosis)

• Etc…

4

DATA -> KNOWLEDGE

5

• Info overload

• Contradictions

• Missing values

• Errors

• Multiple sources

• Bias (systemic, trends,…)

• Etc.

6

Data-driven knowledge representation

Data mining & Analytics

Computational Intelligence

Feature extraction & Transformation

7

”Knowledge transfer process ”

1. Create training data, extract metadata, combine sources

2. Data mining - Understand data

3. Modelling - process data to enable efficient machine learning

4. Preview - test AI (prediction, estimation, recognition, etc.)

5. Publish

6. Validate

7. Feedback behavioural data to improve solution (automatic or semi-automatic)

8

Typical AI setup

Machine Learning

Process (“training”)

End-user

Executing the AI

model (often via

API)

Training Data

Results - Conclusion - Recommendation - Classification - etc.

9

Typical AI setup

Machine Learning

Process (“training”)

End-user

Executing the AI

model (often via

API) – AI/CI model

Training Data

Results - Conclusion - Recommendation - Classification - etc.

Feature Extraction

& Transformations

- Source 1 - Source 2 - Source 3 - etc.

FEEDBACK

10

Typical AI setup

Machine Learning

Process (“training”)

End-user

Executing the AI

model (often via

API) – AI/CI model

Training Data

Results - Conclusion - Recommendation - Classification - etc.

Feature Extraction

& Transformations

- Source 1 - Source 2 - Source 3 - etc.

Modeling Validation

Data mining Analytics

11

Unstructured Data – Feature Extraction

12

Numerical features from any data

• Pre-processing and transformation of data for rapid extraction of numerical features

• Features are easy to process computationally

• Features can be transformed further (aggregation, projections, etc)

• Features can be normalized

• Features can be extectuted dynamically (even driven by inference) as they are needed or when more information is available. Features can be optimized by an evolutionary process, to adapt to certain types of problems or difficulties

13

RAPID MODELING AND DEPLOYMENT OF AI & SYSTEMIC AI

14

Knowledge design

Feature

extraction

configuration

Social

DATA SOURCES

Feeds Databases Unstructured Experts

End-users AI Server

Design,

modelling,

analytics and

feature

extraction

API

Modeling & deployment process

15

Multipe AI Modules (solvers)

Configuration (combine multiple components)

Optimization (finding the best solution for highly complex problems)

Image recognition (specialized domains and/or hierarchical recognition)

Diagnostics (troubleshooting, medical diagnosis: deterministic, probabilistic and hybrid)

Text classification (automated metadata extraction, hierachical classification)

Estimation (provide numerical predictions based on artificial neural networks)

Recommendation (product recommendation based on soft parameters, weight systems, feedback, filters, etc.)

Decisions (planning, decisions under low information conditions)

…and more…

16

News personalization setup

17

Profiling using Facebook

18

Image Processing Pipeline

Server (Machine Learning)

earning) Specific

Classifiers (verticals)

Domain specific Image

feature extractors

Tagging

Duplicate detection

Similar images

Broad classifier

API

Modeling domain specific image processing (kitchen, garden, etc.)

Training set (images)

Image sent via REST API

XML

19

EVOLUTIONARY COMPUTING

20

Bio-Inspired computing COMPUTATIONAL INTELLIGENCE

EVOLUTION AS A MODEL-FREE APPROACH TO AI Genetic Algortihms Genetic Programming Cellular Automata Gene Expression Programming etc.

BRAIN AS AN INSPRIATION

Computational Intelligence (CI)

21

The best algorithms for implementing AI ?

• ARIMA, autocorrelation, exponential smoothing,

distributed lags analysis, trend & seasonality

analysis, moving average, function fitting,

nonlinear estimation, multiple regression, Fourier

analysis, squared coherency, algebraic

estimates, tree induction, neural networks,

nearest neighbor, linear regression, K-means,

CART, projection pursuit, MARS, Parzen’s

windows, hypersphere classifiers, PCA, K-

means, SOM, variance, p values, standard error,

multidimensional scaling, statistical

discrimination, hierarchical clustering, t-test,

Bayesian probabilities, support vector machines

(SVM), tree induction, summary statistics,

profile matching, significance & confidence

analysis, double conjugated clustering, singular

value decomposition sorting (SVD), filtering,

ANOVA, etc…

22

Evolutionary

Computation (EC)

Genetic

Programming

Genetic

Algorithms

(GA)

Evolutionary

Strategy

Evolutionary

Programming

Simulated

Annealing

Artificial

Neural

Networks

Unsupervised

Learning

Supervised

Learning Fuzzy

Computing

Data Mining

K Nearest N.

Induction

MARS

Gaussian

Logistic regr.

Collective

Computational

Intelligence

Swarm &

ant colony

Hybrid

systems

Cellular

Automata

23

GA Flow

Evaluate fitness

Select mates

Reproduce

Mutate

Termination?

Define:

Parameters

Fitness function

Represent

parameters

Create population

Exit

Process

environment (?)

24

25

1000 generations

26

10000 generations

27

30000 generations

28

30000 generations

29

p1

p2

p3

p4 p1

p2

p3

p4

M

Island Master & Slave

Lattice p

p

p

p

p

p1

p

p

p

p

p

p

p

p

p

p

30

First Evolved Hardware in Space

X-band Antenna for NASA's ST5 Mission

31

Evolved antennas

32

Representation

• forward(length, radius) – - add a wire with the given length and radius extending from

the current location and then change the current state location to the end of the new wire.

• rotate-x(angle) – - change the orientation by rotating it by the specifed amount

(in radians) about the x-axis.

• rotate-y(angle) – - change the orientation by rotating it by the specifed amount

(in radians) about the y-axis.

• rotate-z(angle) – - change the orientation by rotating it by the specied amount

(in radians) about the z-axis.

33

Genotype for evolved antenna ST5-3-10 rotate-z(1.984442) 1 [rotate-x(2.251165) 1 [rotate-x(0.062240) 1 [rotate-x(0.083665) 1 [rotate-y(-2.449035) 1 [ rotate-z(-0.894357) 1 [rotate-y(-2.057702) 1 [rotate-y(0.661755) 1 [rotate-x(0.740703) 1 [rotate-y(2.057436) 1 [ forward(0.013292,0.000283) 2 [rotate-z(-1.796822) 1 [ rotate-x(-1.651348) 1 [rotate-y(-2.940880) 1 [rotate-x(0.095209) 1 [rotate-z(1.248723) 1 [forward(0.003815,0.000363) 1 [ forward(0.008289,0.000355) 1 [forward(0.008413,0.000369) 1 [ rotate-x(-0.006494) 1 [rotate-x(-0.592854) 1 [rotate-z(-2.085023) 1 [rotate-z(1.735374) 1 [rotate-z(-2.045125) 1 [ rotate-z(0.203076) 1 [rotate-z(1.750799) 1 [rotate-z(-2.038688) 1 [rotate-z(1.725007) 1 [rotate-y(1.478109) 1 [rotate-x(2.477117) 1 [rotate-x(-2.441858) 1 [forward(0.015082,0.000223) ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] rotate-y(2.335438) 1 [ rotate-y(-1.042201) 1 [rotate-y(-1.761594) 1 [rotate-x(2.518405) 1 [rotate-z(-0.739608) 1 [rotate-x(0.426553) 1 [ rotate-z(-0.291483) 1 [rotate-x(2.152738) 1 [ forward(0.013190,0.000414) ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]

34

Crossover

35

Pathfinding

36

Cellular Automata

One dimensional example: RULE 3 (binary = 00000011)

37

Cellular Automata

38

Bio-inspired models advantages

Evolutionary computing (GA, GP, GEP, etc.) allows us to:

• Simplify “old AI” so that anyone can make use of it without a math PhD

• Perform massive “experimentation” on problems in order to reach robust solutions

• Optimizing really hard problems (like data complexity reduction)

• Model systems in new ways (one example is totally different search engines)

• Reduce complexity

• Robustness

39

MOVIE TWIST Use case

40

Implementing a move recommender

Movie Twist

Why a new movie recommendation app?

• Collaborative filtering (if you bout x then you like y) gives very little exploration, and often gets worse over time

• Discovery & exploration requires multiple ways to access content

• Uncovering new relationships between movies makes it more interesting, but requres new metadata

• Adaptation by self-organization

41

• Metadata extraction from multiple sources (9) – Text classification

– Transformations

– Re-classifications

• Multiple AI to move away from ”collaborative filtering” – Recommendations

– Reverse lookups (multiple- and single point)

– Adaptation

– Multi-point basket creation

42

43

44

45

Next?

• TEK292 – Autumn 2013 - Biological models Genetic Programming

• Expertmaker AI Hackathon 10/5-12/5, Mamlö

46

Expertmaker

(650) 283 1107

lars.hard@expertmaker.com

Making the Internet intelligent