approaches to solving complex real- world problems using...
TRANSCRIPT
Approaches to solving complex real-world problems using systemic Artificial Intelligence and Evolutionary Computing
Lars Hard, CTO, Expertmaker
2
Lecture overview
• Real world applications powered by AI
• Data-driven knowledge representation
• Rapid modeling and deployment of AI
• Systemic AI and integrative modeling process
• The importance of evolutionary computation simplifying complex problem solving
• Automatic optimization and adaptation of AI models
3
WHY AI?
• To increase precision and relevance
• To reduce information complexity
• To enable the use of many more information sources
• To become more personal
• To allow more adaptive user experience
• To function bettwer with incomplete information (example: detailed opt-ins)
• To provide new functionality (discovery)
• To make sensors more useful (context)
• To span over more advanced knowledge and experience (medical diagnosis)
• Etc…
4
DATA -> KNOWLEDGE
5
• Info overload
• Contradictions
• Missing values
• Errors
• Multiple sources
• Bias (systemic, trends,…)
• Etc.
6
Data-driven knowledge representation
Data mining & Analytics
Computational Intelligence
Feature extraction & Transformation
7
”Knowledge transfer process ”
1. Create training data, extract metadata, combine sources
2. Data mining - Understand data
3. Modelling - process data to enable efficient machine learning
4. Preview - test AI (prediction, estimation, recognition, etc.)
5. Publish
6. Validate
7. Feedback behavioural data to improve solution (automatic or semi-automatic)
8
Typical AI setup
Machine Learning
Process (“training”)
End-user
Executing the AI
model (often via
API)
Training Data
Results - Conclusion - Recommendation - Classification - etc.
9
Typical AI setup
Machine Learning
Process (“training”)
End-user
Executing the AI
model (often via
API) – AI/CI model
Training Data
Results - Conclusion - Recommendation - Classification - etc.
Feature Extraction
& Transformations
- Source 1 - Source 2 - Source 3 - etc.
FEEDBACK
10
Typical AI setup
Machine Learning
Process (“training”)
End-user
Executing the AI
model (often via
API) – AI/CI model
Training Data
Results - Conclusion - Recommendation - Classification - etc.
Feature Extraction
& Transformations
- Source 1 - Source 2 - Source 3 - etc.
Modeling Validation
Data mining Analytics
11
Unstructured Data – Feature Extraction
12
Numerical features from any data
• Pre-processing and transformation of data for rapid extraction of numerical features
• Features are easy to process computationally
• Features can be transformed further (aggregation, projections, etc)
• Features can be normalized
• Features can be extectuted dynamically (even driven by inference) as they are needed or when more information is available. Features can be optimized by an evolutionary process, to adapt to certain types of problems or difficulties
13
RAPID MODELING AND DEPLOYMENT OF AI & SYSTEMIC AI
14
Knowledge design
Feature
extraction
configuration
Social
DATA SOURCES
Feeds Databases Unstructured Experts
End-users AI Server
Design,
modelling,
analytics and
feature
extraction
API
Modeling & deployment process
15
Multipe AI Modules (solvers)
Configuration (combine multiple components)
Optimization (finding the best solution for highly complex problems)
Image recognition (specialized domains and/or hierarchical recognition)
Diagnostics (troubleshooting, medical diagnosis: deterministic, probabilistic and hybrid)
Text classification (automated metadata extraction, hierachical classification)
Estimation (provide numerical predictions based on artificial neural networks)
Recommendation (product recommendation based on soft parameters, weight systems, feedback, filters, etc.)
Decisions (planning, decisions under low information conditions)
…and more…
16
News personalization setup
17
Profiling using Facebook
18
Image Processing Pipeline
Server (Machine Learning)
earning) Specific
Classifiers (verticals)
Domain specific Image
feature extractors
Tagging
Duplicate detection
Similar images
Broad classifier
API
Modeling domain specific image processing (kitchen, garden, etc.)
Training set (images)
Image sent via REST API
XML
19
EVOLUTIONARY COMPUTING
20
Bio-Inspired computing COMPUTATIONAL INTELLIGENCE
EVOLUTION AS A MODEL-FREE APPROACH TO AI Genetic Algortihms Genetic Programming Cellular Automata Gene Expression Programming etc.
BRAIN AS AN INSPRIATION
Computational Intelligence (CI)
21
The best algorithms for implementing AI ?
• ARIMA, autocorrelation, exponential smoothing,
distributed lags analysis, trend & seasonality
analysis, moving average, function fitting,
nonlinear estimation, multiple regression, Fourier
analysis, squared coherency, algebraic
estimates, tree induction, neural networks,
nearest neighbor, linear regression, K-means,
CART, projection pursuit, MARS, Parzen’s
windows, hypersphere classifiers, PCA, K-
means, SOM, variance, p values, standard error,
multidimensional scaling, statistical
discrimination, hierarchical clustering, t-test,
Bayesian probabilities, support vector machines
(SVM), tree induction, summary statistics,
profile matching, significance & confidence
analysis, double conjugated clustering, singular
value decomposition sorting (SVD), filtering,
ANOVA, etc…
22
Evolutionary
Computation (EC)
Genetic
Programming
Genetic
Algorithms
(GA)
Evolutionary
Strategy
Evolutionary
Programming
Simulated
Annealing
Artificial
Neural
Networks
Unsupervised
Learning
Supervised
Learning Fuzzy
Computing
Data Mining
K Nearest N.
Induction
MARS
Gaussian
Logistic regr.
Collective
Computational
Intelligence
Swarm &
ant colony
Hybrid
systems
Cellular
Automata
23
GA Flow
Evaluate fitness
Select mates
Reproduce
Mutate
Termination?
Define:
Parameters
Fitness function
Represent
parameters
Create population
Exit
Process
environment (?)
24
25
1000 generations
26
10000 generations
27
30000 generations
28
30000 generations
29
p1
p2
p3
p4 p1
p2
p3
p4
M
Island Master & Slave
Lattice p
p
p
p
p
p1
p
p
p
p
p
p
p
p
p
p
30
First Evolved Hardware in Space
X-band Antenna for NASA's ST5 Mission
31
Evolved antennas
32
Representation
• forward(length, radius) – - add a wire with the given length and radius extending from
the current location and then change the current state location to the end of the new wire.
• rotate-x(angle) – - change the orientation by rotating it by the specifed amount
(in radians) about the x-axis.
• rotate-y(angle) – - change the orientation by rotating it by the specifed amount
(in radians) about the y-axis.
• rotate-z(angle) – - change the orientation by rotating it by the specied amount
(in radians) about the z-axis.
33
Genotype for evolved antenna ST5-3-10 rotate-z(1.984442) 1 [rotate-x(2.251165) 1 [rotate-x(0.062240) 1 [rotate-x(0.083665) 1 [rotate-y(-2.449035) 1 [ rotate-z(-0.894357) 1 [rotate-y(-2.057702) 1 [rotate-y(0.661755) 1 [rotate-x(0.740703) 1 [rotate-y(2.057436) 1 [ forward(0.013292,0.000283) 2 [rotate-z(-1.796822) 1 [ rotate-x(-1.651348) 1 [rotate-y(-2.940880) 1 [rotate-x(0.095209) 1 [rotate-z(1.248723) 1 [forward(0.003815,0.000363) 1 [ forward(0.008289,0.000355) 1 [forward(0.008413,0.000369) 1 [ rotate-x(-0.006494) 1 [rotate-x(-0.592854) 1 [rotate-z(-2.085023) 1 [rotate-z(1.735374) 1 [rotate-z(-2.045125) 1 [ rotate-z(0.203076) 1 [rotate-z(1.750799) 1 [rotate-z(-2.038688) 1 [rotate-z(1.725007) 1 [rotate-y(1.478109) 1 [rotate-x(2.477117) 1 [rotate-x(-2.441858) 1 [forward(0.015082,0.000223) ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] rotate-y(2.335438) 1 [ rotate-y(-1.042201) 1 [rotate-y(-1.761594) 1 [rotate-x(2.518405) 1 [rotate-z(-0.739608) 1 [rotate-x(0.426553) 1 [ rotate-z(-0.291483) 1 [rotate-x(2.152738) 1 [ forward(0.013190,0.000414) ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
34
Crossover
35
Pathfinding
36
Cellular Automata
One dimensional example: RULE 3 (binary = 00000011)
37
Cellular Automata
38
Bio-inspired models advantages
Evolutionary computing (GA, GP, GEP, etc.) allows us to:
• Simplify “old AI” so that anyone can make use of it without a math PhD
• Perform massive “experimentation” on problems in order to reach robust solutions
• Optimizing really hard problems (like data complexity reduction)
• Model systems in new ways (one example is totally different search engines)
• Reduce complexity
• Robustness
39
MOVIE TWIST Use case
40
Implementing a move recommender
Movie Twist
Why a new movie recommendation app?
• Collaborative filtering (if you bout x then you like y) gives very little exploration, and often gets worse over time
• Discovery & exploration requires multiple ways to access content
• Uncovering new relationships between movies makes it more interesting, but requres new metadata
• Adaptation by self-organization
41
• Metadata extraction from multiple sources (9) – Text classification
– Transformations
– Re-classifications
• Multiple AI to move away from ”collaborative filtering” – Recommendations
– Reverse lookups (multiple- and single point)
– Adaptation
– Multi-point basket creation
42
43
44
45
Next?
• TEK292 – Autumn 2013 - Biological models Genetic Programming
• Expertmaker AI Hackathon 10/5-12/5, Mamlö
