sensitivity analysis for building adaptive robotic software
TRANSCRIPT
SENSITIVITY ANALYSIS FORBUILDING ADAPTIVE ROBOTIC SOFTWAREPooyanJamshidi,MiguelVelezandChristianKästner
INTENT DISCOVERY: SENSITIVITY ANALYSIS FOR CONFIGURATION OPTIMIZATION
REDUCING COSTS WITH TRANSFER LEARNING
USE CASES
Systematic System Evolution To automate or guide intelligent design choices.
Runtime Adaptation To enable runtime adaptation of software configurations
to maintain quality of performance under dynamic
conditions (changing environment, goals, and tasks).
Performance Debugging To guide robot software developers to identify potential
bugs causing low quality of performance.
RESULTS
Motivation:Robotic software expose configurable parameters.
These tunable parameters affect performance of robots.
This can be leveraged to optimize performance.
Source Response Target Response
Transfer learning combines:
Lots of data gathered cheaply from
the simulator
With much less data gathered
expensively from the target robot
To make better predictions overall
PUBLICATIONSP. Jamshidi, M. Velez, C. Kästner, N. Siegmund, and P. Kawthekar. Transfer learning for improving model predictions in highly configurable software. Int’l Symp. Software Engineering for Adaptive and Self-Managing Systems (SEAMS), 2017. P. Kawthekar and C. Kästner. Sensitivity analysis for building evolving & adaptive robotic software, Workshop on Autonomous Mobile Service Robots (WSF), 2016.
Predictive Model
Learn Model
MeasureMeasure
DataSourceTarget
Simulator (Gazebo) Robot (TurtleBot)
Predict Performance
Predictions
Adaptation
Use for analysis
5 10 15 20 25number of particles
5
10
15
20
25
nu
mb
er
of
refin
em
en
ts
0
5
10
15
20
25
5 10 15 20 25number of particles
5
10
15
20
25
nu
mb
er
of
refin
em
en
ts
0
5
10
15
20
25
5 10 15 20 25number of particles
5
10
15
20
25
nu
mb
er
of
refin
em
en
ts
0
5
10
15
20
25CPU usage [%] CPU usage [%]
(a) (b)
(c) (d)
Prediction without transfer learning
5 10 15 20 25
5
10
15
20
25
10
15
20
25
Prediction with transfer learning
Using only a few real data points to predict yields poor results across configuration space
Using transfer learning to combine the few real data points with lots of approximate data yields a good model
Machine Learning
Configuration Parameters
Design of Experiment
Configuration Space
Predictive Model
Sensitivity Analysis
DataMeasurements
Configuration Space
Data
AccuracyEnergy
CPU
0 5 10 15 20 25 30 35mean CPU utilization
0
500
1000
1500
2000
2500
3000
3500
num
ber
of c
onfig
urat
ions
5 10 15 20 25number of particles
5
10
15
20
25
nu
mb
er
of
refin
em
en
ts
6
8
10
12
14
16
18
20
22
24
26
5 10 15 20 25number of particles
5
10
15
20
25
num
ber
of re
finem
ents
5
10
15
20
25
30
35
40
45
CPU
Localisation Error