quantify and optimize android user experience
TRANSCRIPT
![Page 1: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/1.jpg)
Xiao-Feng Li
OCT, 2011
Quantify and Optimize User Interactions with Android Devices
![Page 2: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/2.jpg)
Summary
⢠Performance is not enough to characterize a client device
⢠An systematic approach has been established to engineering user interactions
â AWS is a comprehensive engineering workload suite
â UXtune is a toolkit to assist analysis and optimization
⢠Android user interactions can be impacted by the technical factors across the whole software stack
User interaction is a new challenge but not insurmountable
2011-09-12 2
![Page 3: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/3.jpg)
Agenda
⢠Client device user interactions
⢠Android user interaction optimizations
â Android Workload Suite and Android UXtune
⢠Case studies of Android optimizations
⢠Factors that impact Android user interactions
⢠Summary
⢠Information
2011-09-12 3
![Page 4: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/4.jpg)
Performance Is NOT Enough
⢠Performance does not reflect consistently as user perception
⢠Performance is only about the behavior of system in steady state
⢠Then, what are missing?
2011-09-12 4
![Page 5: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/5.jpg)
Observation of Touch Fling Operation
⢠Fling the picture in Android Gallery application
â Current picture slides and switches to next picture
2011-09-12 5
![Page 6: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/6.jpg)
Frames of A Fling Process on Device A
0
10
20
30
40
50
60
70
80
90
100
1
21
41
61
81
10
1
12
1
14
1
16
1
18
1
20
1
22
1
24
1
26
1
28
1
30
1
32
1
34
1
36
1
38
1
40
1
42
1
44
1
46
1
48
1
50
1
52
1
54
1
56
1
58
1
60
1
62
1
64
1
66
1
68
1
70
1
72
1
74
1
76
1
78
1
80
1
82
1
84
1
86
1
88
1
90
1
92
1
Fram
e T
ime
(m
s)
Time (ms)
Frame Times of Touch Fling
Notice the followings: ⢠Max frame time ⢠#frames > 30ms ⢠Frame time variance ⢠FPS
* Data of one fling
2011-09-12 6
![Page 7: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/7.jpg)
Frames of A Fling on Device B (Higher FPS)
0
10
20
30
40
50
60
70
80
90
100
1
21
41
61
81
10
1
12
1
14
1
16
1
18
1
20
1
22
1
24
1
26
1
28
1
30
1
32
1
34
1
36
1
38
1
40
1
42
1
44
1
46
1
48
1
50
1
52
1
54
1
56
1
58
1
60
1
62
1
64
1
66
1
68
1
70
1
72
1
74
1
76
1
78
1
80
1
82
1
84
1
86
1
88
1
90
1
92
1
Fram
e T
ime
(m
s)
Time (ms)
Frame Times of Touch Fling
Notice the followings: ⢠Max frame time ⢠#frames > 30ms ⢠Frame time variance ⢠FPS
* Data of one fling
2011-09-12 7
![Page 8: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/8.jpg)
A Fling on Device B After Optimization
0
10
20
30
40
50
60
70
80
90
100
1
21
41
61
81
10
1
12
1
14
1
16
1
18
1
20
1
22
1
24
1
26
1
28
1
30
1
32
1
34
1
36
1
38
1
40
1
42
1
44
1
46
1
48
1
50
1
52
1
54
1
56
1
58
1
60
1
62
1
64
1
66
1
68
1
70
1
72
1
74
1
76
1
78
1
80
1
82
1
84
1
86
1
88
1
90
1
92
1
Fram
e T
ime
(m
s)
Time (ms)
Frame Times of Touch Fling
Notice the followings: ⢠Max frame time ⢠#frames > 30ms ⢠Frame time variance ⢠FPS
* Data of one fling
2011-09-12 8
![Page 9: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/9.jpg)
User Interactions with Client Device
⢠A sequence of interactions
Human Input
Target Response
Screen transition
Object movement
⢠Does the input trigger the target correctly?
⢠Does the system act responsively?
⢠Does the graphics transition smoothly?
⢠Does the object move coherently?
Device User
interaction
2011-09-12 9
![Page 10: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/10.jpg)
User Interactions in Software Engineering
⢠Dynamic state transitions in the software stack
â Traditional performance more about steady state
â Performance is the links of the interaction chain
⢠User Interaction evaluation is a superset of traditional performance evaluation
⢠For example: a video workload
â âlaunch playerâ âstart playingâ âseek progressâ âvideo playbackâ âback to home screenâ
â Traditionally, only âvideo playbackâ is evaluated
2011-09-12 10
![Page 11: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/11.jpg)
A Bit on User Experience Philosophy
⢠UX is not an objective process, but an interactive process with subjective factors
â Consider watching movie or listening to music
⢠No silver-bullet to measure UX
â Current academic research status with eye-tracking, heart-beat, poll, etc.
11 2011-09-12
![Page 12: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/12.jpg)
UX Philosophy Triangle
12 2011-09-12
Ethos: âDo it correctlyâ
Logos: âAccomplish it systemicallyâ
UX
Pathos: âFollow it naturallyâ
![Page 13: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/13.jpg)
Agenda
⢠Client device user interactions
⢠Android user interaction optimizations
â Android Workload Suite and Android UXtune
⢠Case studies of Android optimizations
⢠Factors that impact Android user interactions
⢠Summary
⢠Information
2011-09-12 13
![Page 14: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/14.jpg)
Interaction Measurement Criteria
⢠Measure the critical path of user interactions in software stack
⢠Criteria
â Perceivable (by a human being)
â Measureable (by different teams)
â Repeatable (in multiple measurements)
â Comparable (between different measured systems)
â Reasonable (about the causality)
â Verifiable (for an optimization)
â Automatable (largely unattended, not strictly)
2011-09-12 14
![Page 15: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/15.jpg)
Interaction Measurement Aspects
⢠User controls device (subject object)
1. Accuracy/fuzziness: Range/resolution of inputs that can trigger a correct response
2. Coherence: Object move delay, difference in move trajectory
⢠Device reacts to user (object subject)
3. Responsiveness: Time between an input delivered to the device response, and to the action finish
4. Smoothness: Maximal frame time, frame time variance, FPS and frame drop rate
Device User
interaction
2011-09-12 15
![Page 16: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/16.jpg)
Example Industry Experience Values
2011-09-12 16
Best Good Acceptable
Response delay 100ms 200ms 500ms
Graphics animation 120fps 60fps 30fps
Video playback 60fps 30fps 20fps
![Page 17: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/17.jpg)
Example âDragâ: Drag Icon in Homescreen
⢠Measure drag coherence
â P1: the position where the icon starts to move at T1
â T2: the time when the icon reaches P1
â P2: the position where the finger touches at time T2
â P1 â P0, P2 â P1, T2 â T1 are the smaller the better
Time T0 Time T1 Time T2 Position P0 P1 P2 P1
2011-09-12 17
![Page 18: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/18.jpg)
Example âDragâ: Map To Engineering Values
⢠Metrics computation
â T1 = Time when Frame F1 is drawn by SurfaceFlinger
â P1 = Position value of the touch event at time T1
â T2 = Time of the frame when iconâs position is P1
â P2 = Position value of the touch event at time T2
P1 Time T1 Frame F1 Time T2
P2 P1
2011-09-12 18
![Page 19: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/19.jpg)
⢠Develop the application APK
Example âDragâ: Dev Engineering Workload
Detected LongPress
Detected Drag
DragView Move
Icon Selected
Create DragView
Icon Layout
Touch Release
Draw New Position
Log the position
Log the time
onTouch()
onDraw()
⢠Note: the touch event time-stamp is not the exact finger touch time. There is a few ms difference. It does not impact the drag lag distance optimization purpose.
2011-09-12 19
![Page 20: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/20.jpg)
Example âDragâ: Optimization Algorithm
⢠Analysis
â T0: the event is delivered to system
â T1: the app finishes drawing the first frame of movement
⢠T1 â T0 is the Android processing time, cannot be 0
⢠P1 â P0 is the distance finger moves during T1 â T0
⢠Two complementary optimization approaches
â Optimize the execution path to reduce T1 â T0
â Draw the icon at predicted position such as P2
Time T0 Time T1 Time T2 Position P0 P1 P2 P1
2011-09-12 20
![Page 21: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/21.jpg)
Example âDragâ: Predicting Drag Position
⢠Move icon to the finger position when next frame is drawn
â SPEEDfinger = (P1 â P0)/(T1 â T0)
â TIMEframe = 1/FPS
â MOVEfinger = SPEEDfinger * TIMEframe
â NextPOSfinger = MOVEfinger + CurrentPOSevent
â NextPOSicon = NextPOSfinger
⢠In reality
â POSITION = (x, y)
â Avoid icon surpassing finger
T1, P1 T0, P0
2011-09-12 21
![Page 22: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/22.jpg)
Optimize User Interaction Systematically
⢠What we need:
â A well-established methodology
â An engineering workload suite
â An analysis/tuning toolkit
â Sightings/requests/feedbacks from users
The key is to map user behavior into software metrics
2011-09-12 22
![Page 23: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/23.jpg)
Agenda
⢠Client device user interactions
⢠Android user interaction optimizations
â Android Workload Suite and Android UXtune
⢠Case studies of Android optimizations
⢠Factors that impact Android user interactions
⢠Summary
⢠Information
2011-09-12 23
![Page 24: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/24.jpg)
Android Workload Suite (AWS)
⢠Goals
â Reflect the representative usage of Android devices
â Evaluate Performance, Power and User interactions
⢠AWS usages
â Drive and validate Android optimizations
â Support comparative and competitive analysis
⢠(Details in another slide deck)
2011-09-12 24
![Page 25: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/25.jpg)
UXtune: An Analysis/Tuning Toolkit
⢠To analyze and optimize Android, we need
â Repeatable inputs operating the device
⢠Android input-Gestures
â Sequence of interaction events between the system components, such as event, frame, thread, etc.
⢠Android UXtune
â Metrics outputs characterizing the behavior
⢠Android meter-FPS
⢠Android app-launch
⢠Android touch-pressure
⢠(Details in another slide deck)
2011-09-12 25
![Page 26: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/26.jpg)
Agenda
⢠Business perspective and executive summary
⢠Client device user interactions
⢠Android user interaction optimizations
â Android Workload Suite and Android UXtune
⢠Case studies of Android optimizations
⢠Factors that impact Android user interactions
⢠Summary
⢠Information
2011-09-12 26
![Page 27: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/27.jpg)
Browser Scroll Response Time
Event1 Event2 âŚâŚ Event
k âŚâŚ
Input Manager
Browser Event1 Event2 âŚâŚ Event
k âŚâŚ
Frame 1
Gesture detecting Frame drawing
âŚâŚ
Response Time
2011-09-12 27
![Page 28: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/28.jpg)
Scroll Response Time Optimizations
⢠Gesture detecting time
â ACTION_DOWN event: record the start position
â ACTION_MOVE event: compute the move Distance
â Scroll is detected when Distance > Threadshold
⢠Frame drawing time
â Skipped in this talk
2011-09-12 28
![Page 29: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/29.jpg)
EventHub
InputManager InputReader InputDispatcher
InputChannel InputPublisher InputConsummer
MessageQueue
InputHandler
ViewRoot
AshMem
WebView
dispatchMotionEvent
User Input Event Dispatching
raw event
event
packaged move event
2011-09-12 29
![Page 30: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/30.jpg)
Move Events Throttling
⢠InputDispatcher throttles the move-event emission rate
â No need to emit move-event faster than platform maximal FPS
â Set a minimum time interval between move-events dispatching , commonly 1/FPS
â The move-events between two dispatches are grouped together
2011-09-12 30
![Page 31: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/31.jpg)
Experiment of Event Emission Throttling
⢠Without throttling, gesture detection time can be reduced by up to 1/fps in a common device
Events and gesture detection impact user interaction
Time of Event Throttling Delay Time to Browser with throttling
Time to Browser Without throttling
0 0 2 2 6 12.096 8
15 3.096 21 17 25 12.14 41 27
45 11.894 59 46 71 4.191 79 72 78 15.355 79
87 6.355 98 88
96 15.897 97
106 5.897 116
2011-09-12 31
![Page 32: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/32.jpg)
Agenda
⢠Client device user interactions
⢠Android user interaction optimizations
â Android Workload Suite and Android UXtune
⢠Case studies of Android optimizations
⢠Factors that impact Android user interactions
⢠Summary
⢠Information
2011-09-12 32
![Page 33: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/33.jpg)
Factors That Impact User Interactions
2011-09-12 33
SOC factors
⢠Memory bus bandwidth, HW gfx acceleration
⢠Touch screen pressure resolution
OS/Runtime
⢠Multi-core software synchronization, load balance
⢠Thread scheduling priority, UI thread vs. other computations
⢠Runtime engine design
Input factors
⢠Event emission rate
⢠Sensor event detection
⢠Touch gesture detection
Animation design
⢠Property animation design
⢠Application transition
⢠Gesture/Sensor inputs response
⢠Operation smoothness
⢠Drag coherence
![Page 34: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/34.jpg)
Agenda
⢠Client device user interactions
⢠Android user interaction optimizations
â Android Workload Suite and Android UXtune
⢠Case studies of Android optimizations
⢠Factors that impact Android user interactions
⢠Summary
⢠Information
2011-09-12 34
![Page 35: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/35.jpg)
Summary
⢠Performance is not enough to characterize a client device
⢠An systematic approach has been established to engineering user interactions
â AWS is a comprehensive engineering workload suite
â UXtune is a toolkit to assist analysis and optimization
⢠Android user interactions can be impacted by the technical factors across the whole software stack
User Interaction is a new challenge but not insurmountable
2011-09-12 35
![Page 36: Quantify and Optimize Android User Experience](https://reader030.vdocuments.site/reader030/viewer/2022012502/617bd378dbea4f6d421e5b1e/html5/thumbnails/36.jpg)
Information
⢠Online resource for reference
â http://ux.stackexchange.com/
â http://www.useit.com/papers/responsetime.html
â http://www.public.navy.mil/navsafecen/Documents/acquisition/MILSTD1472F.pdf
â http://www.measuringux.com/
2011-09-12 36