tizen apps with - amazon web services€¢ general preferences (time), short-term interests...
TRANSCRIPT
1 Tizen apps with Context Awareness, powered by AI by Shashwat Pradhan, CEO Emberify
2
Context refers to information that characterizes a situation, between: • Apps • People • Surrounding environment
• Making apps smarter and more relevant to every individual user by understanding them
Introduction
3
Contextual apps understand what’s going on in the user’s digital and physical world
Introduction
4
• Today, an average smartphone has about 10 sensors
• Contextual data: Current location, time, surrounding brightness, user activity
• User’s digital world: Apps being used, Facebook API, Instagram API
• Wearables & IoTs are bringing in many new data points
Introduction
5
• Alarm based on weather & traffic to work by location sensing
• Phone goes on vibrate based on proximity to office or a movie theatre
• Reminders based on travel tickets on email • Adaptive UI – App theme changes according to
surrounding brightness
Some contextual experiences
6
• Sense, understand and adapt
• Get user data from sensors or social networks • Build algorithms to understand the contextual
data • Personalize content & provide proactive
recommendations
Contextual Lifecycle
Sense
Understand
Adapt
7
Example of a context aware alarm app
• Senses the location of the user • Understands the current weather & traffic
through an API and usual waking up time • Adapts by letting the user sleep longer or
shorter based on these conditions
Contextual Lifecycle
Sense
Understand
Adapt
8
Tizen 2.4.0 on mobile came with a large set of Context APIs so developers don’t need to access sensors directly • Activity Recognition (Wearable Also) • Contextual History • Contextual Trigger • Gesture Recognition (Wearable Also)
Context with Tizen
9
• Activity Recognition – Stationary, walking, travelling, running with accuracy
• Contextual History – Device Usage Patterns like app usage, peak time and commonly used settings
• Contextual Trigger – Based on a contextual event the app or notification is triggered
• Gesture Recognition – Shake, Tilt, Snap, Orientation and other gestures are detected
Context with Tizen
10
• Apart from the data from the Context API’s, user data can be obtained from Social API’s
• Calendar and contacts can provide apps with information about the user’s physical world
• Using Sensor API’s sensors can be directly polled • For most cases Tizen Context API’s are doing the
sense and understand part for developers
Context with Tizen
11
• With Artificial Intelligence, context aware apps can be more proactive and intelligent
• AI algorithms can help make future context predictions
• Context awareness can make AI applications like Chatbots smarter
Context with AI
12
• Example- The chatbot needs to recommend the user places when it’s raining
• Chatbot: Since it is/is going to be raining outside I am recommending you indoor places
• Gives the Chatbot more human intelligence properties considering the user’s context
Context with AI
13
AI can help with contextual apps with: • Sensing with abstraction of data • Sensors can generate huge amounts of data from
which AI algorithms can help extract the relevant data
• Also, understanding data through audio, images and video can be done through AI branches like computer vision & speech recognition
Context with AI
14
• User Profiling • Gathering preferences of a user through sensors,
behavior, social networks or even explicitly • Personalizing the content and Adaptive nature of the
app according to different user profiles • AI techniques like association rules or case-based
reasoning can be used
Context with AI
15
• User Profiling • Example- News app profiles users to give relevant
articles based on user’s interests • General preferences (Time), Short-term interests
(Sporting event) & Long-term interests (Politics) • Keywords can be weighted to prioritize stories
Context with AI
16
• Context Reasoning • Based on the context situation an app may need to
adapt • Examining the contextual information and making a
decision based on rules and logic • The decision logic can further evolve with Machine
Learning algorithms
Context with AI
17
• ML algorithms learn from and make predictions on data
• ML algorithms work on models have to be made based on sample inputs
• Enables context prediction – which sensor data could be most important in the future
Context with Machine Learning
18
• Using a combination of sensors, Machine Learning models can be used to determine user activity
• Extract sensor data and train ML models • Multiple context data used together can give more
specific information about the user • Example Accelerometer & Barometer can be
used together to detect walking vs cycling
Context with Machine Learning
Sensor Data
Machine Learning
Server / On-device model
19
• ML algorithms make sense of noisy/conflicting data from sensors
• Large datasets are useful to train & fine tune Machine Learning models
• ML algorithms use raw sensor data to churn out signals based on training models like high level activities
Context with Machine Learning
20
Six technology forces powering contextual apps: • Mobile • Social Media • Sensor evolution • Cloud & Big Data • Wearable & other IoTs • Artificial Intelligence
Technology Powering Contextual Apps
21
• Launchify – App recommendation widget • Predicts which app the user needs right now in the
widget
• Context signals measured for prioritizing information: • User travelling • App Usage Patterns • Location
Case Study
22
• Launchify – App recommendation widget • Using Machine Learning algorithms to learn based
on user behavior • Simple weighing algorithm to give each contextual
parameter weight to priorities • Senses where, how long, how often, what situations
are the apps being used
Case Study
23
• Some common sense assumptions are needed in addition to the sensor data based on general human behavior to get more accuracy
• Sometimes sensors can give us conflicting data. • Use multiple sensors to confirm it • Simple logic can be applied to the algorithm like
repeating of a certain event occurrence before counting it to avoid random events
Experiences with Contextual Apps
24
• Proactive Recommendations • Recommending the user outdoor places on their lockscreen if
there's a chance of rain
• Lifelogging • Quantified Self apps to track the user’s life automatically
• Adaptive User Experience • Automatically changing the theme according to sorrounding
brightness
Use Cases
25
• Context is the secret sauce making an app smart & unique
• Foursquare doubled down on locations services to give proactive recommendations of food when you’re sitting at a restaurant
• The contextual fabric can provide a personalized experience
• New value for users can help apps find interest in App Stores crowded with millions of apps
Use Cases
26
• Contextual data is not always accurate
• Allow the user to correct and edit the contextual data
• Eg. Slow driving is often confused as cycling
• Machine Learning models take huge amounts of data to train for accuracy
Limitations
27
• With Context Aware apps you need to be transparent what the app is doing with the user’s data
• There needs to be a clear privacy policy • User’s should be able to disable the services • Encryption and security protocols need to be in
place
Limitations - Privacy
28
• Rather than providing the wow factor some contextual apps go over the freaky line
• Nokia’s Trapster (Similar to Waze) would allow it’s users to stalk other users accurately
• Huge user privacy & trust issues
Limitations - Privacy
29
• Sensors and background services can consume lots of battery life
• Data should be polled on triggers rather than a timer
• Rather than going to the sensor every time it would be more efficient to get data through an app that just polled the data
Limitations – Battery life
30
• More IoTs and wearables will bring in new sets of data and better quality too
• Smart Cars & Smart Homes will also add to user information
• Apps will be more automatic with better sensing • Contextual Apps will be more proactive in nature • Smartphone OS’s will take more contextual
information to become more intelligent
Future
31
• Apps will be ‘Headless’, will require minimum interface interaction
• Smart Notifications, voice and chatbots will be the new interfaces since apps will need lesser input with contextual information
Future
