Gesture This! Using the Gesture Interface to Create User-Friendly Touch DevicesDouglas BolingPresidentBoling Consulting Inc.
SESSION CODE: WEM304
About Douglas BolingIndependent consultant specializing in Windows Mobile and Windows Embedded Compact (Windows CE)
On-Site InstructionConsulting and Development
AuthorProgramming Embedded Windows CE
Fourth Edition
AgendaIntroducing Gestures
Basic Gesture Input
AutoGesture support
Physics Engine
Historic Touch ModelTouch panel produced mouse message
WM_LBUTTONDOWNWM_LBUTTONUPWM_MOUSEMOVE Only while touching panel
“Gestures” used for right clickTap and Shift KeyTap and Hold
These still work
GesturesGesture support enhances basic mouse interpretation
Nothing more than could be done before but done for youProvides consistent look and feel across all applications
All Windows controls enhanced to use gestures
Only need to use for custom windows where mouse manipulation was used before
Basic Gesture APIGestures are interpreted touch input
Either single or multiple touch input
Allows flick, pan, select gestures by defaultCustom gestures can be registered
Windows Embedded Compact Gesture API is Win32-basedFollows Windows 7 format
Gesture ActionsPan User presses moves finger and lifts
Scroll (flick) User presses, moves, then lifts finger while moving
Select User presses and lifts
Rotate Two finger press and rotate
Zoom Two finger pinch or spread
Enabling Gesture SupportTo enable gesture support call:
EnableGestures (HWND hWnd, ULONGLONG ullFlags, UINT wScope);Parameters:
hWnd Handle of window that supports gesturesullFlags Flags indicating which gestures you would like
PanHoldSelectDoubleSelect<Custom gestures>All
uScope Flags indicating gestures for specific window or entire application
EnableGestures typically called in WM_CREATE of main window
Using EnableGestures
//-------------------------------------------------// DoCreateMain - Process WM_CREATE message//LRESULT DoCreateMain (HWND hWnd, UINT wMsg, WPARAM wParam, LPARAM lParam) {
EnableGestures (hWnd, TGF_GID_ALL, TGF_SCOPE_PROCESS);return 0;
}
The WM_GESTURE MessageSent to a window when a gesture is recognized by the system
wParam Contains gesture command (Pan, Scroll, Select…)lParam Contains handle to gesture information
If WM_GESTURE not passed to DefWindowProc, you must call CloseGestureInfoHandle while processing the message
DefWindowProc handles WM_GESTURE by passing to parent windowDON’T forward WM_GESTURE to one of your child windows!
GetGestureInfoCalling GetGesutureInfo returns a GESTUREINFO structure
typedef struct tagGESTUREINFO { UINT cbSize; // Size of structure DWORD dwFlags; // GF_BEGIN/GF_INERTIA/GF_END DWORD dwID; // Gesture ID HWND hwndTarget; // Target Window handle POINTS ptsLocation; // Location of gesture DWORD dwInstanceID; // Reserved DWORD dwSequenceID; // Timestamp of Gesture ULONGLONG ullArguments; // Args. See next slide UINT cbExtraArguments; // Possible extra data} GESTUREINFO, *PGESTUREINFO;
GESTUREINFO dwFlagsdwFlags
GF_BEGIN Beginning of gestureGF_END End of gestureGF_INERTIA Indicates “Flick” has happened at end of Pan
Flags can be combined or zero
Multiple End flags can occurOne for each gesture recognized
GESTUREINFO ulArgumentsullArguments Depends on message
Notification: Interim Pan notificationsHIWORD () X delta from previous messageLOWORD () Y delta from previous message
Notification: GF_END + Pan or ScrollGID_SCROLL_ANGLE() Angle of gesture in radiansGID_SCROLL_DIRECTION() Direction of gesture (Up/Down/Left/Right)GID_SCROLL_VELOCITY() Speed in Pixels per second
WM_GESTURE messagesDouglas BolingPresident Boling Consulting Inc.
DEMO
AutoGestureSimple way to enable window scrolling by pan/flick gestures
Enable by adding Horizontal and/or Vertical scroll bars to your window
No need to handle WM_GESTURESimply respond to WM_VSCROLL and WM_HSCROLL messages
OS needs to be built with AutGesture component
Configuring AutoGestureWhile AutoGesture works ‘by default’ it can be configured
Used to customize drawing and presentation given pan and flick gestures
FunctionsSetWindowAutoGesture Configure gesture response and redraw
requirements for a windowGetWindowAutoGesture Query Autogesture settingsGetAnimateMessageInfo Get information about an AutoGesture
message
WAGINFO StructureGet/SetWindowAutoGesture use WAGINFO structure
typedef struct tagWAGI { size_t cbSize; // Size of structure DWORD dwFlags; // See next slide UINT nOwnerAnimateMessage; // ID of WM_USER msg or 0 UINT nAnimateStatusMessage; // ID of WM_USER msg or 0 HBRUSH hExtentBrush; // Reserved UINT nItemHeight; // Used for snapping to item UINT nItemWidth; // Used for snapping to item BYTE bHorizontalExtent; // Scroll ext past last item BYTE bVerticalExtent; // Scroll ext past last item} WAGINFO, *LPWAGINFO;
WAGINFO FlagsFlags
WAGIF_OWNERANIMATE Must set to enable custom responseWAGIF_VSCROLLABLE Enable vertical scrollingWAGIF_HSCROLLABLE Enable horizontal scrollingWAGIF_LOCKAXES Lock gestures to horizontal and verticalWAGIF_IGNOREPAN Don’t send Pan gesturesWAGIF_IGNORESCROLL Don’t send Scroll gestures
AutoGesture Custom MessagesOwnerAnimateMessage
WM_USER+x messages notifying window of gesture actionRedraw window when received. Data queried with GetAnimateMessageInfo()
AnimateStatusMessageWM_USER+x message notifying of gesture status change
WAG_STATUS_ANIMATION_START Start of animationWAG_STATUS_ANIMATION_END End of animationWAG_STATUS_ANIMATION_CHANGE Change from Pan to flick/scroll
Either message can be disabled by setting field to 0 in WAGINFO
The Physics EngineMakes movement look ‘natural’
Things bounce, spring back, and slow down naturally
How is this done?Physics!
The physics engine computes a series of point/time combinations to allow the application to animate movement in a natural looking way
Using the Physics EngineThe physics engine is controlled by 4 functions
CreatePhysicsEngine Create and initialize the engineQueryPhysicsEngine Query position and motion parameters
given a time
SetPhysicsEngineUserTime Set time for queryDestroyPhysicsEngine Destroy engine and free resources
Initializing the Physics Enginetypedef struct tagPHYSICSENGINEINIT{ DWORD cbSize; // Size of the structure DWORD dwEngineType; // Reserved, set to zero DWORD dwFlags; // Specifies system or User time base LONG lInitialVelocity; // Initial velocity in Pixels/sec DWORD dwInitialAngle; // Angle in radians BYTE bXAxisMovementMode; // Horz movement action (Decelerate only opt) BYTE bXAxisBoundaryMode; // Horz boundary action. Bounce/rubber band/None BYTE bYAxisMovementMode; // Vert movement action (Decelerate only opt) BYTE bYAxisBoundaryMode; // Vert boundary action. Bounce/rubber band/None RECT rcBoundary; // Entire movement rectangle SIZE sizeView; // Rectangle of visible region POINT ptInitialPosition; // Initial position in region SIZE sizeItem; // Size of item. Used to snap to item boundary} PHYSICSENGINEINIT;
HRESULT CreatePhysicsEngine(PHYSICSENGINEINIT* pEngineInit, HPHYSICSENGINE* phResult);
Using the Physics EngineInitialize Physics engine with values returned by Flick (Scroll) gesture
Units match so easily done
Create timer to send messages back to window to animateSetTimer()
When WM_TIMER message received, query physics informationUse QueryPhysicsEngine()Redraw with returned Physics position data
When physics status indicates complete, kill timer message
Using the Physics Engineif (gi.dwFlags & (GF_INERTIA | GF_END)) { PHYSICSENGINEINIT pei; memset (&pei, 0, sizeof (pei)); pei.cbSize = sizeof (pei);
pei.dwFlags = 0;pei.lInitialVelocity = GID_SCROLL_VELOCITY (gi.ullArguments);pei.dwInitialAngle = GID_SCROLL_ANGLE (gi.ullArguments);pei.bXAxisMovementMode =
PHYSICSENGINE_MOVEMENT_MODE_DECELERATE; pei.bXAxisBoundaryMode =
PHYSICSENGINE_BOUNDARY_MODE_RUBBERBAND;pei.bYAxisMovementMode =
PHYSICSENGINE_MOVEMENT_MODE_DECELERATE; pei.bYAxisBoundaryMode =
PHYSICSENGINE_BOUNDARY_MODE_RUBBERBAND;pei.rcBoundary = g_rcWork;pei.sizeView.cx = rectImg.right;pei.sizeView.cy = rectImg.bottom;pei.ptInitialPosition.x = gi.ptsLocation.x;pei.ptInitialPosition.y = gi.ptsLocation.y;pei.sizeItem.cx = 1;pei.sizeItem.cy = 1;
hr = CreatePhysicsEngine (&pei, &g_hPEAnimate);SetTimer (hWnd, ID_ANIMATE, 100, NULL);
}}
WM_TIMER with Physics Enginecase WM_TIMER: if (wParam == ID_ANIMATE) { // Query physics engine
PHYSICSENGINESTATE state;memset (&state, 0, sizeof (state));state.cbSize = sizeof (state);QueryPhysicsEngine (g_hPEAnimate, &state);
// Update imageMoveImg (hWnd, state.ptPosition);
// If animation complete, clean upif (state.fComplete){
DestroyPhysicsEngine (g_hPEAnimate);KillTimer (hWnd, 1);
} } break;
Managed SupportNo direct support through managed class library today
AutoGesture works by default
Physics is a simply P/Invoke
Managed framework sample available on codeplex
SummaryGesture support fairly straightforward
Model consistent with desktop API
Autogesture gives simple gesture support for most controls by defaultCustomization available for controls that need more than simply scrolling
Physics engine pretty simple to useCool effects
QuestionsDouglas Bolingwww.bolingconsulting.comdboling @ bolingconsulting.com
Track Resourceshttp://www.WindowsEmbedded.comhttp://msdn.microsoft.com/en-us/windowsembeddedhttp://social.msdn.microsoft.com/Forums/en-US/category/embeddedwindows/http://social.msdn.microsoft.com/Forums/en-US/category/windowsembeddedcompacthttps://connect.microsoft.com/windowsembeddedce
Related ContentBreakout Sessions
WEM201 | Discover Windows Embedded Standard 7 as Your Next Application PlatformWEM301 | Deploying Windows Embedded Standard 7 with StyleWEM302 | Explore the Multimedia Potential of Windows Embedded Standard 7WEM303 | Gamechanger: Using Microsoft Silverlight for Windows Embedded to Create an Amazing Embedded UIWEM305 | How to Choose a Windows Embedded Operating SystemWEM306 | Using the Sensor & Location API on Windows Embedded Standard 7 to Create Exciting Connected ApplicationsWEM307 | Windows Embedded Compact: New Tools and Developer StoryWEM308 | Windows Embedded Overview: Demos of the Latest and Upcoming ReleasesWEM309 | Programming Microsoft Silverlight for Windows Embedded Using Microsoft .NET
Interactive SessionsWEM01-INT | Build a Secure Device with Windows Embedded Standard 7WEM02-INT | Delivering Flexible Peripheral Support for Point of SaleWEM03-INT | How Windows Embedded Solutions Help to Protect the EnvironmentWEM05-INT | What a Desktop Developer Needs to Know to Develop for Windows EmbeddedWEM06-INT | Windows Embedded Compact CompeteWEM07-INT | Server Appliances with Windows Embedded ServersWEM08-INT | Roundtable: Windows Embedded @ Tech·Ed 2011 - Tell Us What You Want to Learn
Related ContentHands-on Labs
WEM01-HOL | Build Your Own Embedded SystemWEM04-HOL | Porting Third-Party Drivers into Image Configuration Editor
Product Demo Stations (all on Windows Embedded booth)TLC-46 | Get Your Hands on Windows EmbeddedTLC-47 | Powered by Windows Embedded POSReady – Touch ScreenTLC-48 | The Intel® Intelligent Digital Signage Proof of ConceptTLC-49 | Windows Embedded AutomotiveTLC-50 | Windows Embedded Device Showcase
Resources
www.microsoft.com/teched
Sessions On-Demand & Community Microsoft Certification & Training Resources
Resources for IT Professionals Resources for Developers
www.microsoft.com/learning
http://microsoft.com/technet http://microsoft.com/msdn
Learning
Complete an evaluation on CommNet and enter to win!
Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31st
http://northamerica.msteched.com/registration
You can also register at the
North America 2011 kiosk located at registrationJoin us in Atlanta next year
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to
be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
JUNE 7-10, 2010 | NEW ORLEANS, LA