Android AccessibilityBangalore Accessibility Week October 6-10, 2014 Ted Drake, Intuit Accessibility

This presentation borrows heavily from the great presentations given by the Google Accessibility team at Google iO: https://drive.google.com/folderview?id=0B8z0EqKVnjoZbEtZNk05LWhsVHM&usp=sharing Android Accessibility Videos: https://www.youtube.com/playlist?list=PLQbD187567ZYe2HQ24sQxB6jhnWoR_JUJ !photo: http://www.pinterest.com/pin/559501953680642594/

Users move accessibility focus by interacting with the touch screen using explore by touch and accessibility gestures. !

AccessibilityServices interpret exploration and gestures to facilitate different types of navigation

AccessibilityService Documentation: http://developer.android.com/reference/android/accessibilityservice/AccessibilityService.html An accessibility service runs in the background and receives callbacks by the system when AccessibilityEvents are fired. Such events denote some state transition in the user interface, for example, the focus has changed, a button has been clicked, etc. Such a service can optionally request the capability for querying the content of the active window. Development of an accessibility service requires extending this class and implementing its abstract methods.

Meet Talk BackTalkBack is preinstalled on most Android devices, but is also available on the Play Store1. !TalkBack is enabled from Settings > Accessibility > TalkBack. !Enabling TalkBack changes the interaction model of the device.

• Explore with a single finger.

• Double-tap to activate the focused item.

• Use two-finger drag to scroll lists.

Talkback is pre-installed, but can also be found in the play store: https://play.google.com/store/apps/details?id=com.google.android.marvin.talkback TalkBack support: https://support.google.com/talkback/ !

Turn on TalkBack

This video shows how to enable TalkBack and how to use it as a developer. It’s also available on YouTube https://www.youtube.com/watch?v=82ivNZI6bIA

Common Problems

• Content Labeling

• Grouping and Ordering

• Font Scaling

• Web Views and Hybrid Apps

• All interactive elements must have a content description:

•  android:contentDescription  

•  View#setContentDescription  

• Decorative images:

• android:contentDescription=“@null”

• For EditText

• android:hint  

• labelFor

• Keep labels short and clear. Send Email

You spend a lot of time making your screens look great, but are you providing the same experience to TalkBack users? contentDescription allows you to own the experience. Setting the contentDescription to null will tell talkback to ignore the image. Use android:hint on form inputs, as the contentDescription will inherit the value of the input.

<TextView

android:layout_height="match_parent"

android:labelFor="@+id/edit_item_name"

android:text=“Invoice amount"/>

<EditText

android:id="@+id/edit_item_name"

android:layout_height="wrap_content"

android:hint=“Invoice Amount"/>

This example shows labelFor and android:hint being used on an input. The hint is not announced when the edit text has a value. More information: https://www.ssbbartgroup.com/blog/2014/03/19/android-accessibility-properties-and-talkback/

Grouping and Ordering• TalkBack uses view hierarchy order and on-screen

positioning to determine grouping and ordering.

• Use ViewGroup containers to associate related information. Set android:focusable=“true”

• The order of items displayed may be controlled. Use android:contentDescription or View#setContentDescription

1 2

3 4

5 6

http://developer.android.com/reference/android/view/ViewGroup.html

Font Sizing

• Android supports large font substitution for low-vision users.

• Always use scale-independent pixels(sp).

• Avoid density-independent pixels (dip or dp)

• This is also important for i18n

WebViews

• Supported in Android 3.1+

• Enable JavaScript within WebView

• Garbage In, Garbage Out! Make sure your HTML is accessible.

http://developer.android.com/reference/android/webkit/WebView.html WebViews are not accessible by default, the user must activate this because it uses a JavaScript driven ChromeVox !• Turning on Accessibility mode, including 'Explore by Touch' in 4.0+. • Enabling 'Enhanced Web Accessibility', or, on older devices, 'Inject Web Scripts'.

Custom ViewsFavorite Actor

Dev Anand

Aamir Khan

Om Parkash

This pie chart could be considered a custom view. TalkBack needs to tell the user about positioning, color, text. It also need to describe interactions, such as gestures, taps, and states.

Virtual View Hierarchy• Pie Chart

• Om Parkash 30%

• clickable

• checkable

• Dev Anand 45%

• clickable

• checkable

• Aamir Khan 25%

• clickable

• checkable

Favorite Actor

Dev Anand

Aamir Khan

Om Parkash

What are the components? What are their states?

What TalkBack Sees• ViewDecor

• Action Bar

• Pie Chart

• LinearLayout

• My Favorite Bars

• Pie Chart

• Om Parkash 30%

• Dev Anand 45%

• Aamir Khan 25%

Favorite Actor

Dev Anand

Aamir Khan

Om Parkash

Custom Views

• Delegate accessibility support

• Map logical items to virtual views

• Expose information about views

• Formalize interaction model

setAccessibilityDelegate ViewCompat.setAccessibilityDelegate(this, mAccessibilityHelper); } @Override

public boolean dispatchHoverEvent(MotionEvent event) { if (mAccessibilityHelper != null && mAccessibilityHelper.dispatchHoverEvent(event)) { return true; } return super.dispatchHoverEvent(event); }

We want to check for mAccessibilityHelper support http://developer.android.com/reference/android/view/View.AccessibilityDelegate.html

Map Logical to Virtual

• Reuse your existing code.

• The pie chart already:

• Is drawn on the screen

• Includes touch events

• Uniquely identified

@Override

protected int getVirtualViewAt(float x, float y) { Wedge wedge = getWedgeAt(x, y); if (wedge != null) return wedge.getId(); return ExploreByTouchHelper.INVALID_ID; }

@Override

protected void getVisibleVirtualViews(List<Integer> virtualViewIds) { for (Wedge wedge : mWedges) { virtualViewIds.add(wedge.getId()); } }

Expose Information• Reuse existing code

• Position

• Color

• Label

• Relative Size

• Click Handling

• Checked Status

@Override protected void onPopulateEventForVirtualView( int virtualViewId, AccessibilityEvent event) { Wedge wedge = getWedgeForId(virtualViewId); event.setContentDescription(wedge.getLabel() + ": " + wedge.getPercent() + "%"); } @Override protected void onPopulateNodeForVirtualView( int virtualViewId, AccessibilityNodeInfoCompat node) { Wedge wedge = getWedgeForId(virtualViewId); node.setContentDescription(wedge.getLabel() + ": " + wedge.getPercent() + "%"); node.setBoundsInParent(wedge.getLabelBounds()); ...

Formalize Interactions! node.setCheckable(true); node.setChecked(wedge.isChecked()); node.addAction(AccessibilityNodeInfoCompat.ACTION_CLICK); } @Override protected boolean onPerformActionForVirtualView( int virtualViewId, int action, Bundle arguments) { if (action == AccessibilityNodeInfoCompat.ACTION_CLICK) { return onWedgeClicked(getWedgeForId(virtualViewId)); }

Assign your pre-existing actions to the accessibility node http://developer.android.com/reference/android/support/v4/view/accessibility/AccessibilityNodeInfoCompat.html !

Android L

• Custom AccessibilityAction

• Live Regions

• Collections

• Window API

Documentation is sparse for these new features. Presentation from Google IO 14 https://drive.google.com/folderview?id=0B8z0EqKVnjoZbEtZNk05LWhsVHM&usp=sharing https://developer.android.com/preview/api-overview.html#TestingA11y !

AccessibilityAction

• Swipes and other hard to discover actions

• Actions are activated from the Local Context Menu

• Provide hints for actions

Documentation for this has not yet been released.

Create AccessibilityAction/** * @param actionId The id for this action. This should either be one of * the standard actions or a specific action for your app. In that case it * is required to use a resource identifier. */ public AccessibilityAction(int id, CharSequence label) new AccessibilityAction(R.id.dismiss, getString(R.string.dismiss));new AccessibilityAction(ACTION_CLICK, getString(R.string.play_song)); !!// Constants for all the standard actions with default label: AccessibilityAction.ACTION_CLICK

Handling a Custom ActioneventView.setAccessibilityDelegate(new AccessibilityDelegate { @Override public onInitializeAccessibilityNodeInfo(View host, AccessibilityNodeInfo info) { super.onInitializeAccessibilityNodeInfo(host, info); info.addAction(new AccessibilityAction(R.id.dismiss, } @Override getString(R.string.dismiss))); public boolean performAccessibilityAction(View host, int action, Bundle args) { if (action == R.id.dismiss) {} // Logic for action } });

Live Regions

• Based on the Live Region experience in HTML + ARIA

• Content is announced when it changes or appears on screen

• Look for  TYPE_WINDOW_CONTENT_CHANGED

https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Live_Regions Documentation on live regions not available yet. TYPE_WINDOW_CONTENT_CHANGED: http://developer.android.com/reference/android/view/accessibility/AccessibilityEvent.html

Collections

• Similar to a data table in HTML

• Supports int, float, and Percent

• Query info about table and elements

• AccessibilityNodeInfo.CollectionInfo

http://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo.CollectionInfo.html

Window API• Introspect all interactive windows: autocomplete,

drawers…

• AccessibilityFocus can move from one window to another

• Retrieve information about properties of windows on the screen that sighted users can interact with.

• android.accessibilityservice.AccessibilityService.getWindows(): Retrieve a list of objects representing windows information

• TYPE_WINDOWS_CHANGED: AccessibilityEvent to see events

new features in

Android Testing

• Use Android Lint

• Use TalkBack’s Read From Top to make sure everything is announced on the screen

• Use the On-Screen Overlay to see what is being spoken

• Martini: Intuit’s UI Test Service

Using android lint: https://www.youtube.com/watch?v=OtwCe-YlD5k&list=PLQbD187567ZYe2HQ24sQxB6jhnWoR_JUJ&index=8 Touch exploration help: https://support.google.com/accessibility/android/answer/6006598?hl=en Martini: https://wiki.intuit.com/display/CTOTools/UI+Test+Service

Testing

• Use Android Lint

• Use TalkBack’s Read From Top to make sure everything is announced on the screen

• Use the On-Screen Overlay to see what is being spoken

Using android lint: https://www.youtube.com/watch?v=OtwCe-YlD5k&list=PLQbD187567ZYe2HQ24sQxB6jhnWoR_JUJ&index=8 Touch exploration help: https://support.google.com/accessibility/android/answer/6006598?hl=en !