wearable technology for museums

Wearable Technology for Museums

MWF 2014

who we are? user experience and technology

Rokivo & Vidiemme

Rokivo Inc. is an American company based in New York and Turin, specialized in user experience, or the design of the experience on different types of screens (computer, smartphone, tablet, etc..)www.rokivo.com#werok

Vidiemme Consulting Srl is an Italian company based in Milan, key player in the digital context and focused on technology innovation for web, mobile and wearableswww.vidiemme.it#weroll

Rokivo and Vidiemme have brought to Italy one of the Google Glass Explorer Edition prototypes, the program opened so far only to US residents with the objective to develop new apps for this revolutionary device.

Giulio Caperdoni COOVidiemme Consulting

the speaker

Valerio SaffirioFounderRokivo

the speaker

what does it mean to develop for Google Glass?

two options to develop on Glass

Mirror API GDK

November 2013March 2013

The Google Mirror API allows you to build web-based services, called Glassware, that interact

with Google Glass, but do not run on it.

The Glass Development Kit (GDK) is an add-on to the Android SDK that lets you build Glassware that

runs directly on Glass.

Real-time User Interaction

Offline Functionality

Access to Hardware

Platform independence

Common infrastructure

Built-in functionality

Source: https://developers.google.com/glass/develop

focus on GDK

The GDK is an Android SDK add-on that contains APIs for Glass-specific features. By using the Android SDK and GDK, you can leverage the vast array of Android APIs and also design an optimal user experience for Glass.

Layouts, view and other UI

widgets

GPS, Sensors and other hardware

Activities, services and

other building block

Voice Cards Gesture Detector

Andr

oid

SDK

GD

K ad

d-on

Source: https://developers.google.com/glass/develop/gdk

because it allows to add functionalities, voice commands and gestures

why GDK is so important?

because it opens up endless possibilities to leverage all the “senses” of the device

because it allows a better integration with external services

designing for Glass

Glass has its own rules and guideline for design and use, that are fundamentally different from existing mobile and desktop platforms and that need to be taken into consideration when creating the user experience.

While computers and smartphones may store a medium to large quantity of data, Glass works best with information that is simple, relevant, and current.

Source: https://developers.google.com/glass/design/principles

UI elements allow 3 pattern types:

• Static card - Displays text, HTML, images, and video. Static cards can invoke live cards or immersions

• Live card – Show content that is important at the current time and constantly update to keep information fresh and relevant

• Immersion - Customized UI experiences that take over a user's focus and display outside of the timeline experience

forever last year last month now

GOOGLEGLASS4LIS - museum for deaf

what the LIS language is?

The LIS is the Italian Sign Language used by those deaf people that cannot speak or write Italian.Every community may have its own sign language; the Italian one is different from the one used in America for example and even in Italy there are different “dialects”.

LIS is based on “signs”. Signs are the results of different parameters:

“Mom’s car” becomes “Mom + Car + Hers”

Combining cheremes in new ways it is possible to create new signs.

Non manualFacial expression

Oral behavior Posture

ManualHand movements Hands orientation

PlaceConfiguration

Each sign is obtained combining basic units of meaning, called cheremes in Italian. Cheremes are ordered in accordance to LIS specific syntactic rules.

GoogleGlass4LIS

a project by

in collaboration with

sponsored by

GoogleGlass4LIS is the first experimental project to develop an App intended to

deaf people to provide a full and accessible museum experience

the concept

PLAY

how does it work?

Within the museum the deaf person wears Google Glass and looks at the statue of interest

Google Glass displays the avatar on its display

GoogleGlass4LIS App recognizes the statue based on its position

Information on the statue are linked with the corresponding translation produced by the ATLAS platform and rendered through an avatar signing the LIS

1

4 3

2

Statue of king Ramses II …

Final User Experience

the ATLAS platform

GoogleGlass4LIS is based on the ATLAS (Automatic Translation into Sign Languages) platform, a huge project started in 2009 funded by Regione Piemonte and the EU. The platform is able to translate any Italian text or audio to LIS. The platform creates in real time an avatar that can then be displayed on any digital device (smartphone, tablet and – of course – Google Glass).

Input:SubtitlesAudioAudio/Video

Text* Intermediate Format

“Signs” for the virtual avatar

*ATLAS core

Weather forecastTrain station

Museum (to be created for the project)…

SET of VOCABULARY

(r)evolutions

Build an App useful for many types of deafness, disabilities or educational

needs: Subtitles for non-LIS speaking, Children, Foreigners , …

Making the entire museum interactive with indoor positioning systems

press release

DOWNLOAD THE FULL PRESS RELEASE HERE

http://vidiemme.it/comunicati_stampa/GOOGLEGLASS4LIS%20-%20Rassegna%20stampa.pdf

indoor positioning systems

indoor location and positioning

• Bringing the power of GPS and maps indoor

• Tying digital information to physical places

• Instead of punching a 3 digit number into a handheld tour guide, you could walk up to a painting, pull out your smartphone, and find additional info right there waiting for you

• No single technology is ubiquitous

• There are trade-offs to each of the technologies. WiFi is low cost and ubiquitous, but not very accurate*; high precision location usually requires higher cost and infrastructure work

*High accuracy is considered to be 1 to 5 meters, medium accuracy is 6 to 10 meters, and low accuracy is over 11 meters

WiFi fingerprinting

• The localization technique is based on measuring the intensity of the received WiFi signal and generating a “fingerprint” for every different position

• Glass compares its current WiFi fingerprint to Fingerprint/Location database in order to find its position

• The accuracy depends on the number of positions that have been previously entered into the database

Accuracy• medium

Availability• ubiquitous

Glass issue• WiFi signal sampling

frequency is much slower than usual in order to safeguard battery life o localization takes

longer than with a smartphone

beacons

• They are low-cost gizmos that continually beam out a Bluetooth signal, running for up to a year on a small watch battery and located at defined positions within a building

• Beacons come in different formats - including small coin cell powered devices and USB sticks - and software versions

• A beacon could pinpoint an item’s location in a room

Accuracy• high

Availability• iOS7 – Android 4.3

Glass issue• Not compatible with current

Android version (4.0.4)• Temporary workaround:

1. Beacon is discovered by the smartphone paired with the Glass

2. Beacon Id is sent to Glass by the smartphone

contacts

Rokivo [email protected] - www.rokivo.com244 Fifth Avenue Suite F263, New York, NY 10001Rokivo Srl - Via Giolitti, 41 - 10123 Turin (TO)

contactsValerio Saffirio+39.3358485602/[email protected]

Vidiemme Consulting Srl [email protected] - www.vidiemme.itVia Natale Battaglia, 8 - 20127 Milan (MI)

contacts Giulio Caperdoni +39.3489154293/[email protected]