Mobile Interactionwith the Real World

Dr. Enrico RukzioLecturer in Mobile HCIComputing DepartmentLancaster University (UK)

Outline

• Mobile interaction with the real world

– Overview, related research areas, classification

• Focus on

– Touching (NFC based interaction techniques)

– Pointing (personal projectors)

– Scanning and user mediated object selection

• Application areas

• Summary & Conclusion

2

Mobile Interaction with the Real World

sensors in mobile device interaction with augmented and not augmented “things”

Location based services (e.g. tour guide, city guide, mobile navigation)

Exchange (images, audio files, messages), Play (mobile gaming) proximity of users

[Kindberg et al. 2002, Rukzio 2007]

to {call, text, play, surf the internet}, PIM

Related Research Areas

[Rukzio 2007] based on [Rekimoto and Nagao 1995]

RR

C

Before the computer Usage of everyday computers

(laptop, mobile phone)

R

C

Mobile Augmented Reality

R

Ubiquitous Computing

(Real World Computer)

R

C

Virtual Reality

R

C

Mobile Interaction

with the Real World

CC

Human – Computer - Interaction

Human – Real World - Interaction

Real World – Computer - Interaction

Computer – Computer - Interaction

C – Computer

R – Real World

CC

Mobile Interaction with the Real World: Interaction Techniques

Communication:

Bluetooth, GPRS,

UMTS

No direct linkLocation:

Bluetooth, WLAN,

GPS

Visual: visual

marker, light

beam, IrDA

Radio: RFID,

NFC, proximity

sensors

Device -

Smart Object

Interaction

Line of sight,

often indoors

Line of sight and

readable

Maximal distance

object - mobile

device: 100m

Distance object -

mobile device:

10 cm...10 m,

line of sight

Distance object -

mobile device:

0...10 cm, line of

sight

Real world

aspects

Illustration

The user controls

a remote display

with a mobile

device.

The user types in

information

provided by the

object to establish

a link between

them.

A link between

mobile device and

smart object is

established

because of their

proximity.

The user points

on a smart

object with a

mobile device to

establish a link.

The user touches

a smart object

with a mobile

device to

establish a link.

Description

Indirect Remote

Controls

User-mediated

object interaction

ScanningPointingTouchingInteraction

technique

Communication:

Bluetooth, GPRS,

UMTS

No direct linkLocation:

Bluetooth, WLAN,

GPS

Visual: visual

marker, light

beam, IrDA

Radio: RFID,

NFC, proximity

sensors

Device -

Smart Object

Interaction

Line of sight,

often indoors

Line of sight and

readable

Maximal distance

object - mobile

device: 100m

Distance object -

mobile device:

10 cm...10 m,

line of sight

Distance object -

mobile device:

0...10 cm, line of

sight

Real world

aspects

Illustration

The user controls

a remote display

with a mobile

device.

The user types in

information

provided by the

object to establish

a link between

them.

A link between

mobile device and

smart object is

established

because of their

proximity.

The user points

on a smart

object with a

mobile device to

establish a link.

The user touches

a smart object

with a mobile

device to

establish a link.

Description

Indirect Remote

Controls

User-mediated

object interaction

ScanningPointingTouchingInteraction

technique

[Välkkynen et al. 2003]

[Rukzio 2007]

Touching: Introduction

• Select and interact with object by touching it with the mobile device

– Intuitive, direct interaction, user has to be nearby, augmentation

• [Want et al. 1999] (Xerox PARC)

– One of the first who presented a prototype: RFID tags + RFID reader connected to a mobile device(tablet computer)

– Applications: augmented books, documents and business cards RFID tags provide links to corresponding services ordering a book / picking up an email address

• [Välkkynen et al. 2003] (VTT)

– Further implementation called TouchMe

– Based on proximity sensors (IR) which sense the distance between augmented object and mobile device

7

Touching: NFC / RFID

• Products & prototypes based on short range passive RFID and Near Field Communication (NFC) [Want 2006]

• NFC (Near Field Communication)

– Short range data communication technology (13.56 MHz)

– Standardized: Near Field Communication - Interface and Protocol NFCIP 1/2 (ECMA-340, ECMA-352, ISO/IEC 18092)

– Compatible to MIFARE (ISO/IEC 14443A), FeliCa (complies with ISO/IEC 18092) and ISO/IEC 15693.

– Several NFC phones available (e.g. Nokia 6212/6131 NFC)

– Future

• Håkan Djuphammar (Ericsson's VP of systems architecture): “A year from now, basically every new phone that's sold will have [Near Field Communication]” [Sherwood 2009]

• Costs to add NFC to phones $1 by 2012/2013 [Clark 2009]

• http://www.nfc-forum.org/news/

Touching: NFC interaction styles

NFC Device

NFC Device

NFC Device

NFC Chip

NFC Chip

NFC Chip

Tag

Information

Interaction Point

Reader

NFC Device

NFC Chip

RFID

Contactless smart card

Bluetooth

[Rukzio 2007]

Touch based mobile applications

9

Tag / Display Static Dynamic

Single Interact with sensors, payment and ticketing readers

Interaction with laptops, access control, payment readers

Multi Marked-up maps, Touch & Interact

Touch & interact

• Tag: How many tags are on the object?

• Single: Link to corresponding service

• Multi: Link to many services / options

• Display: Which kind of object is touched?

• Static: e.g. poster or newspaper, no feedback provided by display

• Multi: e.g. projection or LCD display, feedback by display and mobile phone

Single tag / static display

10

[Touch & Travel]

[VTT 2007]

[missphones 2009]

[missphones 2009]

Single tag: NTT DOCOMOFeliCa, Osaifu-Keitai & ToruCa

• i-mode FeliCa allows a mobile phone to perform the function of traditional Sony FeliCa cards with additional functionality

• Osaifu-Keitai are ‘mobile wallets’ that use the system

• ToruCa: coupon service

• Restaurant flyers

• Promotional coupons

• Compatible handsets in Japan (2009)

• DoCoMo: 29 million

• Softbank: 10 million

• KDDI: >10 million

• Mobile wallet supportedby 640.000 stores

[i-mode FeliCa]

Single tag / dynamic display

12

[ECMA 2004]

[Dhiram 2007]

[Dhiram 2007]

• Selection– M locations

– 1 out of M

– N out of M

Multi tag / static display: Marked-up Maps

[Reilly et al . 2006]

Multi tag / static display: Touch & Interact

[Hardy and Rukzio 2008]

• Selection

– M meals

– 1 out of M

Multitag / static display: Meal service for elderly people

[Häikiö et al. 2007]

Multitag / static display: RFID McDonalds

• RFID dongle augments a standard Korean mobile phoneswith a standard hardware interface

• ‘Touch order’ is a system whichallows selection and payment to be made from a menu at every table

• Secure NFC phone applications areused for billing (phone bill) and data services

• RFID cards are trialed in the US for useat the drive-thru.

• Selection, guidance ?

16[Nearfield 2007]

Multitag / static display: cinema poster

• Tasks– Order movie ticket

– View movie details

• Selections– C cinemas, P persons,

T timeslot

– [1, X] out of C

– [Y] out of P

– [Z] out of T

• Guidance: visual on theposter, on mobile phone

[Broll et al. 2007]

Multitag / dynamic displayTouch & Interact

• User interface: mobile phone& dynamic display

[Hardy and Rukzio 2008]

[Hardy andRukzio 2008]

Multitag / dynamic displayTouch & Interact

Multitag / dynamic displayTouch & Interact

Coordinates assigned to each tag in the mesh

[Hardy and Rukzio 2008]

Multitag / dynamic displayTouch & Interact

• Simple upload and download of picturesfrom mobile phone to laptop

21

upload

download[Seewoonauth et al. 2009]

NFC: granularity issue

• Problem: NFC tag size & phone size

[Hardy and Rukzio 2008]

Pick & Drop

[Rekimoto 1997]

Touch based mobile interactionwith interactive surfaces

Visual marker (Byte and Identity Tags)

BlueTable (Shape detection & IRDA blinking)1. Detect the placement of a new phone-shaped object by visual means.2. For each switched-on Bluetooth device:• a. Attempt to connect to the device over Bluetooth. Continue if the

device advertises globally unique identifier (GUID), else move on to thenext device.

• b. Command the device to blink its IRDA (infrared) port.• c. If the blink is detected at the position of the object go to step 3,

else move on to the next Bluetooth device.3. Determine the exact orientation of the device (optional).

24

[Carpenter 2008, surface.com]

[Wilson & Sarin 2007]

25

• Select or control an object by pointing on it with the mobile device

• Intuitive, direct interaction, user has to be nearby, often augmentation

• [Fitzmaurice 1993] (University of Toronto)

• Using mobile devices for pointing based interactions to interact with related services

• Application: map on which the user can point to get additional information / a computer augmented library

• [Rekimoto, Nagao 1995] (Sony Computer Science Laboratory)

• NaviCam project / prototype

• Visual markers interpreted by a camera attached to the mobile device

• Markers: 4-bit visual colour codes (sequence of red and blue stripes), size of 3cm x 5cm

• Distance between device and object: 30-50 cm

Pointing: Introduction

Pointing: Implementations

Principle Visual Marker Image

Recognition

Light beam Personal

projector

Infrared Recognizing the

mobile device

Illustration

References [Rekimoto and

Nagao 1995]

[Föckler et al.

2005]

[Välkkynen et

al. 2003]

[Raskar et al.

2004]

[Ailisto et al.

2003]

[BahnHandyTick

et, Miyaoku et

al. 2004]

Advantages Makers are

simple,

inexpensive,

disposable. No

power supply.

Smart objects

do not need to

be augmented.

No power

supply.

Very natural

interaction

(remote control)

Project user

interface onto

object

IrDA is

integrated in

many mobile

devices.

Handy

possibility for

identification

(tickets, etc.)

Disadvan-

tages

Visual

obtrusiveness,

limited storage

capabilities

Great demands

on image

recognition and

data model.

Smart object

must provide a

feedback

channel (RF,

Bluetooth, etc.)

Tracking Smart object

must be

enhanced by

IrDA

functionalities.

Code (e.g. paid

ticket) can not

be shown when

phone is out of

power.

[Rukzio 2007]

27

Pointing: Implementations: Visual Marker

• Uses the built-in camera of mobile phones, different marker types

• NTT DoCoMo / 2003

– first mobile phones (505i Series) with a preinstalled QR code application, 30 million mobile phones with QR code suppport [Fowler 2005]

• Two-dimensional codes

– Can store more information then one-dimensional codes (EAN-13 bar codes)

– Comprehensive overviewin [Rohs 2005]

• Application areas:

– Advertisement posters,magazines, newspapers

NaviCam

[Rekimoto,

Nagao 1995]

QR Codes

[QRCode]

Visual Codes

[Rohs and

Gfeller 2004]

28

Pointing: Implementations: Image recognition

• [Fritz et al. 2004] (Joanneum Research, Graz)

– System for outdoor object recognition with camera equipped PDA

– Analysis of the focused object by server

• [Föckler et al. 2005] (Bauhaus University)

– PhoneGuide: museum guide

– Mobile phone & on phone objectidentification

• Nokia Point & Find

• Advantages

– Smart object does not has to be augmented. No power supply on the smart object is needed

• Disadvantages

– Great demands on image recognition capabilities and data model representing the smart object

29

Pointing: Implementations: Light beam

• [Välkkynen and Tuomisto 2005] (VTT)– Light sensors attached to poster

– Laser pointer on mobile device

– Feedback channel: RF

• [Rukzio et al. 2006]– Light sensors attached to DVD player, radio, laptop, thermostat

– Feedback channel: GPRS/UMTS

• Advantages– Very natural interaction

• Disadvantages– Smart object has to be enhanced by a communication channel

(RF, Bluetooth, etc.) feedback channel

– Mobile device with laser pointer

30

Pointing: Mobile Projectors

• Projector phone and accessory projectors available

Epoq EGP-PP01 Projector Phone Samsung i7410 Projector Phone

Optoma Pico Projector

Aiptek Pocket Cinema V10

Handheld projector: Interaction

• Mouse pointer interactionwith a “stabilized” web browser

• Projecting on the a fuse box: showing the related rooms, selecting a particular room on the projection

• Hold & Drag operation to select a certain area

[Beardsley et al. 2005]

Handheld projector: RFIG Lamps

[Raskar et al. 2004]

• Wireless tags: radio frequency identity and geometry (RFIG) transponder with photo sensor

• Geometry: location & shape

Handheld projector: Multi-user

• Two peep holes into the same virtual layer

• Passing objects, creating a large projection area, overlapping projections (calendar), focus & context

[Cao et al. 2007]

Pointing: Sixt Sense

http://www.pranavmistry.com/projects/sixthsense/

Pointing: Brainy hand

[Tamaki et al. 2009]

Projector Phone research

• Interaction design: mobile phone screen & projection – Where to display what?– Problem: context switching– How to interact with

projection?

• Personal projectors in the wild: acceptance, social protocols, usage [Greaves et al. 2009]

• Projector phone games: environment (objects, lines) part of the playing field[Löchtefeld et al. 2009]

36

[Hang et al. 2008, Greaves et al. 2008]

37

Pointing: Implementations: IrDA

• MobilePoint [Mobilepoint]

– Mobile phone receives a command via IrDA to send a SMS to the advertiser

– Advertiser sends then a SMS including an URL, Number, etc. to the user can be used for theparticipation in a lottery

– User should read the small print, agree to receive further information

• Advantages– IrDA is integrated in many mobile devices

• Disadvantages– Smart object has to be enhanced

by IrDA functionalities

38

Pointing: Implementations: Recognizing the mobile device

• Visual marker as Ticket [BahnHandyTicket]– User gets the ticket in form of an visual marker included

within an MMS.

– Train conductor checks the visual marker

• C-Blink system [Miyaoku et al. 2004]

– Direct interaction with a remote display

– Mobile phone acts as a visible light source by dynamically changing its displayed information

Sensed by a camera attached to the remote display trough which the position and the movement of the mobile phone can be sensed.

• Advantages– Handy possibility for identification (tickets, etc.)

• Disadvantages– Code (e.g. paid ticket) can not be shown when phone is out of power

39

Scanning

• Based on the proximity of nearby objects (places, smart objects)– Triggered by the user (Bluetooth)

– Environment is permanently scanned (Mobile tourist guide)

Result: List of nearby objects, select one, using the provided services

• Concept: tricorder [WikipediaTricorder] – Star Trek television series (1966-1969)

– Handheld device equipped with several sensors, was usedfor scanning unknown environments, to diagnose a patientor for interactions with smart objects or computers

• Prototypes / Products– CyberGuide [Abowd et al. 1997] and Lancaster Guide project [Cheverst

et al. 2000]: mobile context-aware city or tourist guides

– NTT DoCoMo i-area (2001): information (restaurants, location of the user, local whether, local news) about points of interest [i-area]

• Technologies– Bluetooth, RF, WI-FI, GPS, ultrasound, etc. [Küpper 2005]

40

User mediated object selection

• User types in a link to a corresponding service

– Link: number, URL, phone number, etc.

– No specific device is needed, efficient for small number of options (museum), frustrating when typing in an URL

– Example: Typing in a number in a mobile guide [BUGAbutler]

41

Application Areas

• Active posters & advertising– Posters, flyers, business cards and announcements

present already the needed information augmentation representing a link

– Purchase ring tones, wallpapers or music [J-Ware, Mobipoint, NFCCaen, PhilipsNFC]

– ToruCa service available in Japan [ToruCa]: touch a ToruCa reader / writer with the Osaifu-Keitai phone,to get a coupon or a flyer

• Tourist and museum guides– Getting information about places, buildings and exhibits

– Indoor (e.g. a typical museum, exhibition or gallery) and outdoor guides (e.g. horticultural show, park or garden)

– BUGA butler [BUGAbutler], NFC technology to getinformation about landmarks [NFCCaen], PhoneGuide museum guide [Föckler et al. 2005], Semapedia [Semapedia], Google and Nokia maps

42

Application Areas

Electronic key and ticketing• Identify or to prove that she has a valid ticket

enter a building or a room

• Easily transferable, do not need space, can be readby another device

• Osaifu-Keitai phones: electronic tickets, membershipcards and airline tickets [Osaifu-Keitai], access code [PhilipsNFC], public transport ticket [BahnHandyTicket]

Payment• Mobile phones acts as an electronic wallet or

provides access to the credit card or bank account

• Osaifu-Keitai phones: credit card [Osaifu-Keitai], paying parking fees [NFCCaen] or buy a soft drinkat a vending machine [cmode]

43

Application Areas

• Peer-to-Peer

– Exchange of images, audio files or synchronizing address books / downloading a gaming from a laptop [ECMA_NFC 2004, PhilipsNFC]

– Nokia Sensor: peer-to-based social interaction between different mobile phone users [NokiaSensor, Persson and Jung 2005]http://www.nokia-asia.com/A4416020

44

Summary & Conclusion

• Summary

– Mobile interaction with the real world: overview & classification

– Focus on: touching (NFC), pointing (projection), scanning and user mediated object selection

• Conclusion– Trend towards mobile interaction with the real world

– Mobile phone = universal platform with many built-in sensors and communication capabilities: camera, microphone, NFC/RFID, GPS, Bluetooth/WLAN/IrDA, GPRS/UMTS, sensors

– Asia (Japan, Korea): 3-5 years ahead of Europe / US few English publications blogs of tourists

– Innovations happen first in Asia (Japan/Korea/China) not in Europe or US• i-area (2001), Osaifu-Keitai (2004) , ToruCa (2005)

– Next big things (?): projector phones, NFC

[Christian 2002]

45

Communication:

Bluetooth, GPRS,

UMTS

No direct linkLocation:

Bluetooth, WLAN,

GPS

Visual: visual

marker, light

beam, IrDA

Radio: RFID,

NFC, proximity

sensors

Device -

Smart Object

Interaction

Line of sight,

often indoors

Line of sight and

readable

Maximal distance

object - mobile

device: 100m

Distance object -

mobile device:

10 cm...10 m,

line of sight

Distance object -

mobile device:

0...10 cm, line of

sight

Real world

aspects

Illustration

The user controls

a remote display

with a mobile

device.

The user types in

information

provided by the

object to establish

a link between

them.

A link between

mobile device and

smart object is

established

because of their

proximity.

The user points

on a smart

object with a

mobile device to

establish a link.

The user touches

a smart object

with a mobile

device to

establish a link.

Description

Indirect Remote

Controls

User-mediated

object interaction

ScanningPointingTouchingInteraction

technique

Communication:

Bluetooth, GPRS,

UMTS

No direct linkLocation:

Bluetooth, WLAN,

GPS

Visual: visual

marker, light

beam, IrDA

Radio: RFID,

NFC, proximity

sensors

Device -

Smart Object

Interaction

Line of sight,

often indoors

Line of sight and

readable

Maximal distance

object - mobile

device: 100m

Distance object -

mobile device:

10 cm...10 m,

line of sight

Distance object -

mobile device:

0...10 cm, line of

sight

Real world

aspects

Illustration

The user controls

a remote display

with a mobile

device.

The user types in

information

provided by the

object to establish

a link between

them.

A link between

mobile device and

smart object is

established

because of their

proximity.

The user points

on a smart

object with a

mobile device to

establish a link.

The user touches

a smart object

with a mobile

device to

establish a link.

Description

Indirect Remote

Controls

User-mediated

object interaction

ScanningPointingTouchingInteraction

technique

[Välkkynen et al. 2003]

Questions

References

• [Rukzio 2007] Enrico Rukzio. Physical Mobile Interactions: Mobile Devices as Pervasive Mediators for Interactions with the Real World.PhD Dissertation. Faculty for Mathematics, Computer Science and Statistics. University of Munich. 2007.

• [Rekimoto, Nagao 1995] Jun Rekimoto ; Katashi Nagao: The World Through the Computer: Computer Augmented Interaction with Real World Environments. In: Proceedings of Proceedings of the 8th ACM Symposium on User Interface Software and Technology ( UIST '95) (1995). http://www.csl.sony.co.jp/person/rekimoto/uist95/uist95.html

• [Välkkynen et al. 2003] Pasi Välkkynen; Ilkka Korhonen; Johan Plomp; Timo Tuomisto; Luc Cluitmans; Heikki Ailisto ; Heikki Seppä: A user interaction paradigm for physical browsing and near-object control based on tags In: Proceedings of Physical Interaction (PI03) http://www-alt.medien.ifi.lmu.de/en/events/pi03/papers/valkkynen.pdf.

• [Ailisto et al .2003] Heikki Ailisto; Ilkka Korhonen; Johan Plomp; Lauri Pohjanheimo; Esko Strömmer: Realising Physical Selection for Mobile Devices. Physical Interaction (PI03) – Workshop on Real World User Interfaces in conjunction with the Fifth International Symposium on Human Computer Interaction with Mobile Devices and Services (Mobile HCI 2003), Udine, Italy, 2003. http://www-alt.medien.ifi.lmu.de/en/events/pi03/papers/ailisto.pdf

• [Want et al. 1999] Roy Want; Kenneth P. Fishkin; Anuj Gujar ; Beverly L. Harrison: Bridging physical and virtual worlds with electronic tags, in Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit. 1999, ACM Press: Pittsburgh, Pennsylvania, United States. http://guir.berkeley.edu/courses/cs198/papers/p370-want.pdf.

• [Clark 2009] Sarah Clark. Handset suppliers gear up for NFC. Near Field Communications World. 2.6.2009. http://www.nearfieldcommunicationsworld.com/2009/06/02/31252/handset-suppliers-gear-up-for-nfc/

• [Touch & Travel] http://www.touchandtravel.de/site/touchandtravel/de/idee/funktionsprinzip/funktionsprinzip.html

• [missphones 2009] http://www.missphones.co.uk/article/nfc-field-communication

• [VTT 2007] VTT brings touch-based services to mobile phones. http://www.vtt.fi/uutta/2007/20071009xml.jsp

• [i-mode FeliCa], http://www.nttdocomo.co.jp/english/service/imode/make/content/felica/

• [ECMA_NFC 2004] Near Field Communication White paper (Ecma/TC32-TG19/2004/1), http://www.ecma-international.org/activities/Communications/2004tg19-001.pdf

• [Dhiram 2007] Dhiram. O2 NFC put to trial in the UK http://www.newlaunches.com/archives/o2_nfc_put_to_trial_in_the_uk.php

• [Reilly et al . 2006] Reilly, D., Rodgers, M., Argue, R., Nunes, M., Inkpen, K. Marked-up Maps: Combining Paper Maps and Electronic Information Resources. In Personal and Ubiquitous Computing, 10 (4), 215-226, 2006.

• [Hardy and Rukzio 2008] Robert Hardy, Enrico Rukzio. Touch & Interact: Touch-based Interaction of Mobile Phones with Displays. 10th International Conference on Human-Computer Interaction with Mobile Devices and Services (Mobile HCI 2008). Amsterdam, Netherlands. 2-5 September 2008.

46

References

47

• [Häikiö et al. 2007] Häikiö, J., Wallin, A., Isomursu, M., Ailisto, H., Matinmikko, T., and Huomo, T. 2007. Touch-based user interface for elderly users. In Proceedings of the 9th international Conference on Human Computer interaction with Mobile Devices and Services (Singapore, September 09 - 12, 2007). MobileHCI '07, vol. 309. ACM, New York, NY, 289-296. DOI= http://doi.acm.org/10.1145/1377999.1378021

• [Nearfield 2007] 'Touch orders‟ with „RFID dongles', http://www.nearfield.org/2007/10/touch-orders-with-rfid-dongles, 2007

• [Broll et al. 2007] Gregor Broll, Sven Siorpaes, Enrico Rukzio, Massimo Paolucci, John Hamard, Matthias Wagner, Albrecht Schmidt. Supporting Mobile Service Usage through Physical Mobile Interaction. Fifth Annual IEEE International Conference on Pervasive Computing and Communications (Percom 2007). White Plains, NY, USA. 19-23 March 2007.

• [Seewoonauth et al. 2009] Khooviraj Seewoonauth, Enrico Rukzio, Robert Hardy, Paul Holleis. Touch & Connect and Touch & Select: Interacting with a Computer by Touching it with a Mobile Phone. Conditionally accepted for 11th International Conference on Human-Computer Interaction with Mobile Devices and Services (Mobile HCI 2009). Bonn, Germany. 15-18 September 2009.

• [Rekimoto 1997] Jun Rekimoto, "Pick-and-Drop: A Direct Manipulation Technique for Multiple Computer Environments", Proceedings of UIST'97, pp. 31-39, 1997

• [Wilson & Sarin 2007] Andrew Wilson and Raman Sarin. BlueTable: connecting wireless mobile devices on interactive surfaces using vision-based handshaking. In Proceedings of Graphics interface 2007 (Montreal, Canada, May 28 - 30, 2007).

• [Fitzmaurice 1993] George W. Fitzmaurice: Situated information spaces and spatially aware palmtop computers. In: Commun. ACM, 36 (1993), S. 39-49. http://www.dgp.toronto.edu/~gf/papers/Chameleon%20-%20Situated%20Info%20Spaces.pdf

• [Carpenter 2008] Brad Carpenter. Developing For Microsoft Surface. PDC 2008. http://mschnlnine.vo.llnwd.net/d1/pdc08/PPTX/PC17.pptx

• [Tamaki et al. 2009] Emi Tamaki, Takashi Miyaki, Jun Rekimoto: Brainy hand: an ear-worn hand gesture interaction device. CHI Extended Abstracts 2009: 4255-4260

• [Greaves et al. 2009] Andrew Greaves, Panu M Akerman, Enrico Rukzio, Keith Cheverst, Jonna Hakkila. Exploring User Reaction to Personal Projection when used in Shared Public Places: A Formative Study. MobileHCI 2008 workshop on Context-Aware Mobile Media and Mobile Social Networks. Bonn, Germany. 15 September 2009.

48

References

[Föckler et al. 2005] Paul Föckler; Thomas Zeidler; Benjamin Brombach; Erich Bruns ; Oliver Bimber: PhoneGuide: Museum Guidance Supported by On-Device Object Recognition on Mobile Phones. In: Proceedings of International Conference on Mobile and Ubiquitous Computing (MUM'05) (2005). http://www.uni-weimar.de/%7Ebimber/Pub/PhoneGuide.pdf

[Mobilepoint] Deutsche Post - MobilePoint, http://www.mobilepoint.de/

[BahnHandyTicket 2006] Handy - Ticket: NEU: Das Handy-Display wird zur Fahrkarte! , http://www.bahn.de/-S:PtVORd:efw2UtNNdiz6v9NNNWxM/p/view/planen/reiseplanung/mobileservices/handy_ticket.shtml

[Miyaoku et al. 2004] Kento Miyaoku; Suguru Higashino ; Yoshinobu Tonomura: C-blink: a hue-difference-based light signal marker for large screen interaction via any mobile terminal, in Proceedings of the 17th annual ACM symposium on User interface software and technology. 2004, ACM Press: Santa Fe, NM, USA. http://portal.acm.org/citation.cfm?id=1029657&coll=GUIDE&dl=GUIDE&CFID=15151515&CFTOKEN=6184618&ret=1

[Fowler] 2005) Geoffrey A. Fowler. QR codes: In Japan, Billboards Take Code-Crazy Ads to New Heights. Wall Street Journal 10.10.2005, http://www.mindfully.org/Technology/2005/QR-Codes-Japan10oct05.htm

[Rohs and Gfeller 2004] Michael Rohs ; Beat Gfeller: Using Camera-Equipped Mobile Phones for Interacting with Real-World Objects. In: Advances in Pervasive Computing, Austrian Computer Society (OCG) (2004), S. 265-271. http://www.vs.inf.ethz.ch/publ/papers/rohs-gfeller-visualcodes-2004.pdf

[Rohs 2005] Michael Rohs: Linking Physical and Virtual Worlds with Visual Markers and Handheld Devices. Dissertation2005, Swiss Federal Institute of Technology

[Fritz et al. 2004] Gerald Fritz; Christin Seifert; Patrick Luley; Lucas Paletta ; Alexander Almer: Mobile Vision for Ambient Learning in Urban Environments. In: Proceedings of International Conference on Mobile Learning (MLEARN 2004) (2004). http://www.joanneum.at/cms_img/img2450.pdf

[Löchtefeld et al. 2009] Markus Löchtefeld, Johannes Schöning, Michael Rohs, Antonio Krüger. Mobile Little Big Planet: An Augmented Reality Game for Camera Projector Phones. MIRW 2009. Bonn, Germany. 15 September 2009.

[Christian 2002] Tunes for drinks, http://www.spiritoffootball.com/2002/archives/000022.html

References

• [Rukzio et al. 2006] Enrico Rukzio; Karin Leichtenstern; Vic Callaghan; Paul Holleis; Albrecht Schmidt ; Jeannette Chin: An Experimental Comparison of Physical Mobile Interaction Techniques: Touching, Pointing and Scanning, in Eighth International Conference on Ubiquitous Computing (Ubicomp 2006). 2006c, Springer: Orange County, California, California. http://www.medien.informatik.uni-muenchen.de/fileadmin/mimuc/rukzio/ubicomp2006_topoisc.pdf

• [Välkkynen and Tuomisto 2005] Pasi Välkkynen; Timo Tuomisto: Physical Browsing Research. Workshop Pervasive Mobile Interaction Devices (PERMID 2005): Mobile Devices as Pervasive User Interfaces and Interaction Devices Permid 2005 in conjunction with 3rd International Conference on Pervasive Computing (Pervasive 2005), Munich, Germany, 2005. http://www.medien.ifi.lmu.de/permid2005/pdf/PasiVaelkkynen_Permid2005.pdf

• [@WikipediaTricorder] Wikipedia: Tricorder, http://en.wikipedia.org/wiki/Tricorder.

• [BUGAbutler ] BUGAbutler: Der Einsatz des BUGAbutler auf der Bundesgartenschau 2005 in München., in Auswertung. 2005, Bundesgartenschau 2005 GmbH München. http://www.blisinfo.de/download.php?file=/MEDIA/doc/Auswertung_121005.pdf

• [@i-area] NTT DoCoMo i-area: Location Based Services, http://www.nttdocomo.com/corebiz/services/imode/iarea.html.

• [Abowd et al. 1997] Gregory D. Abowd; Christopher G. Atkeson; Jason Hong; Sue Long; Rob Kooper ; Mike Pinkerton: Cyberguide: a mobile context-aware tour guide. In: Wirel. Netw., 3 (1997), S. 421-433. portal.acm.org/ft_gateway.cfm?id=272199&type=pdf

• [Küpper 2005] Axel Küpper: Location-based Services : Fundamentals and Operation: John Wiley & Sons Ltd. 2005. - ISBN 0470092319.

• [J-Ware] Philips demonstrates ‘with a swipe’ content downloading from a ‘smart poster’ based on a unique combination of NFC and Java™ technologies for cellular handsets, http://www.software.nxp.com/about/technologies/j-ware/article-14745.html

• [@NFCCaen 2005] City of Caen, France, to demonstrate simplicity of Near Field Communication (NFC) technology, News From Philips Semiconductors, http://www.semiconductors.philips.com/news/content/file_1193.html

• [Cao et al 2007] Xiang Cao, Clifton Forlines, Ravin Balakrishnan. Multi-user Interaction using Handheld Projectors. 20th Annual ACM Symposium on User Interface Software and Technology (UIST 2007). ACM Press. Newport, Rhode Island, USA. 2007.

• [Beardsley et al. 2005] Paul Beardsley, Ramesh Raskar, Clifton Forlines, Jeroen van Baar. Interactive Projection. TR2004-107. January 2005.

49

References

• [ToruCa] ToruCa, http://www.nttdocomo.co.jp/english/service/imode/osaifu/toruca.html

• [PhilipsNFC] Near Field Communication, http://www.semiconductors.philips.com/products/identification/nfc/

• [Osaifu-Keitai] Osaifu-Keitai, http://www.nttdocomo.co.jp/english/service/imode/osaifu/index.html

• [@cmode] Cmode, The unofficial independent imode FAQ, http://www.eurotechnology.com/imode/faq-cmode.html

• [NokiaSensor] Nokia Sensor, www.nokia.com/sensor

• [Persson and Jung 2005] Per Persson ; Younghee Jung: Nokia sensor: from research to product, in Proceedings of the 2005 conference on Designing for User eXperience 2005, AIGA: American Institute of Graphic Arts: San Francisco, California. p. 53. http://portal.acm.org/citation.cfm?id=1138235.1138297

• [Cheverst et al. 2000] Keith Cheverst; Nigel Davies; Keith Mitchell; Adrian Friday ; Christos Efstratiou: Developing a context-aware electronic tourist guide: some issues and experiences, in Proceedings of the SIGCHI conference on Human factors in computing systems. 2000, ACM Press: The Hague, The Netherlands. http://www.guide.lancs.ac.uk/CHIpaper.pdf

• [QRCode] QRCode, http://www.qrcode.com/

• [@Semapedia] http://www.semapedia.org/

• [Hang et al. 2008] Alina Hang, Enrico Rukzio, Andrew Greaves. Projector Phone: A Study of Using Mobile Phones with Integrated Projector for Interaction with Maps. 10th International Conference on Human-Computer Interaction with Mobile Devices and Services (Mobile HCI 2008). Amsterdam, Netherlands. 2-5 September 2008.

• [Want 2006] Roy Want. An introduction to RFID technology. IEEE Pervasive Computing 2006.

• [Sherwood 2009] James Sherwood. Contactless payment tech in all phones next year, says Ericsson boss. 26.06.2009 http://www.reghardware.co.uk/2009/06/26/ericsson_nfc_claim/

• [ECMA 2004] ECMA International. Near Field Communication White paper. Ecma/TC32-TG19/2004/1. http://www.ecma-international.org/activities/Communications/2004tg19-001.pdf

• [Raskar et al. 2004] Raskar, R., Beardsley, P., van Baar, J., Wang, Y., Dietz, P., Lee, J., Leigh, D., and Willwacher, T. 2004. RFIG lamps: interacting with a self-describing world via photosensing wireless tags and projectors. ACM Trans. Graph. 23, 3 (Aug. 2004), 406-415. DOI= http://doi.acm.org/10.1145/1015706.1015738

• [Greaves et al. 2008) Andy Greaves and Enrico Rukzio. Evaluation of Picture Browsing using a Projector Phone. Mobile HCI 2008.

• [Kindberg et al. 2002] Tim Kindberg; John Barton; Jeff Morgan; Gene Becker; Debbie Caswell; Philippe Debaty; Gita Gopal; Marcos Frid; Venky Krishnan; Howard Morris; John Schettino; Bill Serra ; Mirjana Spasojevic: People, places, things: web presence for the real world. In: Mobile Networks and Applications, 7 (2002), S. 365--376. http://www.hpl.hp.com/techreports/2001/HPL-2001-279.pdf

50