Mobile Augmented Reality: An Emerging Technology

Mary Patricia SmithFlorida Atlantic University

Overview

• Research Project• Purpose:

– Understand augmented reality– Use in mobile environment– How it can be used in an instructional setting

AUGMENTED REALITY BACKGROUND

Definition

Augmented reality is the overlay of real-time, real-world environments with digital information designed to enhance a person’s perception of that view.

History

• Concept appeared in 1960’s & 70’s: computerized headsets, just-in-time training.

• Term coined by Tom Caudell in 1990 to refer to digital display headset used as a guide in aircraft assembly.

• Well-established technology tied to specific location or bulky equipment.

• Extensive use in Science fiction (e.g., Star Trek)

Applications• TV Broadcasts

– Examples: “first down” line on football fields, virtual line indicating world record time in swim meets.

• Medicine– Used in radiology & surgery to provide additional information such as

test results, scans, surgical training• Military

– immersive training environments for combat preparation; maintenance and repair

• Tours of colleges and historical sites• Marketing• Gaming• Training

– Example: complex electronics assembly and repair– Turning sketches into 3D objects– http://www.bmw.com/com/en/owners/service/augmented_reality_worksh

op_1.html

Mixed Reality Continuum• Augmented Reality vs. Virtual Reality

• Designed to apply to immersive technologies,

usually involving helmets or special glasses

Virtuality Continuum (Milgram & Kashino’s (1994) Mixed Reality Spectrum , as cited in Klopfer, 2008)

Real Environment

Augmented Reality

Augmented Virtuality

VirtualEnvironment

Mixed Reality

Augmented Reality Continuum

Designed to describe non-immersive environments

Augmented Reality

Lightly augmented Heavily augmented

Location-based Immersivemobile games technologies

(Klopfer (2008))

Real-world input

Virtual input

MOBILE AUGMENTED REALITY

Mobile Media Devices in Education• A potentially disruptive force

– Portability– Price Point– Usability– Functionality– Ubiquity

• Enable– Connectivity– Social interactivity– Context sensitivity– Individuality

Mobile Augmented Reality

• Enabled by mobile media devices combining– Camera– Screen– GPS location– Compass and accelerometer– Image recognition capability– Internet access – Affordability

• Availability of AR Browsers, apps

Mobile Augmented Reality Approaches

• Marker-based Augmented Reality– Use specific visual cues (logo, etc.) to

call up info from a database or direct users to a website

– QR codes, ‘Magic Symbols’, Semacodesread by mobile app

• Markerless Augmented Reality– Uses GPS capability of smartphones combined

with AR Browsers– Geotagging,geolocation, ‘floaticons’

Educational Applications

• AR Books• Dynamic 3-D objects• Field Trips to Museums & Historical Sites• Just-in-time learning• Skills training• Discovery-based learning• Educational Games, or structured

activities with game-like qualities

Educational Philosophy• Aligned with

– Constructivist philosophy of education– Socio-cultural learning theory– Situated model of cognition– Authentic learning experiences– Discovery learning

• Build 21st century skills: – Collaboration– Analysis– Synthesis

Characteristics of Mobile Augmented Reality Games

• Story-driven learning experiences rooted in particular places

• Role-playing • Designed around authentic resources • Require social interaction with other game

participants• Exploit real physical environment in which

game is located• Authentic and engaging

Examples

Frequency 1550

Explore!Alien Contact!

Environmental Detectives

Charles River City

Limitations

• Logistical support and management• Hardware/software issues• Student cognitive overload• Portability

FUTURE DIRECTIONS

Augmented Reality Software• Enable creation of AR applications• Browsers, Tools, API’s• Examples:

– AR Toolkit: open-source, marker-based tool– Layar: markerless augmented reality browser– Wikitude: browser, API http://www.wikitude.org/en/– Junaio: image and object recognition, mixes marker-based &

markerless methods– Google Goggles: http://www.google.com/mobile/goggles/#text– Tagwhat: www.tagwhat.com– Hoppala Augmentation: Content Platform for Mobile Augmented Reality,

supports Layar, Junaio & Wikitude mobile AR Browsers. http://www.hoppala-agency.com/

Technology convergence

• Context aware computing• Gesture-based computing• Holographic images

New Educational Possibilities• Toolkits for developing Mobile Augmented Reality

Games for education – MIT’s Outdoor Augmented Reality Toolkit: drag and drop

authoring tool. • Museums, cities, historic sites developing MAR

content– Museum of London: “StreetMuseum”– iTacitus (Intelligent Tourism and Cultural Information through

Ubiquitous Services) Project– CultureClic

• Student-developed MAR content– “Augmented Reality Scratch” tool from Georgia Tech’s AR Lab– Wikitude.me

• Incorporation into project–based learning opportunities

ReferencesArdito, C., Buono, B., Cosabile, M.F., Lanzilotti, R., & Piccinno, A. (2009). Enabling interactive

exploration of cultural heritage: an experience of designing systems for mobile devices. Knowledge, Technology & Policy, 22, 79-86. DOI: 10.1007/s12130-009-9079-7.

Common Craft. Augmented Reality Explained by Common Craft. Retrieved October 20, 2010 from Common Craft website: http://www.commoncraft.com/augmented-reality-video

Chen, B. X. (2009, August). If you’re not seeing data, you’re not seeing. Wired Magazine. Retrieved from http://www.wired.com/gadgetlab/2009/08/augmented-reality/#ixzz11mW0LK42.

De Souza e Silva, A. & Delacruz, G.C. (2006). Hybrid reality games reframed: potential uses in educational contexts. Games and Culture, 1(3), 231-251. DOI: 10.1177/1555412006290443.

Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18. 7-22. DOI: 10.1007/s10956-008-9119-1.

Huizenga, J., Admiraal, W., Akkerman, S. & Dam, G. T. (2009), Mobile game-based learning in secondary education: engagement, motivation and learning in a mobile city game. Journal of Computer Assisted Learning, 25, 332–344. DOI: 10.1111/j.1365-2729.2009.00316.x.

Jenkins, H. (2008, July 7). Augmented Learning: an interview with Eric Klopfer. Retrieved October 19, 2010 from Confessions of an ACA-fan: The Official Weblog of Henry Jenkins: http://henryjenkins.org/2008/07/an_interview_with_eric_klopfer.html

Johnson, L., Levine, A., Smith, R., & Stone, S. (2010). The 2010 Horizon Report. Austin, Texas: The New Media Consortium.

Johnson, L., Smith, R., Willis, H., Levine, A., and Haywood, K., (2011). The 2011 Horizon Report. Austin, Texas: The New Media Consortium.

References, cont’d..

Ketelhut, D.J.. (2010). Eric Klopfer: Augmented learning: research and design of mobile educational games. Journal of Science Education and Technology, 19, 212-214.

Klopfer, E. (2008). Augmented learning: research and design of mobile educational games. Cambridge, MA: The MIT Press.

Klopfer, E., Yoon, S. & Perry, J. (2005). Using palm technology in participatory simulations of complex systems: a new take on ubiquitous and accessible mobile computing. Journal of Science Education and Technology, 14(3), 285-297. DOI: 10.1007/s10956-005-7194-0.

National Center on Education and the Economy. (2007). Tough Choices or Tough Times: The report of the new commission on the skills of the American workforce. San Francisco: Jossey-Bass. Kindle Edition New Media Consortium. (2010). 2010 Horizon Report. Retrieved from http://wp.nmc.org/horizon2010/.

Rose, S, Potter, D & Newcomb, M. (2010). Augmented reality: A review of available augmented reality packages and evaluation of their potential use in an educational context. Retrieved from http://blogs.exeter.ac.uk/augmentedreality/files/2010/11/Augmented-Reality-final.pdf.

Specht, M., Ternier, S. & Greller, W. (2011). Dimensions of mobile augmented reality for learning: A first inventory. Journal of the research Center for Educational Technology. 7(1), 117-127. Retrieved from http://rcetj.org/index.php/rcetj/article/view/151/241.

Squire, K. & Klopfer, E. (2007). Augmented Reality Simulations on Handheld Computers. Journal of the Learning Sciences, 16(3), 371-413. DOI:10.1080/10508400701413435.

Frequency 1550• Hybrid reality game about Medieval Amsterdam designed to be

played in one day• Groups of 4-5 children are divided into 2 teams – a city team, who

walk the streets of Amsterdam, (each group is assigned a different area of the city) and a headquarters team who are at a computer back at the school.

• Teams communicate via cell phones and switch places after lunch. • The city team can view a map of Medieval Amsterdam on their

smart phones; the headquarters team can access that map and a map of present day Amsterdam.

• The city team has tasks to complete, and the headquarters team can follow them through GPS and can use other computer resources to guide them.(Huizenga, J., Admiraal, W., Akkerman, S. & Dam, G. T. (2009), Mobile game-based learning in secondary education: engagement, motivation and learning in a mobile city game. Journal of Computer Assisted Learning, 25, 332–344. DOI: 10.1111/j.1365-2729.2009.00316.x.)

Back

Explore!• Enables interactive explorations of archaeological sites in Italy by

groups of 3-5 middle school students• Each group given 2 cell phones;

– one phone contains the game application, which provides the information necessary to play the game,

– the other phone contains an application which provides further hints for identifying the places in the park and contains 3D reconstructions of how the places may have looked.

• Game built on a platform that can be adapted to a variety of different historic sites

Ardito, C., Buono, B., Cosabile, M.F., Lanzilotti, R., & Piccinno, A. (2009). Enabling interactive exploration of cultural heritage: an experience of designing systems for mobile devices. Knowledge, Technology & Policy, 22, 79-86.

Back

Alien Contact!• Designed to teach math, language arts and scientific literacy skills to

middle school students• Adapted to any outdoor environment, superimposes a map of a

virtual world on that space based on GPS coordinates:• Based on scenario that aliens have landed on earth; teams of

students consisting of a chemist, cryptologist, computer hacker, and FBI agent must figure out why (peace, plunder, invasion, etc.)

• Students can interview virtual characters, collect digital items, and solve science, math and language problems to answer the question

• Students see different pieces of evidence depending on role they play, requires team work to solve

• Game based on Massachusetts state standards, and fosters multiple higher order thinking skills.

Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18. 7-22

Back

Environmental Detectives• Students play the role of environmental scientists exploring the

source of a hypothetical toxic spill.• Real environments used, and students engage in complex,

collaborative problem solving typical of real environmental engineers• Each pair of students is given a mobile media device equipped with

GPS and can see their location on a map. • Students conduct virtual interviews with people in designated

locations in the physical space; some of the experts they interview can provide documents, and students can take virtual samples of the water and the soil.

• Developed by researchers in MIT’s Games-to-Teach project, in conjunction with faculty in the environmental science department

Klopfer, E. (2008). Augmented learning: research and design of mobile educational games. Cambridge, MA: The MIT Press.Squire, K. & Klopfer, E. (2007). Augmented Reality Simulations on Handheld Computers. Journal of the Learning Sciences, 16(3), 371-413.

Back

Charles River City• Example of the second generation of games being developed by

MIT’s Games-to-Teach project • Students participant on teams in an investigation of a large scale

outbreak of illness coinciding with a major event in the Boston metro area

• Similar to Environmental Detectives,• Enhancements made to foster collaboration among participants,

including distinct player roles, increased data beaming from player to player, and cascading events, in which some events trigger other events

Klopfer, E. (2008). Augmented learning: research and design of mobile educational games. Cambridge, MA: The MIT Press.

Back

Layar

Layar is an application that overlays your view of the real world with waypoints representing your favorite coffee place, the movie theatre you're trying to find, or in this case, where some of that $787 billion from the American Recovery and Reinvestment Act is going.

Back