learning technical drawing with augmented reality and ...€¦ · recent advances in educational...

10
Learning Technical Drawing with Augmented Reality and Holograms MAURO FIGUEIREDO University of Algarve ISE, CIMA, CIAC Estrada da Penha, 8005-139 Faro PORTUGAL mfi[email protected] LU ´ IS SOUSA University of Algarve ISE Estrada da Penha, 8005-139 Faro PORTUGAL [email protected] PEDRO J. S. CARDOSO University of Algarve ISE, LARSyS Estrada da Penha, 8005-139 Faro PORTUGAL [email protected] J.M.F. RODRIGUES University of Algarve ISE, LARSyS Estrada da Penha, 8005-139 Faro PORTUGAL [email protected] C ´ ESAR GONC ¸ ALVES University of Algarve ISE Estrada da Penha, 8005-139 Faro PORTUGAL [email protected] RICARDO ALVES University of Algarve ISE Estrada da Penha, 8005-139 Faro PORTUGAL [email protected] Abstract: Technical drawing is very important for mechanical engineers. The ability to understand and work with technical drawings is a necessary skill for most of them. In general, first year students of mechanical engineering have difficulties in drawing orthographic views and perspectives, since they find it difficult to understand 3D shapes from 2D views. In this paper, we present several tools that we explored to help students visualize 3D models. We studied the most popular AR systems and show examples of using an AR system for the visualization of 3D models. We also present the creation of a low cost prototype, the EducHolo, that enables the visualization and interaction with holograms. With this prototype students can visualize and interact with the hologram of mechanical parts. Using augmented reality and interactive holograms we aim at providing a better perception of the model 3D shape and improving the ability of making the 2D orthographic views and perspectives that they study in the first year of mechanical engineer. Key–Words: Visualization; Augmented Reality; Holograms; 3D models; e-learning. 1 Introduction When students start learning technical drawing, in the first year of their mechanical engineering degree, they commonly have many difficulties in understand- ing and drawing the shape of three-dimensional (3D) objects from two-dimensional (2D) representations. The same is also true when they have a 3D model per- spective representation of a mechanical part and they need to draw the two-dimensional front, side and top views. However, most mechanical engineers need to make and understand technical drawings, which is a fundamental tool that allows them to create designs and work with manufacturers. In this paper, we present several examples using augmented reality and interactive holograms, that help students in the drawing of orthographic views and per- spectives of technical drawings. There are many augmented reality applications that can be used in a mobile device [1]. Similarly, there are many ways to produce an hologram [2]. Our first goal is to let students bring their own mobile devices and let them explore the 3D models using augmented reality. Tablets and smartphones are becoming less expensive and many students already bring them to classes. In this way, this paper presents Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11

Upload: others

Post on 15-Jul-2020

7 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Learning Technical Drawing with Augmented Reality and ...€¦ · Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11. an augmented reality application

Learning Technical Drawing with Augmented Reality andHolograms

MAURO FIGUEIREDOUniversity of Algarve

ISE, CIMA, CIACEstrada da Penha, 8005-139 Faro

[email protected]

LUIS SOUSAUniversity of Algarve

ISEEstrada da Penha, 8005-139 Faro

[email protected]

PEDRO J. S. CARDOSOUniversity of Algarve

ISE, LARSySEstrada da Penha, 8005-139 Faro

[email protected]

J.M.F. RODRIGUESUniversity of Algarve

ISE, LARSySEstrada da Penha, 8005-139 Faro

[email protected]

CESAR GONCALVESUniversity of Algarve

ISEEstrada da Penha, 8005-139 Faro

[email protected]

RICARDO ALVESUniversity of Algarve

ISEEstrada da Penha, 8005-139 Faro

[email protected]

Abstract: Technical drawing is very important for mechanical engineers. The ability to understand and work withtechnical drawings is a necessary skill for most of them. In general, first year students of mechanical engineeringhave difficulties in drawing orthographic views and perspectives, since they find it difficult to understand 3D shapesfrom 2D views. In this paper, we present several tools that we explored to help students visualize 3D models. Westudied the most popular AR systems and show examples of using an AR system for the visualization of 3D models.We also present the creation of a low cost prototype, the EducHolo, that enables the visualization and interactionwith holograms. With this prototype students can visualize and interact with the hologram of mechanical parts.Using augmented reality and interactive holograms we aim at providing a better perception of the model 3D shapeand improving the ability of making the 2D orthographic views and perspectives that they study in the first year ofmechanical engineer.

Key–Words: Visualization; Augmented Reality; Holograms; 3D models; e-learning.

1 Introduction

When students start learning technical drawing, inthe first year of their mechanical engineering degree,they commonly have many difficulties in understand-ing and drawing the shape of three-dimensional (3D)objects from two-dimensional (2D) representations.The same is also true when they have a 3D model per-spective representation of a mechanical part and theyneed to draw the two-dimensional front, side and topviews. However, most mechanical engineers need tomake and understand technical drawings, which is afundamental tool that allows them to create designs

and work with manufacturers.In this paper, we present several examples using

augmented reality and interactive holograms, that helpstudents in the drawing of orthographic views and per-spectives of technical drawings.

There are many augmented reality applicationsthat can be used in a mobile device [1]. Similarly,there are many ways to produce an hologram [2].

Our first goal is to let students bring their ownmobile devices and let them explore the 3D modelsusing augmented reality. Tablets and smartphones arebecoming less expensive and many students alreadybring them to classes. In this way, this paper presents

Recent Advances in Educational Technologies and Methodologies

ISBN: 978-960-474-395-7 11

Page 2: Learning Technical Drawing with Augmented Reality and ...€¦ · Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11. an augmented reality application

an augmented reality application that enables theteacher to prepare 3D models that can be show on topof a 2D drawing in an efficient way and that do notrequire programming knowledge. Furthermore, theaugmented reality application that we selected for thiseducational project is free, without any type of watermarks.

Holograms are widely used in the amusementarea (e.g., theaters, magic illusion, thematic parks).In this paper, we also present an interactive hologramprototype system that delivers an excellent educationtool for learning technical drawing. It provides a depthimpression of the analyzed three-dimensional objectswhich is further useful by enabling the user interac-tion.

The paper is organized as follows. Section 2surveys the most common augmented reality eco-systems, the different holograms systems and the ex-plored user interaction aspects. It is presented inSection 3 a marker based example for improving thelearning of orthographic views by showing the 3Dmodel in an augmented reality application. Section 4presents the EducHolo prototype that enables studentsto visualize holograms with any monitor from a stand-ard personal computer or laptop. Finally, conclusionsare presented in Section 5.

2 State of the Art2.1 Augmented RealityAugmented reality applications combine 3-D virtualobjects with a 3-D real environment in real time. Vir-tual and real objects appear together in a real timesystem in a way that the user sees the real world andthe virtual objects superimposed with the real objects.The user’s perception of the real world is enhancedand the user interacts in a more natural way. Virtualobjects can be used to display additional informationabout the real world that are not directly perceived.

Paul Milgram and Fumio Kishino [3] introducedthe concept of a Virtuality Continuum classifying thedifferent ways that virtual and real objects can be real-ized. In this taxonomy scheme augmented reality iscloser to the real world.

Ronald Azuma [1] defines augmented reality sys-tems as those that have three characteristics: 1) com-bines real and virtual; 2) are interactive in real time;and 3) are registered in 3-D.

In general, augmented reality applications fall intwo categories: geo-base and computer vision based.

Geo-based applications use the mobile’s GPS, ac-celerometer, gyroscope, and other technology to de-termine the location, heading, and direction of the mo-bile device. The user can see 3D objects that are su-

perimposed to the world in the direction he is lookingat. However, this technology has some problems. Themajor problem is imprecise location which makes dif-ficult for example the creation of photo overlays.

Computer vision based applications use image re-cognition capabilities to recognize images and overlayinformation on top of this image. These can be basedon markers, such as QR (Quick Response), Microsofttags or LLA (latitude/longitude/altitude), or markerless that recognize an image that triggers the overlaydata.

There are currently many augmented reality ap-plications and development systems for Android andiOS (iPhone Operating System) smartphones and tab-lets. The most popular ones are: Wikitude [4], Layar[5], Metaio [6], Aurasma [7], and Augment [8].

Wikitude delivers the Wikitude World Browserfor free, which is an augmented reality web browserapplication, and the Wikitude SDK (software devel-opment kit) for developers which is free for educa-tional projects. However, the educational version ofthe Wikitude SDK always displays a splash screen andthe Wikitude logo.

The Wikitude browser presents users with dataabout their points of interest, which can be the sur-roundings, nearby landmarks or target images, andoverlays information on the real-time camera view ofa mobile device.

Augmented reality learning activities can be real-ized with the Wikitude SDK. The Wikitude SDK canbe used to display a simple radar that shows radar-points related to the location based objects. It isalso possible to recognize target images and super-impose 2D or 3D information on top of them. Thedeveloper can also combines image recognition andgeo-base augmented reality. However, the buildingof these capabilities using the Wikitude SDK requiresprogramming knowledge.

Layar has the Layar App, an augmented realityweb browser, and the Layar Creator, which is a toolfor creating interactive printing documents. With theLayar Creator it is very easy to make an interactivedocument for a teaching activity. There is no need todo any programming and, in this way, it does not re-quire any developers with programming skills. Theteacher can easily upload the trigger page to whichhe wants to associate augmented information. Markerless image recognition techniques are used and withthe Layar Creator interface the teacher can easily as-sociate a video, for example. Later, with the LayarApp, the student can view, on the camera of his mo-bile device, the overlaid information associated to thepage. These applications are both free. However,every trigger image published within the Layar’s pub-lishing environment is paid. For this reason, it is not

Recent Advances in Educational Technologies and Methodologies

ISBN: 978-960-474-395-7 12

Page 3: Learning Technical Drawing with Augmented Reality and ...€¦ · Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11. an augmented reality application

affordable for developing interactive printing docu-ments for teaching. Geo-location based augmentedreality information is free of charge.

Metaio delivers the junaio, the metaio Creator anda development SDK. Junaio is the metaio’s free aug-mented reality browser and is free. The metaio Cre-ator is an augmented reality tool to create and publishaugmented reality scenarios and experiences withinminutes. With the metaio Creator the teacher can con-nect 3-D content, videos, audio, and web pages to anyform of printed medium or 3D map (object-based orenvironment-based). However this tool is paid. If auser wants to develop augmented reality applicationsfor iOS or Android, the developer can use the metaioSDK. However, this development SDK is also paid.

Aurasma delivers the Aurasma App and the Aur-asma Studio. The Aurasma App is available for An-droid and iOS and uses advanced image recognitiontechniques to augment the real-world with interactivecontent such as videos, 3D objects or animations as-sociated to trigger images or geo-based information.The Aurasma Studio is an online platform that lets theteacher create and publish their own augmented real-ity information in an intuitive and user friendly envir-onment. It is not required any programming know-ledge and every teacher can easily upload trigger im-ages that can be associated to videos, images, 3D ob-jects or other information. The Aurasma eco-systemdelivers these application for free.

Augment is a free application for Android andiOS that uses augmented reality to visualize 3D mod-els triggered by QR codes and recently it also enablesthe use of a trigger image. After registering at theaugment website, the teacher can easily upload a 3Dmodel that is triggered by a QR code or an image.

Our concern is to find augmented reality eco-systems that do not require programming, that arefree and easy to use for learning activities. For thisreason, we chose the Augment systems which is free,do not require programming and teachers can preparerequired learning activities in an easy way.

2.2 HologramsIn Time Machine movie (2002), directed by SimonWells, the library scene features a hologram that hosts,communicates and interacts naturally with a time trav-eler. Amazingly a product of this kind to operate inits fullness does not yet exist, although there is thetechnology needed to develop it. On the other hand,the global communication market is increasingly de-manding. Alone the existing creativity in education,media, design, advertising and marketing companiesis no longer enough, especially for those companiesthat focus on the global market. A solution might be

the use of holography to draw the attention of moreusers.

Holography is a technique for recording interfer-ence patterns of light which can generate or displayimages in three dimensions [9, 2]. Unlike photo-graphy, which only allows the record of the differentintensities of light from the scene being photographed,holograms also record the phase of the light radiationfrom the object. The phase contains information onthe relative position of each point of the illuminatedobject, enabling to reconstruct a three-dimensionalimage from that information.

Some times, an hologram is also defined as aphotographic image that is 3D and appears to havedepth. In this case, holograms work by creating animage composed of two superimposed 2D picturesof the same object seen by different reference points.The use of slightly offset reference points is designedto mimic the image interpreted by the human brain,which likewise receives a distinct, slightly offset im-age from each eye that the brain combines into a 3Dimage [10].

One of the most common technique to generateso called “holograms” is the Pepper’s Ghost [11], dueto John Henry Pepper that popularized the effect. ThePepper’s Ghost is an illusion technique used in theaterand in some magic tricks. In its basics, a large piece ofglass at a 45 degrees angle to the audience and speciallighting techniques are used, showing the audience acombination o light passing through from behind theglass and light reflecting off the glass at a 90 degreeangle from the line of sight. The so called “hologram”actually is an object or image hidden from the audi-ence and reflected off of the screen. A better effectis achieved by using dark backgrounds. An exampleapplied to the theater and holography illustrating theentire length technique can be found in [12]. Anotherexample is the D’Strict 3D Sensing Hologram Install-ation [13] product which incorporates a hologram anda monitor in a small box, that you can interact withthrough gestures.

2.3 User InteractionThe interaction between humans and machines can bemade in many different ways, being crucial to obtainthe expected level of control. Desktop or mobile ap-plications and Internet browsing make use of Graph-ical User Interfaces (GUI), being currently prevalentamong the available solutions. Other solutions likeVoice User Interfaces (VUI) use speech recognitionto control devices. More recently, multimodal inter-faces [14] allow humans to interact with machines ina way that cannot be achieved with other interfaceparadigms.

Recent Advances in Educational Technologies and Methodologies

ISBN: 978-960-474-395-7 13

Page 4: Learning Technical Drawing with Augmented Reality and ...€¦ · Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11. an augmented reality application

One of the new paradigms for human computerinteraction (HCI), are the three-dimensional (3D)sensors, such as Kinect [15], Leap Motion [16], Stru-ture Sensor [17], Asus Xtion [18]. Those can be usedto interpret specific human gestures, enabling a com-pletely hands-free control of electronic devices, themanipulation of objects in a virtual world or the in-teraction with augmented reality applications. Thesetracking and gesture recognition sensors have a hugeimportance mainly in the videogames industries, butthese sensors, with appropriate software, have the cap-ability of detecting the user skeleton and/or the userhand, while replicating with accuracy the user move-ments in a 3D mesh.

There is in the literature an enormous amount ofapplications, where gesture recognition and trackingis referenced, e.g., interactive art installations [19],applications to help disable or old people [20], airpainting application [21], robotic arm manipula-tion [22] or applications in sign language [23].

We chose the Leap Motion sensor [16, 21, 22, 23]for the prototype presented in this paper due to its size,price and specific range of applications. It is a recent,but widely known sensor for hand, fingers and ges-ture recognition, with a very high accuracy and speed,which uses two monochromatic IR cameras and threeinfrared LEDs, for more details see [24]. A smallerobservation area and a higher resolution differentiateit from the Kinect and Asus Xtion sensors, which aremore suitable for body tracking.

3 The use of Augmented Reality indrawing orthographic views

First year students of Mechanical Engineering learnthe basic concepts and techniques of technical draw-ing as a language definition and transmission char-acteristics of systems and industrial products, withgradual introduction of the use of computer aideddesign (CAD) systems. However, when students startlearning technical drawing, in the first year of theirstudies, they commonly have many difficulties in un-derstanding and drawing the shape of objects fromtwo-dimensional representations. The same is alsotrue when they have a representation of the 3D modelof a mechanical part and they need to draw the front,left and top views.

Wu and Chiang [25] show that applying anima-tions provided more enthusiasm for the learning activ-ity, better performance in understanding the appear-ances and features of objects and improve the spatialvisualization capabilities.

This section presents examples of using interact-ive augmented reality tools to show 3D models that

Figure 1: The isometric drawing of model 1.

are used to help students improve learning of ortho-graphic views.

Our first approach is to let students use their tab-lets or smartphones to visualize 3D models using aug-mented reality. Students use an iOS or Android aug-mented reality free app to show 3D models that areused to help them improve their understanding of theshape of the 3D model enabling them to draw the or-thographic views.

For this purpose, the first thing the teacher needsis a 3D modeling tool. At our faculty, most teachersuse AutoCAD which is free for education. With Auto-CAD, teachers can create 3D models that are storedas dwg files. If the teacher wants, it is also possible toadd textures to the model and make it look like a realobject, made of wood for example.

In this way, it was easy to create the 3D modelthat is represented in an isometric view (see Figures 1and 2).

To help students visualize and understand this3D model, we used augment reality to render the 3Dmodel in a mobile device triggered by a QR code, animage trigger or a url link. Figures 3 and 4 presentsthe visualization of the augmented 3D models cor-responding to the previous representations of the 3Dmodels that the student can use to draw the ortho-graphic views.

Students with the help of the 3D augmented mod-els draw the corresponding front, left and top ortho-graphic views (Figures 5 and 6, respectively).

In present time, with the Augment application, weare replacing educational materials with virtual ones.Students can feel as if they have the actual material bywatching the 3D virtual object from various orienta-tions with a tablet or a smartphone. In this way, weprovide various educational materials for each studentrapidly, easily and with no extra cost.

Recent Advances in Educational Technologies and Methodologies

ISBN: 978-960-474-395-7 14

Page 5: Learning Technical Drawing with Augmented Reality and ...€¦ · Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11. an augmented reality application

Figure 2: The isometric drawing of model 2.

Figure 3: Visualization of the 3D model 1 with theAugment application.

Figure 4: Visualization of the 3D model 2 with theAugment application.

Figure 5: The front, left and top views of the 3Dmodel 1.

Figure 6: The front and top views of the 3D model 2.

Recent Advances in Educational Technologies and Methodologies

ISBN: 978-960-474-395-7 15

Page 6: Learning Technical Drawing with Augmented Reality and ...€¦ · Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11. an augmented reality application

4 Interacting with Mechanical Holo-grams

A different, but complementary tool to help studentsvisualize the 3D models for better understanding oftheir shapes, is the use of interactive holograms. Thissection starts by presenting the implementation of theinteractive component that allows the interaction withthe holograms. Then, we present the example of usinginteractive holograms to help students draw the ortho-graphic views.

4.1 3D Gesture Recognition with Leap Mo-tion

Leap Motion [16] has an application Programming In-terface (PI) capable of detecting multiple hand ges-tures, such as straight line movement by hand withfingers extended, a circle movement by a finger, a for-ward tapping movement by a finger or a downwardtapping movement by a finger. These gestures (swipegesture, circle gesture, screen tap gesture and key tapgesture) can be seen in [16] or in [26].

For each of the gestures there are a few optionalconfigurations properties, to improve gesture detec-tion. For example in a circle gesture, where the usercan do a circle with a finger, there are two propertiesselectable: minimum radius and minimum arc (by de-fault, minimum radius is set to 5mm and minimumarc set to 1.5π radians). For a swipe gesture, thereare also two properties selectable, minimum lengthset by default to 150mm and minimum velocity setto 1000mm/s. These are examples of the “out of thebox” gestures, possible to retrieve with Leap MotionPI.

It is also possible to join some of these gestureswith the positions and rotation of each finger and bothhands at the same time to recognize other gestures,such as, open or close hand, zoom in or out, done withone or both hands. For the development of the inter-face it was pretended the minimum of movements andthe most intuitive ones - the swipe, was considered thebest approach. Thus, swipe gesture was the gesturemainly used in the interface developed in this paper,e.g., for the interaction with the 3D holograms of themechanical parts.

After regulating the minimum length swipe andvelocity to the default values, it is possible to detectswipe gestures at any direction. Leap Motion PI has adirection vector for the swipe gesture, after a gestureis completely recognized, it is associate with a 3D dir-ection vector. This vector have values ranging fromnegative 1.0 to positive 1.0.

As shown on Fig. 7, the Leap Motion “sees” the3D space with standard Cartesian coordinate system,

Figure 7: Leap Motion coordinate system [26].

also known as right-handed orientation. The originof the coordinate system is centered at the top of thedevice, being the front of the device, the side with thegreen light (see Fig. 7). The x-axis is placed horizont-ally along the device, with positive values increasingfrom left to right. The z-axis is placed also on hori-zontal plane, perpendicular with x-axis with values in-creasing towards the user (the front side of the device).The y-axis are placed is the vertical, with positive val-ues increasing upwards (see Fig. 7).

As a swipe gesture can be any swipe that meetsthe minimum length and velocity configurations prop-erties, and they can obviously have any direction. Asthis gesture is the only used, it is needed to detectvarious types of swipe. The interface was designedto react to two of the six different independent typesof swipe gestures as shown by Fig. 8, three of eachswipes are the opposite of the other three:

(i) Up and Down: is realized by a top to bottomswipe or a bottom to top swipe respectively (y-axis).

(ii) Front and Back: is realized by a front to backand back to front swipes (z-axis).

(iii) Left and Right: is done by a left to right and rightto left swipe respectively (x-axis).

In the first case, (i), of a top to bottom or bottomto top swipe, the movement depends mainly on the y-axis. If the direction vector has an upward direction(y ≈ +1) then “deselect” action has occurred, other-wise, a vector with a downward direction (y ≈ −1),it is considered “select” action. Since it is almost im-possible to do a swipe gesture with a vector directioncomponent of exactly

x = 0 ∧ y = ±1 ∧ z = 0, (1)

it is needed to select a range of values to detect anddifferentiate between swipes types. Any swipe direc-

Recent Advances in Educational Technologies and Methodologies

ISBN: 978-960-474-395-7 16

Page 7: Learning Technical Drawing with Augmented Reality and ...€¦ · Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11. an augmented reality application

tion that agrees with the condition

y ≤ −0.5 ∧ |x| ≤ 0.5 ∧ |z| ≤ 0.5, (2)

is considered as a downward swipe. Contrariwise, if aswipe direction agrees with the condition

y ≥ −0.5 ∧ |x| ≤ 0.5 ∧ |z| ≤ 0.5, (3)

then it is considered an upward swipe.In the second case, (ii), we need to analyze mainly

the z-axis. As shown on Fig. 7, a vector with z-direction value approximately equal to 1, is con-sidered to be a back to front swipe, otherwise, a frontto back swipe. Similar to (i), any swipe direction thatagrees with the condition

z ≤ −0.5 ∧ |x| ≤ 0.5 ∧ |y| ≤ 0.5, (4)

is considered a front to back swipe. Contrariwise, if aswipe direction agrees with the condition

z ≥ −0.5 ∧ |x| ≤ 0.5 ∧ |y| ≤ 0.5, (5)

then it is considered a back to front swipe.In the last case, (iii), similar to (i) and (ii), x-axis

is the principal axis. Any swipe direction that agreeswith the condition

x ≤ −0.5 ∧ |y| ≤ 0.5 ∧ |z| ≤ 0.5, (6)

is considered a right to left swipe. Contrariwise, if aswipe direction agrees with the condition

x ≥ −0.5 ∧ |y| ≤ 0.5 ∧ |z| ≤ 0.5, (7)

then it is considered a right to left swipe.These swipes are mutually independent. For

every type of swipes there are only one possiblechoice, see Fig. 8.

4.2 Interactive Holograms to Help StudentsLearning Technical Drawing

This section describes the use of interactive holo-grams for the visualization of 3D models to help stu-dents make and understand technical drawings. In thiscase, we use a computer display or a projector can alsobe used for the visualization of the hologram usingPepper’s Ghost technique. For that, the only mater-ial needed is a transparent polyester film, for a cheapand easy implementation solution, or a sheet of glass.Fig. 9 shows a diagram of the structure where thefilm/glass is placed at at roughly 45 degree angle re-lative to the computer monitor, which is placed hori-zontally at the top of a table. The monitor presentsthe models (preferably over a dark background for

Figure 8: Six types of swipes possibles with LeapMo-tion gesture recognition.

Figure 9: Diagram of the structure used to implementthe holographic display.

a finest experience) and the person is placed suchthat he can see the reflection of the object’s imageon the film/glass, which gives an 3D illusion of themodel. To improve the illusion, the background be-hind the film/glass should be a dark hall and the modeldisplayed in the tablet animated. Another importantfactor to improve the illusion, common to this kind ofsolutions, is to keep the room in a blackish environ-ment.

Figures 10 and 11 show two examples of holo-grams achieved with the describe device. In the firstcase the previously explored model 1 was used and,in the second case, the also already presented model2. It is important to note at this point that the pro-totype presented in Figures 10 and 11 cost less than10Euros, it is made simply of an aluminum structurewith some bolts and one acetate usually used in anyclassroom. Better results can be achieved when usinga Mylar film.

In this prototype the student can interact with the

Recent Advances in Educational Technologies and Methodologies

ISBN: 978-960-474-395-7 17

Page 8: Learning Technical Drawing with Augmented Reality and ...€¦ · Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11. an augmented reality application

Figure 10: Interactive hologram visualization of the3D model 1 with the EducHolo.

holograms with the integrated Leap Motion sensor.To use it, the student places its hands in front of thedevice, not too close, nor too far (in height and width).In this way, the student experience in enhanced. Thissensor gives the student the ability to interact with therepresentation of the 3D models.

Currently several solutions can be used to makesuch 3D representation. We decided to use the Unitycross-platform game engine [27]. Unity is a game cre-ation system developed by Unity Technologies that in-cludes a game engine and integrated development en-vironment (IDE). It is currently used to develop videogames for web sites, desktop platforms, consoles, andmobile devices. It is now the default software devel-opment kit (SDK) for the Nintendo Wii and has beenextended to target more than fifteen platforms [27].This platform is also the ideal one to create the graph-ics interfaces for our application.

In this way, we developed the hologram applic-ation interface in Unity 3D [27], with three differentzones: the video, the menu and the views. Figures 10and 11 show the interface in operation. In the top, thesystem shows the 3D model perspective that is rotatedand the student understands, in this way, its shape. Asexplained previously, there are six different swipes theuser can do to navigate in the interface with Leap Mo-

Figure 11: Hologram visualization of the 3D model 2.

tion. Only two were used, the (i) to select the front,top or left views and (iii) to select the 3D model and,where the respective perspective is played.

5 Conclusions

We explored the most popular augmented reality ap-plications available for tablet devices. In this paper,we looked augmented reality applications that couldbe used for teaching technical drawing in the first yearof mechanical engineering. Such application shouldbe user friendly, free and do not require programmingknowledge, such that, every teacher can use them ineveryday learning activities.

We chose the Augment application to show 3Dmodels on top of trigger image or a QR code. In thisway, students explore the visualization of the 3D mod-els with a mobile device and they better understand themodel shape to draw the orthographic or the isometricviews.

In this paper, we also show that with a low costinstallation it is also possible the holographic visual-ization and interaction with the 3D models. This will

Recent Advances in Educational Technologies and Methodologies

ISBN: 978-960-474-395-7 18

Page 9: Learning Technical Drawing with Augmented Reality and ...€¦ · Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11. an augmented reality application

improve the comprehension of the model before andduring the process of learning how to draw it. It is alsoshown the use of a Leap Motion sensor in this proto-type for the interaction with the hologram. The pro-totype makes use of easy achievable materials (poly-ester films or glass) that in conjunction with monitorcapacity to display good quality images and videosproduce an helpful device to help the students in thevisualization of the studied models.

In the future we pretend to test these two tech-nologies in a full classroom context, creating twogroups: one of control, and the other where these tech-nologies are going the be applied during a semester.At the beginning and at the end of the semester in-quiries are going to be applied to the two groups andfinal conclusion will be taken for a final validation ofthis “prove of concept”.

Acknowledgements

This work was partly supported by the PortugueseFoundation for Science and Technology, projectsLARSyS (PEst-OE/EEI/LA0009/2013), and CIAC(PEst-OE/EAT/UI4019/2013) and project PRHOLOQREN I&DT, n. 33845. We also thanks to projectleader SPIC - Creative Solutions [www.spic.pt].

References:

[1] Ronald T. Azuma. A survey of augmented real-ity. Presence: Teleoperators and Virtual Envir-onments, 6(4):355–385, August 1997.

[2] Emilia Mihaylova, editor. Holography - Ba-sic Principles and Contemporary Applications.InTech, 2013.

[3] P. Milgram and F. Kishino. A taxonomyof mixed reality visual displays. IEICETrans. Information Systems, E77-D(12):1321–1329, December 1994.

[4] Wikitude. Available: http://www.wikitude.com/.Accessed: September, 20th, 2014.

[5] Layar. Available: https://www.layar.com/. Ac-cessed: September, 20th, 2014.

[6] Metaio. Available: http://www.metaio.com/.Accessed: September, 20th, 2014.

[7] Aurasma. Available: http://www.aurasma.com/.Accessed: September, 20th, 2014.

[8] Augment. Available: http://augmentedev.com/.Accessed: September, 20th, 2014.

[9] Sanchez Antonio, Rodolfo Herrera, and Erik En-riquez. Projection’s panel of models for touchscreen. International Journal of Innovative Re-search in Computer and Communication Engin-eering, 1(9):2057–2064, November 2013.

[10] Jana M. Moser. Tupac lives! whathologram authors should knowabout intellectual property law.http://www.americanbar.org/publications/blt/2012/09/-02 moser.html (retrieved 26/2/2014), AmericanBar Association, 2014.

[11] Julien C Sprott. Physics Demonstrations: Asourcebook for teachers of physics. Univ of Wis-consin Press, 2006.

[12] John Rennie. The tupac hologram,virtual ebert, and digital immortality.http://www.smartplanet.com/blog/the-savvy-scientist/the-tupac-hologram-virtual-ebert-and-digital-immortality/454 (retrieved 26/2/2014),2014.

[13] D’Strict. 3D sensing holographic installa-tion. http://global.dstrict.com/projects/j4.php(retrieved 26/2/2014), 2014.

[14] Bruno Dumas, Denis Lalanne, and Sharon Ovi-att. Multimodal interfaces: A survey of prin-ciples, models and frameworks. In Human Ma-chine Interaction, pages 3–26. Springer, 2009.

[15] Kinect for windows. Available: www.micro-soft.com/en-us/kinectforwindows. Accessed:September, 20th, 2014.

[16] Leap motion. Available: www.leapmotion.com.Accessed: September, 20th, 2014.

[17] Struture sensor. Available: structure.io. Ac-cessed: September, 20th, 2014.

[18] Xtion pro. Available: www.asus.com/pt/Mul-timedia/Xtion PRO. Accessed: September,20th, 2014.

[19] R. Alves, M. Madeira, J. Ferrer, S. Costa,D. Lopes, B. Mendes da Silva, L. Sousa, J. Mar-tins, and Rodrigues J.M.F. Fatima revisited: Aninteractive installation. In Proc. Int. Multidiscip-linary Scientific Conf. on Social Sciences andArts, pages 141–148, Varna, Bulgaria, 2014.

[20] I-Ching Chung, Chien-Yu Huang, Shyh-ChingYeh, Wei-Chi Chiang, and Mei-Hui Tseng. De-veloping kinect games integrated with virtualreality on activities of daily living for children

Recent Advances in Educational Technologies and Methodologies

ISBN: 978-960-474-395-7 19

Page 10: Learning Technical Drawing with Augmented Reality and ...€¦ · Recent Advances in Educational Technologies and Methodologies ISBN: 978-960-474-395-7 11. an augmented reality application

with developmental delay. In Advanced Techno-logies, Embedded and Multimedia for Human-centric Computing, pages 1091–1097. Springer,2014.

[21] Jeremy Sutton. Air painting with corel painterfreestyle and the leap motion controller: a re-volutionary new way to paint! In ACM SIG-GRAPH 2013 Studio Talks, page 21. ACM,2013.

[22] D Bassily, C Georgoulas, J Guettler, T Linner,and T Bock. Intuitive and adaptive robotic armmanipulation using the leap motion controller.In ISR/Robotik 2014; 41st International Sym-posium on Robotics; Proceedings of, pages 1–7.VDE, 2014.

[23] Leigh Ellen Potter, Jake Araullo, and LewisCarter. The leap motion controller: a viewon sign language. In Proceedings of the 25thAustralian Computer-Human Interaction Con-ference: Augmentation, Application, Innovation,Collaboration, pages 175–178. ACM, 2013.

[24] Mischa Spiegelmock. Leap Motion Develop-ment Essentials. Packt Publishing Ltd, 2013.

[25] Chih-Fu Wu and Ming-Chin Chiang. Effect-iveness of applying 2D static depictions and3D animations to orthographic views learningin graphical course. Comput. Educ., 63:28–42,April 2013.

[26] Leap developer portal. Available: https://deve-loper.leapmotion.com/documentation/skeletal/c-sharp/devguide/Leap Overview.html. Accessed:September, 20th, 2014.

[27] Unity. Available: unity3d.com. Accessed:September, 20th, 2014.

Recent Advances in Educational Technologies and Methodologies

ISBN: 978-960-474-395-7 20