fujitsu's case study of ar/vr - documents.ossforum.jpdocuments.ossforum.jp/documentsfiles/6-9...

Copyright 2017 FUJITSU LIMITED

Fujitsu's case study of AR/VR

2017-11-16

Fujitsu Limited

0


The AR/VR market is rapidly-growth area of innovation, and it is predicted to be 18 trillion yen in 2020. (2016-2020 CAGR:134%, source:Digi Capital)

Consumer Head Mount Display was released from companies in 2016-2017, and the device market became full-scale

In the future, the “content / application” will be launched (2017-2018 VR/AR/360° video market overview, Yano Research Institute, Japan)

Trends of IT companies

Trends of AR/VR market

Facebook • Acquisition of Oculus VR for 240 billion yen (2014) • Announcement of “Oculus Rift” product version (2016)

Google • Investment of 65 billion yen in Magic Leap

Apple • Acquisition of Metaio (2015)

Microsoft • Announcement of “Hololens” (2016) • Announcements of “MR”, Windows Mixed Reality Framework,

application download on Microsoft Store (2017)

SONY • Announcement of “PlayStation VR (Morpheus)” (2015) • Declare to introduce 100 titles of VR games (2016)

1/ 19

AR (Virtual Reality)

Copyright 2017 FUJITSU LIMITED 2/ 19

What is AR?

AR is stands for “Augmented Reality”

Technology that expands and strengthen the suman senses by superimposing digital information by using ICT to real information obtained by human five senses

Real information by human senses

+

letter

Image/Movie

Sounds

Digital information

Information beyond reality


Three ways to get information to display over the image

What is AR?

AR technology is used in the field of entertainment such as games and promotions.


Position Get the information recognized by position from GPS.

Image/Object Get the information recognized by a specific image/object.

Marker Get the information recognized by marker/QR code/barcode.

4/ 19

AR proposed by Fujitsu


Inspection Maintenance Field trip

Field Work AR : Innovation on field work

Middleware Technology of Fujitsu

State-of-the-art technology of the

Fujitsu Lab. integration

Starting demonstration experiment from 2012 Productized in 2013

Sales Sightseeing

・・・

Education

5/ 19

AR integration base by Fujitsu

FUJITSU Software Interstage AR Processing Server


Use Fujitsu original AR marker and position information according to tasks

Display appropriate contents just by holding the smart device

Easy editing of contents by intuitive operation by Smart Authoring

Front Office (Field) Back Office (System)

Store data

前回の結果

Inspection Same Marker

Component replacement

Interstage AR Processing Server

Change layout Input sentence/image

Edit contents with intuitive operation

Backend System

Appropriate information

Use specific AR marker and location data for specific tasks

Display different content according to tasks

Work Procedure

AR Marker

Location information

6/ 19

Benefit of introduction AR to fields


Display the information with ease by simply holding over

Fill the gap between the field and the system

Field Office

Reflect results and awareness on the field in the system

Check the analysis results on the field

Easy to share information

Work procedure/notes Manual Result input field

Rotate to the right

Don’t touch

Manual

Directions

Tap to display File

No problem? YES/NO

7/ 19

Image of AR display


Maintenance History

Inspection result

Real time data

(temperature, pressure, etc.)

Necessary information is displayed in the correct position.

Tendency

8/ 19

VR (Virtual Reality)


What is VR?

Abbreviation for Virtual Reality

Technology to experience the cyberspace created by computers as if it were real


Methods of VR


Two methods: overhead view type and immersive type

Overhead view type Immersive type

Stationary Type Head Mount Display Cave Method

Good at looking over the whole VR space

Good at getting into VR space

Good at getting into VR space

Advantage ✔ Less VR sickness (good for children, long time use) ✔ Able to Share experiences with multiple people ✔ Ideal for use in education Disadvantage X Narrow field of vision X Low reality

Advantage ✔ High reality ✔ Full-scale world experience Disadvantage X Burden on the boy by wearing on the head X difficult to share experience and communicate with multiple people X VR sickness

Advantage ✔ High reality ✔ Able to Share experiences with multiple people Disadvantage X Large and expensive equipment X Low image quality

11/ 19

Introduction of zSpace


Stationary VR display developed by zSpace, Inc.

Stereoscopic viewing that the objects pop up from the screen through the polarized glasses.

Natural feeling to look into from the top and bottom, right and left according to the position of the viewer’s eyes by head tracking.

Able to share VR experiences with multiple people by polarized glasses for observers. It leads to share images and promote communication.

Users feel less VR sickness can use for a long time because there is little gap between image and human sense.

Using a stylus pen, users can move or rotate easily with a feeling like stabbing an object with a skewer.

Corresponding to haptic (tactile technology) through a stylus pen.

12/ 19

Stereopsis Stereoscopic vision is realized by displaying parallax images for the right eye and the left eye, and passing through polarized glasses.

Head tracking Recognize the position of the eyeglasses by receiving the reflection from the tracking point of the eyeglasses with the camera. Realized natural display that follows the position of the user’s eyes.

Stylus pen operation Recognize the position and direction of the stylus pen by infrared reflection and gyroscope function. It realizes smooth operability following the movement of the hand.

How it works


Tracking point

Stylus pen

Work station

Polarized glasses

Infrared irradiation Tracking with camera Infrared

Infrared

Tracking

Tracking point

Gyroscope

13/ 19

Benefits of VR

Visualization of invisible things (simulation)

We can touch or see things that we cannot actually see or touch in the real world.

Improvement of proficiency (training)トレーニング）

In VR, we can repeat trial and error any number of time, so we can understand the essence from that experience.

Deeper understanding (presentation)

Impress your opponent more clearly the image of what you show


Usage at University / Vocational School


Information Technology Mechanical Systems

Building Systems Electrical Engineering

VR content creation (Unity) Research of next generation VR

Research on automatic driving technology

Collision simulation Fluid simulation Robot simulation Disassembly/Assembly

training Understand product

structure

Housing design Building design Office design Urban design

Robot control simulation ATM control simulation Visualization of

electromagnetic field Visualization of electric

waves

Object recognition from in-vehicle camera

Cooperation between visual and tactile

15/ 19


Chemical, Biological, Pharmaceutical Science Design, Clothing, Art School

Medical School Game Vocational School

Visualization of protein molecular structure

Art Costume design Product design Housing design

Medical training Surgical simulation

VR game production (Unity)

The VR and game industry is really seeking Unity engineers.

Usage at University / Vocational School

16/ 19

Use Cases


１．Development of VR application using Unity

Example: Housing Simulation

Change housing layout

Simulation of sunshine an shadow conditions

Walk through roads and houses

Create animation by walking characters

Change wallpaper

• Customers can create original VR application for zSpace by yourself using Unity.

• Integrated development platform provided by Unity Technology SF.

• We can create animation, set light and shadows, set sounds, create various original screens, create menus, and so on.

17/ 19

Use Cases


２．VR display of existing 3D data (FZViewer)

• FZViewer can display 3D models in VR. • We can use VR for design review, presentation, and so on.

3D-CAD

CG Software

SfM(*)

3D-DXF format data

STL format data

OBJ format data

Converter Software

FBX, DAE, 3DS, OBJ format data

OBJ format data

(*) Structure from Motion： Technology to track the movement of feature points in a moving image and calculate the structure of the object and the position change of the camera

18/ 19

Use Cases


3. Display the 3D software screen in VR (AVR)

• We can connect the post processor and zSpace, and we can get VR results of analysis results displayed by zSpace.

• It can also display the screen of 3D-CAD, Computer Graphics Software, etc.

AVR: Middleware that makes VR display on zSpace graphic display by 3D-CAD or OpenG post-processor ( Developer: FiatLux, Co.,Ltd.)

19/ 19

fujitsu's case study of ar/vr - documents.ossforum.jpdocuments.ossforum.jp/documentsfiles/6-9...

Documents