차세대 홀로그램 정보기술의 현황 및향후 발전전망

2012. 10.10

김 은 수

광운대학교 전자공학과 교수

차세대 3D 디스플레이 연구센터 센터장 (3DRC)/ www.3drc.org홀로디지로그 휴먼미디어 연구센터 센터장 (HoloDigilog)/ www.holodigilog.org‘3D Research’ 국제저널 편집장/ www.springer.com, www.3dresearch.org

■ 2012 IT 산업전망 컨퍼런스 초청강연

1

CONTENTSI. CURRENT DEVELOPMENT STATE OF HOLOGRAPHIC 3D

INFORMATION TECHNOLOGY & INDUSTRY 1. Impacts & Lessons from a 3D Movie of ‘Avatar’2. Why and What is 3D Information Technology ?3. Paradigm of 3D Display Technology 4. Development Trend of 3DTV Technology & Industry 5. Issues in 3D Information Technology Business & Discussions

II. FUTURE PROSPECTS OF HOLOGRAPHIC 3D INFORMATION TECHNOLOGY & INDUSTRY

1. Vision 2020: Interactive Holographic Virtual World 2. Why and What is Interactive Hologram Human Media ?3. Core Technologies for Interactive Hologram Human Media4. Brief Introduction of ‘HoloDigilog human Media Research Center (HoloDigilog)’

2

I. CURRENT DEVELOPMENT STATE OF 3D TECHNOLOGY

3

1. Impacts & Lessons from a 3D Movie, ‘Avatar’

4

■ Since ‘AVATAR’ was released on 2009. 9- 3D fever is prevailing throughout the world - Strong impact on our daily life- Now, making a new IT business paradigm- Birth of various 3D IT fusion industry• 3D Tech + Traditional 2D IT Industries

■ Backgrounds- Natural evolution from 2D to 3D IT industries• Saturated 2D IT technology development & market

- Matured 3D technologies• 3D Display, 3D Camera, 3D Contents, 3D S/W, etc

- Sonny’s Announcement of 3D Business(2009) • Samsung & LG had to respond for Sony

■ Limitations of a stereo 3D movie, ‘AVATAR’- Human factor problems• Eye fatigue, dizziness (Fundamental)• Cross-talks (almost solved/ less then 4%) • Mismatch between the focusing & convergence point

- Back depth, mixing with lots of 2D scenes - Limited running time (2.5 hours) - Need of polarized glasses

■ Development need of a real 3D display- Holographic 3D display- 3D Integral-imaging, Volumetric 3D display

AVATAR(2009.12/ JAMES CAMERON)

‘AVATAR’

■ Fever of a Hollywood 3D Movie, ‘Avatar’ in 2009 (Screened in 76 Nations)■ ‘3D’ Just an EVENT or a NEW PARADIGM ?

- Strong impact on our daily life- Opening & triggering the future ‘3D World’

5

Theater in Korea

Screen Types Ticket Price

CGV

Digital 3D 13, 000 won

IMAX 3D 16, 000 won

Stadium 3D 14,000 won

Mega box Digital 3D 13,000 won

Lotte Cinema Real D 3D 13,000 won

■ 3D Movie of ‘Avatar’- Production cost : 0.3 billions (USD)- Production period : 4 years

■ Total grosses as of June 24, 2010- Worldwide: 27.3 billions (USD) (910% Profit)

■ Number of nations where ‘Avatar’ is screening- 76 nations including USA

■ Number of Audience to go to the movie in Korea- Audience: more than 15 millions (Population:50 million) - Ticket Price: 2D Avatar (8,000), 3D Avatar (13,000)

IMAX 3D(16,000)

Box-office Records

BOX-OFFICE RECORD OF ‘AVATAR’

■ 3D Movie of ‘Avatar has made a new box-office record in terms of net gross profit,

number of audience

6

IMPACTS & LESSONS OF ‘AVATAR’

■ IMPACTS▪ Triggering a new 3D business throughout the world

- Bring up the 3rd peak pinnacle of the 3D Industry in 2010▪ Timely motivation for promotion of various kinds of 3D fusion industries

- 3D Mobile, 3D Game, 3D Broadcasting, 3D Medicine, 3D Advertisement, etc- Particularly, accelerating the 3D Movie, 3DTV broadcasting business

- 3D Camera- Computer graphics- 3D Scanner- etc.

- Format- Compression- Transmission- etc.

- Stereoscopic- Autostereoscopic- Volumetric 3D,- Real 3D Display

3D Data Acquisition, Computer Synthesis

3D Image Dataprocessing

3D ImageDisplay

Convergence of 3D Technology with Conventional Industries

3D FUSION INDUSTRY3D Game/Animation

3D Home Networkng

3D Fashion

3DConstruction

3DArchitecture

3D Performance

3DMuseum

3D Advertizement

3D Security

3D Mobile 3D Machinery

3D Medicine

3D Broadcasting

3D Communication

3D Education

3D Movie

3D Shopping

7

■ LESSONS1) Recognition of Importance of 3D Contents and Displays▪ Need of lots of experiences for production of 3D movies - Totally different from 2D movie - 3D camera shooting techniques, 3D contents generation techniques

(Real + CG scenes), 3D video projection technology - Director, James Cameron is the right person to make a good 3D movie of ‘Avatar’ ▪Why prefer to 3D Movies ?

- At the first time, curiosity on 3D might make lots of audience to move to the 3D movies- But, as the number of screened 3D movies increases, movie story and message were

regarded as more important things for the audience (Avatar) - Movies just made with 3D technique couldn't get much attention from the audience (‘Titan’)- Of course, 3D movies have their killer applications (Sports, etc)

2) Recognition of limitation of the conventional stereoscopic 3D movies ▪ Stereoscopy was developed in 1838(174 years before)

- Not a brand new, but a old technology▪ 3D Avatar is basing on stereo disparity

- A pair (right & left) of 2D views are generated and separately input to the corresponding eyes with polarized or shuttered glasses

8

CONTINUED

▪ Need of polarized glasses▪ Human factor problems

- Dizziness and eye fatigue: ‘fundamental’- Mismatch between the converging & focusing points of a 3D image▪ Need of safety guideline, evaluation and verification criterions of 3D contents quality ▪ Need of international standards of 3D contents and display technology

- Various types of 3D movie

3) Same to the case of 3DTV Broadcasting business (more severe than 3D Movie): ‘watching 24 hours’

4) Development need of a real (Hologram) 3D display technology- Being free from human-factor problems- No need of special glasses- Implementation of a cyber space in the real world

9

CONTINUED

2. Why & What is 3D Technology ?

10

WHY 3D DISPLAY TECHNOLOGY ?

• SF 영화 (Hollywood, USA) -다양한미래의모습이현실처럼묘사-물리적인실체는아니지만말과행동외형모든것이실제사람을직접대하는

것과똑같음

-차세대 3D Display의전형적인예임

• 미래정보디스플레이-정보의최종수혜는인간임-보다자연스럽고, 인간친화적인실감정보요구 (3D 공간정보 = 완전입체)

- Virtual Reality 구현기술

• 향후 3D 연구개발방향-차세대완전입체형 3D 디스플레이핵심기술연구개발추진

-차세대신성장동력 3D 융합산업개발- FPD 에이어 3D 디스플레이에서도세계최강국 지속

-차세대 3D 융합산업의세계시장선점및주도 (3D Blue Ocean)

꿈의 차세대 완전 입체형3D 디스플레이 기술

IRON MAN (Hollywood/2009)

11

■ Since the 1st 3D device called ‘Stereoscope’ was developed by C. Wheatstone in 1838 ■ 3D Technology has been evolved with a period of 60 years during the past 174 years(1838-2012)

- 1st development stage of 60 years (1840s-1890s) – Reached its 1st peak popularity in 1890s (Stereoscope)- 2nd development stage of 60 years(1900s-1950s) – Reached its 2nd Peak pinnacle in 1950s (3D Movie)- 3rd development stage of 60 years(1960s-2010s) – Reached its 3rd peak popularity, but expects to continue to go up (3DTV, 3D Movie, etc…….3D fusion industries)

BS11 (2008)

Worldcup 2002

Nagano Olympic1998

The Wizard of Oz1974 NTV

1890’s 1950s 2010’s

■ 3rd 60 years (1960s-2010s)- 3D Movie, 3DTV, 3D Fusion Industries- Its popularity is now rapidly going up

Worldcup 2010Avatar(2009)

EVOLUTION HISTORY OF 3D TECH.

■ 2nd 60 years (1900s-1950s)- 3D movie reached its 2nd peak pinnacle

(Viewmaster, Kodak Realist Camera) ■ 1st 60 years (1840s-1890s)- Stereo photography reached its 1st

peak popularity (Stereoscope)

12

WHAT IS 3D TECHNOLOGY ?

3D DATA CAPTURING, GENERATION,

PRODUCTION, EDITING

3D COMPRESSION, TRANSMISSION, STORAGE

3D RECONSTRUCTION, REPRESENTATION

3D Interaction

3D camera, Depth Camera, 3D CG,

2D/3D Conversion

3D IPTV, 3D DMB, 3D Satellite TV, 3D CATV, 3D Terrestrial TV

3D Media Player (Blue ray)

3D Projector, 3D HMDMulti-view display,

Holographic 3D display

■ 3D Technology includes 3D data acquisition, processing, transmission,display and interaction

13

WHAT IS 3D DISPLAY

STEREOSCOPIC 3D DISPLAY(TRUE 3D)

HOLOGRAPHIC/VOLUMETRIC 3D Display

(REAL 3D)

- 2D views in 2D space- Stereoscopic/

Autostereoscopic- Multiview/Supermultiview- Compatible with 2D-HDTV- Eye fatigue, Dizziness

• 안경방식 (Stereoscopic)-사람이안경을씀 (편광안경, 셔터글라스)

• 무안경방식 (Autostereoscopic)-디스플레이장치가 안경을씀(Lenticular/ Parallax barrier)

• 완전입체방식(Holographic/Volumetric)-안경필요없음, 이상적인 3D기술

- 3D views in 3D space- Ideal 3D Technology- ∞ views/HPO- Limited Implementation

■ Mimic Human Visual System (HVS)

Stereo Disparity

[현재 50 여가지 3D 디스플레이 기법 개발]

14

WHAT IS 3D DISPLAYS ?

2D images on conventional 2D displays (2D) Rendered 2D images on conventional 2D displays (2D)

Two-pair image (Stereo) on conventional 2D displays (2.5D) Real 3D image on space (3D)

15

3. PARADYEM OF 3D DISPLAY TECHNOLOGY

16

TYPES OF 3D DISPLAYS■ Comparison of True & Real 3D Displays

True 3D Real 3D

With Glasses

■ Stereoscopic Display- Polarized Glasses, Shutter Glasses- Head Mounted Display (HMD)

Without Glasses

■ Autostereoscopic Display- Lenticular Sheet- Parallax Barrier- HOE

■ Realistic 3D Display- Holographic 3D (Electro/Digital)- Volumetric 3D (Integral imaging,

Scanning 3D)- Spatial 3D (Floating 3D, Virtual 3D)

상의 변화

■ 시점을변화시켜도불변

- 깊이감 : 양안시차

■ 시점을 변화시키면 변화

- 깊이감 + 변화감 : 양안시차 + 운동시차

연구개발

방향

■ 원리적으로는 2D 영상 (양안시차)- 현재 TV와 인터페이스 가능

- 주변기술 확보/실용화가능- 초 다시점 개발 중

■ 원리적으로는완전 3D 기술- 주변기술 미개발

(신호처리 및 디스플레이소자 기술)- Huge Data (TB 이상), LCD (극 초고해상, 고집적)

17

■4 Kinds of Key Technologies for 3D Displays 1. Stereoscopy (Wheatstone Stereoscope in 1838)

- Mimic THE human visual system (HVS) 1) Stereoscope (1838)

2) Stereoscopic 3D Displays with Glasses

(C. Wheatstone)

(David Brewster)

(Oliver Wendell Holmes )

(Ivan Sutherlang)

(Karl Pulfrich)

(Joseph D'Almeida)

(John Anderton)

(Wilhelm Rollmann)

PARADYM OF 3D DISPLAYS

18

3) Auto-stereoscopic & Multi-view Stereoscopic 3D Displays without Glasses

■ Lenticular-sheet/ Parallax-barrier

■ Lenticular sheet/ Parallax barrier-based Commercial 3D Displays

(F. E. Ives)

(H.E. Ives)

CONTINUED

19

4) Achievements and Limitations of the Conventional Stereoscopic 3D Displays

Stereoscopic 3D : Mimic of human eye-brain system

Stereo disparity

▪ Achievements- Commercially available 3D display technology- Easy and simple implementation - Compatibility with conventional 2D systems- So far, it has so many practical applications

※ BS11: Commercial 3D Broadcast TV (2007.12, Japan)

※ 7,000 3D Screens : 3D Movies in Digital Cinemas

▪ Limitations- Human factor problem; Fundamental - Eye fatigue, dizziness (2D views in 2D plane)- So far, 3D displays lose their value or fatigue

us within 10 minutes

▪ Issues- How can we solve the human factor problems of the

conventional stereoscopic 3D displays- Inside Media/3D@Home Consortium (USA)

: 3D contents generation

CONTINUED

20

5) Why not a SMART 3D product on market ?

So far, 3D displays lose their value or fatigue us within 10 minutes“10 MINUTES MEDIA LIMITATION”

Real World (3D space)

3D Display System

Recognition of 3D object

■ Psychophysically, human recognizes 3D images not just with input data of the 3D object itselfF, but with the results filtered with brain information processing

■ Fully understanding of brain information processing is very important for development offuture real 3D display systems : Need of interdisciplinary co-works

Stimulus

Response

Brain information processing(Filtering)

CONTINUED

21

2. Real 3D displays

1) Volumetric (Integral Imaging) 3D Displays (G. Lippmann in 1908)

(G. Lippmann/ 1908)

(Toshiba)

(Kwangwoon Univ.)

(Kwangwoon Univ.)

(Seoul Nat’l Univ.)

(Seoul Nat’l Univ.)

(NHK)

(Seoul Nat’l Univ.)

CONTINUED

22

2) Spatial 3D (Floating/ Virtual) Displays

■ Fresnel lens based-Floating 3D Display, Half mirror-based Virtual 3D Display

(Haruka, Pioneer)

(Virtual stage, Denmark)

(3DRC)

(3DRC)

(Bauhaus-University Weimar,

Germany)

(Musion, UK)

(Provion, USA)

(Holoart, Japan)

CONTINUED

23

3) Holographic 3D Display (D. Gabor in 1948) – 60th Anniversary

CONTINUED

24

4) Achievements & Limitations of Real 3D Displays

■ Achievements- No human factor problems (eye fatigue, dizziness)- Final solution for human-friendly 3D displays

■ Limitations1) Holographic 3D

- Prof. Bove(Media Lab of MIT, USA)- SeeReal Technology(Germany), QinetiQ (UK)- Huge data, unavailability of key devices

(SLM, CCD…), Computational complexity2) Integral 3D display

- Toshiba (3D game), NHK (3DTV)- Huge data, unavailability of key devices (LCD,

MLA, CCD…), Narrow viewing angle & depth, Low resolution

3) Floating 3D display- 3DRC (3DRC-60), Pioneer (Haruka) - Narrow viewing angle, image distortion- 2D videos floated in real space

4) Virtual 3D display- 3DRC (3DVD-360), Musion (Denmark),- Bulky system- 2D videos projected in virtual plane

(mirror images, known as hologram images)

CONTINUED

25

■ Two approaches for SLMs for displaying hologram patterns(1) Ultra high-resolution LCD SLM (pixel size: a few μm): HoloEye (Germany)(2) Photorefractive photopolymer-based dynamic holographic SLM

(resolution: 1,000 - 20,000 lines/mm) : Arizona Univ.(USA) - Photorefractive photopolymer expects to be developed (Bayer, Dupont)- Then, holographic 3DTV could be implemented- Photorefractive photopolymer: Dynamic grating can be formed, hologram pattern can be changed

in real time, so that it can be used as a SLM for holographic 3DTV

HOLOGRPHIC 3DTV

Digital 2DTV Stereoscopic 3DTV Holographic 3DTV

2D image Two 2D images

Holographic pattern

26

27

WHY HOLOGRAPHIC 3D DISPLAY ? ■ 3차원 정보를 관측자가 인지하는 유일한 방법은 물체로 부터 산란된 빛을 눈으로 직접

봄으로 써 만 가능

■ 진정한 의미의 3차원 입체정보는 광학적으로 얻어진 홀로그램임

■ 따라서 , 현재까지 홀로그래피는 단 하나의 완전 3차원 입체 영상기술

■ 차세대 3D 영상통신 시스템의 궁극적인 모델은 홀로그래피 기술임

28

WHAT IS HOLOGRAPHY ? ■ Optical Holography

-광파의간섭과회절현상을이용한파면의기록기술

■ For real-time application: Digital Holography/ Electro-optic Holography- Recording (CCD Camera), Computer-generated Hologram (CGH)- Reconstruction (LCD Display)

PIONEERS IN HOLOGRAPHY■ Pioneers

- D. Gabor, “ A new microscope principle,” Nature 161, 777-778 (1948) - E. N. Leith and J. Upatnieks, “Reconstructed wavefronts and communication theory,” JOSA 52,

1123- 1130 (1962). - Y. N. Denisyuk, “Photographic reconstruction of the optical properties of an object in its own scattered

radiation field,” Soviet Phys Doklady 7, 543-545 (1962)■ Computer Generated Hologram (CGH)

- A. W. Lohmann and D. P. Paris, “Binary Fraunhofer holograms generated by computer,” AO 6, 1739- 1748 (1967)

■ Practical Hologram - S. A. Benton, “Hologram reconstructions with extended incoherent sources,” JOSA 59, 1545-1546 (1969)

■ Digital Holography - J. Goodman and R.W. Lawrence, “Digital image formation from electronically detected holograms,”,

Applied Physics Letters, 11, 77-79 (1967)

29

30

WHY HOLOGRAPHIC 3D IS SO DIFFICULT?

Note BookImage

Hologram(Fringe Pattern)

Transmission Display

■ Computational Complexity-초고속홀로그램계산

S/W, H/W 개발요구

■ Computation- 1 frame hologram

pattern with aworkstation

- a few minutesEx) MIT(HPO)- 7 seconds

■ Transmission B/W- 30 billion x

256 x 39 frame/sec= 720THZ

■ SLM- Size : 10 inch - Pixel Size : 1um- Pixel number

: 30 billion

■ Vast data to be transmitted -초고속정보통신망개발요구

■ Super-resolution LCD

-초고정세, 초고해상도LCD SLM 개발요구

4. DEVELOPMENT TRENDS OF 3DTV TECHNOLOGY & INDUSTRY

31

DEVELOPMENT TREND OF 3D INDUSTRY

■ Mass production of 3D Movies: Hollywood■ Deployment of 3D Movie Projection System: Real D, Dolby, Master Image, XpanD, etc■ On market of 3D, 3D-Ready TV: Samsung, LG, Hyundai IT, Sony, Panasonic, Toshiba, Sharp, etc■ Start up of 3DTV Service: BS11(Japan), Skylife(Korea), ESPN, Direct TV(USA), BSkyB (UK), etc■ 3D Image Format & Safety Standards: SMPTE, ISO, ITU-R SG6, etc)

■ LOCALENTERTAINMENT

MARKETS

THEME PARKS

EXHIBITIONS

MUSEUMS

HOME THEATERGAME

WEB

■ HOME ENTERTAINMENT

MARKETS

HOME THEATER

GAME

WEB

BROADCASTING

DIGITAL MULTIPLEX CINEMAS(MOVIE)

■ Local entertainment markets are fast transferred to the home entertainment markets (Huge markets) - By recent development of 3D technology- Now 3D movie business is booming up, so that 3D movies can be moved to the home

32

DEVELOPMENT TREND OF 3D CONTENTS

■ 3D CAMERA (LIVE)• DUAL(STEREO) CAMERA

- 3ality Digital, PACE, Fusion,

Cobalt, etc• SINGLE LENS

- Sony(240fps)• TWIN LENS

- Panasonic P2• LENS ADAPTER

- Wasol

■ 2D-3D CONVERSION S/W

• COST

- 10-15 millions(USD)/movie

• Company

- IMAX, Sony Pictures, In Three,

Magnetic 3D, HDLogix, Zero

Creative, Samsung, JVC, etc

■ CG 3D (SYNTHESIS)• Use of various 3D S/Ws

- 3D MAX

■ Real 3D + CG 3D• Live action 3D scenes+

Computer generated 3D

scenes: ‘Avatar’

Generation of 3D

Contents

■ 3D Contents can be generated by1) 3D Camera (Live scenes)2) 2D-3D Conversion S/W (?)- Samsung 3DTV3) 3D CG (Computer-generated 3D scenes)4) Real 3D + CG 3D (Mixed: Avatar) - Common approach for making 3D films

33

DEVELOPMENT TREND OF 3D EQUIPPMENTS

■ STORAGE & SWITCHING

• Sony- 3Gbps Input/Output VTR

(SRW-5800), - AV Routing Switcher

■ EDITING

• Quantel iQ/Pablo- Color Compensation of

L/R images

■ PROCESSING, COMPENSATION

• 3ality Digital- Stereo image processor

(SIP) 2100• Fraunhofer

- 3D image analysis tool

■ ENCODER/DECODER

• TDVision- Under development of

H.264-based 3D Codec

EQUIPPMENTS FOR 3D IMAGE STORAGE,

EDITING, PROCESSING &

COMPRESSION

■ Several key 3D equipments have been developed or under developing

1) 3D data storage and switching 2) 3D data editing 3) 3D data encoding and decoding 4) 3D data processing and compensation

34

PLATFORMS FOR 3DTV BROADCASTING

• BSkyB (UK) - 2008. Experimental Broadcasting (Sports, Concert )- Start of 3D Service (1H, 2010)

• BS11 (Japan)- Dec. 2007 a half-hour broadcasting every day- Side by Side Format

• Sky life (Korea)- 2010, Education Channel Experimental Broadcasting (1.1) CH1

BSkyB 3D

BS11 3D

Satellite

Anaglyph

Side by Side

• Comcast (USA)- Experimental Broadcasting (1H, 2010)

• Cablevision (USA)- Experimental Broadcasting (2010)

• Korea- 2010 Pilot Broadcasting- 50 households, CJ (Pilot-projects), HCN, T-broad

Cable

• YouTube - Anaglyph, Side by Side Format support

• CINEMANOW- 2009. 4Q - 3D cinema service- Using Nvidia 3D Vision System

Broadband

■ VARIOUS KINDS OF PLATFORMS ARE USED 3DTV BROADCASTING- Satellite, Cable, Terrestrial, Blue-ray, Broadband, IPTV, Mobile DMB, etc

35

5. Issues in 3D Information Technology Business & Discussions

36

ISSUES IN 3D IT BUSINESS 1. Killer Applications of 3D Information Technology

- Entertainments- Sports- Cyber Educations- Advertisements

2. Risk of 3D IT Market- Human factor problems (dizziness, eye fatigue)- Expansion of 3D porno contents- Side effects: addiction to 3D games- Social inverse functions

3. Business Model of 3D IT- BM of Killer applications- Optimized service configuration including platform - Providing and delivering model of 3D Contents- What is the business market for making money※ A Case of Global business Model: LG Electronics provides 3DTV to the

Valuable Group of India. They are all installed in restaurants nation widely and Cricket games are lively broadcasted to the restaurants customers through these 3DTV

37

CRITICAL ISSUES IN 3D IT BUSINESS1. Safety

1) Human factor problems (Eye fatigue & dizziness) in 3DTV, 3D Movie- Fundamental problem in stereoscopic 3D displays --- can be alleviated - 3D Movie: Screening time is limited (Avatar: 2.5 hours, Weak & back depth, mixed with 2D scenes). But, it was reported that one man died in Taiwan

- More critical for the case of 3DTV, because they can watch 3DTV 24 hours(Terrestrial 3DTV Broadcasting in Korea on Nov. 2010)

2) Need of guideline for safely watching 3DTV- Old and drunken person, children including normal person- Preparation of safety guideline for their 3DTV in LG, Samsung

3) Development Need of the Real 3DTV(Hologram 3DTV): 2015-2020

2. 3D Standards1) There are so many types of 3D camera, display, 3D broadcasting system

- Need of Global 3D Forum for 3D standardization- Especially, 3D Display Forum for standardization is mostly needed

2) Various 3D image formatting, compression, transmission and storage methods 3) Need of domestic (Samsung, LG) and international 3D standardization

3. 3D Contents1) How much can we supply 3D Contents worth watching on 3DTV ?2) 2D-3D conversion (experience-based works - time-consuming)3) Need of lots of 3D CG experts, 3D camera shooting experts: takes some times

38

CRITICAL ISSUES IN 3D BUSINESS

4. Need of Evaluation standards for 3D Contents and 3DTV Systems1) Development of evaluation standards for 3D Contents, 3D Displays2) Need of Verification institute for the 3D Contents and displays

5. Development of Next-generation Real 3DTV 1) Free from Human factor problem2) Real Volumetric 3D imaging and display3) Hologram, Integral imaging 4) Expects be developed during 2015-2020(Korea, Japan)

39

II. FUTURE PROSPECTS OF HOLOGRAPHIC 3D INFORMATION

TECHNOLOGY & INDUSTRY

40

1. VISION 2020: Emotionally Interactive

Hologram Virtual World

41

▣ 2020 VISION: - Live mobile-hologram broadcasting of soccer game from Seoul

to London (Free-space Hologram Mixed Reality at the Office or Home)

Now in 2010

In 2020

A person or family at the office and home is watching the soccer game

viewed on the smart phone & TV A person or family at the office and home is seeing the hologram soccer game

playing on the free-space

42

2. Why & What is Interactive Hologram Human Media ?

43

44

WHY HOLOGRAM HUMAM MEDIA

■ Various imagination scenes in the SF Movies provide us- Visions of the future-coming science and technology- Potential needs for demanding technologies and products in the future society (70-80%)- Specific features of the final products of the high-techs now under developing

■ Hologram Human Media viewed in most SF Movies - Not a physical reality, but it can make emotional communications with real persons on the same space - No more a science fiction, but it is now coming closely to our real life- Matured technologies in 3D display, CG, Emotion recognition, Haptic technologies, etc. - Expect the revolutionary changes of human life in the near future society (2020)

▶▶▶ Hologram Human Media: 21C Dream Project for Scientists and Engineers

● SF Movies show us some features and aspects of the future society

45

홀로그램메시지 점열

(깜빡임)

+ 문자가도착했습니

다(음성)

①

②

버튼은 누르면 화면 사라짐

③

문자의 발신이 공간에 Floating

■ PR video to attract and host of the World Cup 2022 (Japan)• Development plan of the Hologram 3DTV for live broadcasting of the World Cup 2022• Football games played in Japan---Live Broadcasting throughout the world with the Hologram 3DTV system- Audience on the stadium(Real World), Hologram virtual soccer players in the lawn ground(virtual World)- Free-space hologram–based Mixed Reality(Real + Virtual) on the same space- Free-space hologram: 360 viewing, Projecting into the real space, coexist with the real world, mutual interaction

■ Conventional approaches(Monitor-based) • SeeReal Technologies(Germany), Arizona University(Dynamic Gratings in photopolymer, USA), Qinetiq(UK)• Virtual world in the monitor, Cannot applied to the Mixed Reality

▶▶▶ Revolutionary changes in the future Broadcasting Media

● Live broadcasting of football games being held in Korea to Londonthrough a free-space holographic 3DTV

WHY HOLOGRAM HUMAM MEDIA ?

46

홀로그램메시지 점열

(깜빡임)

+ 문자가도착했습니

다(음성)

①

②

버튼은 누르면 화면 사라짐

③

문자의 발신이 공간에 Floating

■ SK Telecom PR on TV(Popular singer ‘I You’ co-playing with the late singer(Kim Kwang-Suk)• A singer can sing with the past singer on the same stage (HoloSpace)

- Provide us lots of space and time freedom in the future art performance- Emotional interaction with the audience

• I You(Real world) + Kim Kwang-Suk (Virtual World)---- Mixed Reality on the same space• Kim, Hyun-Sik & His Son (2010)▶▶▶ Revolutionary Art and Culture beyond time and space in the HoloSpace

● Future Art performances beyond Time and Space

WHY HOLOGRAM HUMAM MEDIA ?

47

■HoloDigilog- Co-existence of the Real World and the Virtual World(Free-space hologram-based) in the same space- Hologram Digilog(Holo-Digilog): New concept for Free-space Hologram Mixed Reality

■HoloSpace- Free-space generated with the concept of Holo-Digilog

■Interactive HoloSpace- HoloSpace in which emotional interactions are made between the real and the virtual spaces

1. New Concept of ‘HoloDigilog’

WHAT IS HOLOGRAM HUMAN MEDIA ?

WHAT IS HOLO-DIGILOG ?

48

2. Hologram Human Media

■HHI(Human-Hologram Interaction)- Core technologies for making interactions between the Human and the virtual hologram

(※We assign this new academic research field as the ‘HHI’ for the first time in the world)

■HoloDigilog Human Media (Hologram Human Media)- Human-centric HHI (Representation, Interaction, communication, recognition, etc)

3. Core Technologies for Hologram Human Media

49

▣ Display Technology of 360° Holographic 3D Videos in the free-space• 3D Hologram Display in SF Movies: Monitor-less/ Free-space projection

- Totally different from the conventional 3D display technology- Taking the images out of the monitor - Co-existence of Virtual and Real spaces on the same(free) space

: True Mixed Reality - 360° 3-D images in the free-space- Physical contact and emotional interaction with the hologram person

• Key issue in free-space 3D Hologram- There is no scattering medium in the free-space- Need of new approaches

50

CORE TECHNOLOGY

51

▣ Generation of Free-space Hologram Contents▶ Free-space Hologram Contents enhanced in both visual and emotional realism

• Camera-based generation of real 3-D contents (Real Objects)- Depth and intensity data

• CG-based generation of 3D contents (Virtual Objects)- Depth and intensity data

• Generation of hologram patterns for the camera-based/CG-based 3D contents- N-LUT

CONTINUED

52

▣ Human-Hologram Interaction(HHI) Technology▶ Object sensing/ recognition,

Human activity/ emotion/ context awareness, inferenceHuman emotion evaluation, decisionHaptic technologies

▶ Platform technology for Interactive HoloSpace Media• Real and virtual world sensing/recognition• Multi-Modal sensor-based activity/ emotion/ context recognition architecture • Brain/Body-based emotion/ intelligence recognition and evaluation• Haptic interaction with free-space hologram• Platform S/W for Interactive HoloSpace Media

CONTINUED

4. Brief Introduction of - HoloDigilog Human Media Research Center

(www.holodigilog.org)- 3D Display Research Center

(www.3drc.org)

53

1. NRL(National Research Lab) Project : True 3D (Stereoscopic 3D)• Name of NRL

- NRL of 3D Media • Supported by

- Ministry of Science and Technology• Duration of support and Research fund

- 2000.6 - 2005.5 (5 years)

2. ITRC Project: Real 3D (Holography, Integral Imaging) • Name of ITRC- 3D Display Research Center(3DRC)

• Supported by - Ministry of Information & Technology(Ministry of Knowledge Economy)

• Duration of support and Research fund - 2003.9 - 2011.12 (8 years)

3. ERC Project: Free-Space/ touchable/ 360/ Hologram Virtual Reality• Name of ERC

- HoloDigilog Human Media Research Center (HDHM-ERC) • Supported by

- Ministry of Education, Science and Technology• Duration of support

- 2011.9 - 2021.8 (7 + 3 years)

54

NATIONAL PROJECTS ON 3D MEDIA

BREIF INTRODUCTION OF ‘HOLODIGILOG’

55

■ Name of ERC• HoloDigilog Human Media Research Center (HoloDigilog)

(Kwangwoon Univ.)

■ Supported by• National Research Foundation (NRF)

Ministry of Education, Science and Technology of Korea (MEST)

■ Duration of support• 2011.9 - 2021.8(7+3)

■ Research fung• 1.5 millions(USD)/ year

■ Member of Researcher (2011-2012)• Professor: 13 (8: Kwangwoon Univ., KAIST, GIST, KIST, Korea Univ., Kyunghee Univ.,

Sangmyung Univ., Chungbuk National Univ.)• Dr. and Ph. D & MS candidates: 60• Industrial Collaboration: 16 (Samsung, LG, SKT, ……)• International Collaboration: 25 (MIT, Stanford Univ.(USA), Chiba Univ., Tokyo Tech (Japan),

Ben-Grion Univ.(Israel), Peking Univ.(China), Valencia Univ.(Spain)….)

INTERNATIONAL COLLABORATION

56

HDHMRC (Korea)

Anhui Univ./ Appl. Sci. Tech. Res. Inst./ BIT/ C3D/ Jilin Univ./ Peking Univ./ Tsinghua Univ./ Zeh Jiang Univ. (China)

Chiao Tung Univ. / 3DIDA /Nat'l Chiao Tung Univ. (Taiwan)

3DC/ Chiba Univ./ Kobe Univ./ Nagaoka Univ./ Nihon

Univ./ Sharp/ SIT/ Takarazuka Univ./ Tokyo Institute of Tech/ TUAT/

Utsunomiya Univ./ WasedaUniv.(Japan)

Curtin Univ. / Australian National Univ. (Australia)Univ. of Canterbury (New Zealand)

MacGill Univ. (Canada)

3D@Home Consortium/ Boston Univ./ Connecticut Univ./ Diversified Res. Co./ MIT/

Pennsylvania State Univ./ Stanford Univ./ UNC-Chapel/ Univ. Central Florida/ Univ. of Arkansas/ Univ. of Califor. Irvine/ Univ. of Colorado/ Univ. of Florida/ Univ. of North

Texas/ Univ. of Texas at Austin/ USC/ Virginia Tech. (USA)

• England : Cambridge Univ./ De Montfort Univ./ (Univ. of Ulster)• Germany : Dresden Univ./ SeeReal Tech./ Univ. of Siegen/Tech. Univ. of Dortmund/ Univ. of Stuttgart/

Franauhofer FIT/ Fraunhofer IDMT • Ireland : Univ. of Ireland Maynooth• Russia : Saratov State Univ. • Hungary :Budapest Univ./ Holografika• Denmark : Riso Nat’l Lab. • Sweden : Luleå Univ. of Technology• Netherland : Erasmus Univ. • France : Louis Pasteur Univ.• Israel : Ben-Gurion Univ./ 3DV Systems Ltd. • Turkey : Bilkent Univ.• Spain : Valencia Univ. • Italy : Istituto Nazionale di Ottica Applicata

■International collaboration network ▪ 75 universities and institutes from 20 Nations ▪ Joint research, Invited talks, Technical consultants, CC3DR, Journal editors of 3D Research

Thank you !

57