chris wren (wrenar llc): beyond the hands
TRANSCRIPT
Join the conversation @ARealityEvent & #AWE2017
Exploring the interactive boundaries of VR
Beyond the Hands
Chris Wren
WHAT’S THIS TALK ABOUT?
• Survey of Existing/Upcoming Sensory Tech for VR • Showcase my work in this area / Lessons learned
• VR Avatars • Full Body VR • Mixed Reality • Hand Tracking
• A little bit of VR Social • Where are we going?
WHO AM I
Chris Wren • 12 years professionally developing games (Microprose/EA/Namco) • 9 years instructing Game Design at George Mason • I do AR/VR research/consulting in Virginia: www.wrenar.com • I work with an amazing AR/VR company: www.notiontheory.com
MyFirstJob
“TheSims”I shipped 7 Sims titles in 4 years.
Warhammer:MarkofChaos
Inspirational signposts of what AR/VR interaction might be like
More real than real.
WHY ARE WE DOING THIS?
TAPPING THE SENSES
SIGHT
• Higher FOV • Better Pixel Density • Better Lenses • Higher Framerate • Beyond the Visual Spectrum • Transferring to AR
FUTURE SIGHT
• Thermal • IR • Lightfield • Retinal Projection • Bypassing Optic Nerve?
SOUND
Ambisonics
TOUCH
Reach out, reach out and touch someone. – 1980’s ad campaign from AT&T
HAPTIC GLOVES
NOTION THEORY GLOVES
VR TOUCH
Haptic Fingertips
RAINBOW JELLIES AND LEAP MOTION
Synesthesia
MONSTER SHOP AND USENS
Maybe we don’t need controllers…
TACTICAL HAPTICS
Strafe Haptics
CORNELL ORGANIC ROBOTICS
OmniPulse
SOFT ROBOTICS AND VR
Future Peripherals
UNLIMITEDHANDS
H2L Inc (China)
MUSCLE STIMULATION
Hasso-Plattner-Institut in Potsdam
WHERE IS THIS HEADED?
VIRTUAL HANDSHAKES
THE SUITS The outside of the suit was covered with an elaborate exoskeleton, a network of artificial tendons and joints that could both sense and inhibit my movements. Built into the inside of the suit was a weblike network of miniature actuators that made contact with my skin every few centimeters. These could be activated in small or large groups for the purpose of tactile simulation—to make my skin feel things that weren’t really there. They could convincingly simulate the sensation of a tap on the shoulder, a kick to the shin, or a gunshot in the chest. -Ernest Cline, Ready Player One
TESLA SUIT
SKINTERFACE
Student Project from London’s Royal College of Art
SUBPAC
HARDLIGHT VR
NullSpace VR
WHERE IS THIS HEADED?
VIRTUAL HUGS
SMELL
OLFACTORY BULB
FEEL REAL
Heat, Wind + Smells
NOSULUS RIFT
Not all smells are welcome in VR
BRAIN HACKED COOKIES
University of Tokyo
MOVEMENT I spent the majority of my time in my Shaptic Technologies HC5000 fully adjustable haptic chair. It was suspended by two jointed robotic arms anchored to my apartment’s walls and ceiling. These arms could rotate the chair on all four axes, so when I was strapped in to it, the unit could flip, spin, or shake. -Ernest Cline, Ready Player One
CABLE ROBOTS
FraunHofer IPA
GIMBALS
TREADMILLS
VIRZOOM
MARS BUS
• Made by Framestore
• Uses Unreal Engine
• Designed for real-world movement
RE-DIRECTED WALKING
VESTIBULAR STIMULATION
SCANNING
WALK AROUND 360 VIDEO
HYPE VR
Demoed by Intel at CES 2017
SHARED DATASETS
• Think: Google Maps
• Tracking Matrices
• Only updating that which has changed
• Confirming Data from multiple sources
BACKPACK VR
• Allowed Freedom of Movement
• Great with Redirected Walking
• NOT Good for Athletics: Jumping
• Made Layups, Running, Jumping Possible
• Wireless non-backpack solution determined
VR AVATAR DEVELOPMENT
MY ORIGINAL VR AVATAR TEST
Kyle the Unity Robot(Just Head – No IK) - Oculus DK2 - May 2015
KYLE PLAYING BASKETBALL
Basketball Avatar Test – June 2016 (Steam VR IK Rigging)
KYLE JELLIES
Jelly Avatar Test – June 2016 (Unity IK Rigging)
THE UNCANNY VALLEY
Getting Away From The Robot
SCANNED-IN PEOPLE
Scanned Model from Sketchfab – Auto-rigged w/Mixamo (Unity IK Rigging)
HOVER JUNKERS IK
Using “Final IK” Rigging
SCANNED AVATAR + LEAP MOTION + FOOT CONTROLLERS
1st Foot Rigging - August 2016
FULL BODY VR
• Running, Jumping in Real Environments
• Interacting with real tracked objects
ENTER THE FEET
STRAP/CLIP SHOE EXPERIMENTS
OVERSHOE PUCK SOLUTIONS
SOCCER DEMO
ANOTHER MIXED REALITY DEMO
SOCIAL VR
MY VRML DAYS
ALTSPACE VR
BIGSCREEN VR
Hayden from (ConVRge)
VR CHAT
OCULUS SOCIAL
TENCENT VR HMD
Integration of WeChat – VR Social
V TIME
REC ROOM
PROJECT SANSAR
Linden Labs (Second Life)
HIGH FIDELITY
Philip Rosedale on stage Above
AR: EYE TRACKING
EYE TRACKING HMDS
SMI, Fove, Qualcomm
MAGIC LEAP
AR Lightfield
AVEGANT
First Standalone Headset to do Lightfield Synthesis
LOCATION BASED VR
Room Scale àHouse Scale à Warehouse Scale àArena Scale à MR Ubiquity ~5 years
THE VOID
• Smell of Burnt Marshmallows
• Misting Effect for Sliming
• Vibrations in Haptic Vest and Gun
• Clever Use of Redirected Walking/Elevators
• Multiplayer
ZERO LATENCY
• LaserTag-Type Arena Focus
• System Limited To open warehouse setup
• PS3 Cameras and Move Controller Tracking
UNIVERSAL ORLANDO HAUNTED HOUSE
• Similar to Void with tracking
• Real Actors Involved
• Mixed Experience – Putting headset on as part of experience
IMAX VR LOS ANGELES
• Vive + Star VR Experiences
• Custom Star Trek, DC SuperHero, John Wick experiences
• Buy tickets to “rides”
END OF 2018 HARDWARE ANTE
BY 2018 VR/AR HMDs will need to have: • Wireless (Tracking and Power) • Mobile, not confined to a space • Scanning in Real-Time & Sharing Scan Data • Eye + Facial Gesture Tracking • Predictive Interface • 180+ Degrees FOV + 8K Resolution • Small Form Factor (Thick Pair of Sunglasses)
THE STANDALONE HMDS OF 2018
HTC, Lenovo, Intel, Samsung, Asus, Qualcomm…
Apple?…Amazon?…Sony?...Avegant?...Magic Leap?
BEYOND THE BASELINE
THANK YOU
Questions?
Chris Wren [email protected] [email protected] [email protected]
game.gmu.edu www.wrenar.com
www.notiontheory.com
APPENDIX A
Sports Examples
BASKETBALL
SOCCER
APPENDIX B
Augmented Reality HMDS and USES
AUGMENTED REALITY
Recognition
WINDOWS HOLOGRAPHIC
MS not even talking about VR anymore - only Mixed Reality
INTEL PROJECY ALLOY
Intel tech is similar to Hololens - Real Sense
AR HMDS
Meta 2
AR HMDS
Hololens
AR HMDS
Daqri
AR HMDS
ODG
AR HMDS
Epson
AR HMDS
Atheer
AR HMDS
Occipital Bridge
AR HMDS
Vuzix
APPLE’S SECRET EFFORT
“The next big thing, like the cellphone” - Tim Cook
SAMSUNG AR
Monitorless
AR: FEATURES
● Recognition ● Using Real World Data To Drive Graphical Experiences ● Integrating Graphics into Real World Environments
LYTRO
Capturing & Processing Ultra HD lightfield video
AR: HAND GESTURES
Ability to See and Distinguish Hands and Gestures
AR: LEGO
LEGO
AR: BOOKS & MAGAZINES
AR: SHOPPING
ALIBABA BUY+