hololens inspiration session
Post on 23-Jan-2018
105 Views
Preview:
TRANSCRIPT
Even kennismaken...
• Alexander Meijers
• Architect @ ETTU
• HoloLens Evangelist
• Oprichter van MixUG
• Twitter: @ameijers
• Email: a.meijers@ettu.nl / alexander@appzinside.com
HoloLens | Apps | Azure SharePoint | Office 365 | Dev Functional |
Architecture Speaker | Blogger | 3 kids Fitness |
Airsoft
Agenda• About realities
• Microsoft HoloLens device
• Fields of application
• Overview
• Input controls
• Sound
• Spatial perception & understanding
• Other cool stuff
• Data & Cloud
• Design considerations
Generation of realistic images,
sounds and other sensations
that replicate a real environment
or create an imaginary setting
An immersive experience created
entirely from computer-generated
Content. Also similar to 360 degree video
Virtual Reality
Direct or indirect view of a physical,
real-world environment
whose elements are augmented
by computer-generated sensory input
such as sound, video, graphics or GPS data
An overlay of content on the real world where
that content is not anchored to or part of it
Augmented Reality
Merging of real and virtual worlds
to produce new environments
and visualizations where physical
and digital objects co-exist
and interact in real time.
Overlay of synthetic content on the real world
that is anchored to and interacts with the real world
Mixed Reality
Now and the future!
October 2016
HoloLens v1
October/ November 2017
Immersive headsets
2019HoloLens v3
History of HoloLens
• Codenamed Project HoloLens
• Chief inventor Alex Kipman• Official title is technical fellow
• He dreamed up Kinect in at the end of 2007 and set a vision which incorporated HoloLens
• It is the start of a transforming world
• In the new reality, sensors will be anywhere
• A visual computing platform controlled by speech and gesture
Satya Nadella - “The emergence of
the next computing interface”
Microsoft Hololens
• Windows 10 device
• First Holographic computer running on Windows 10
• Contains a CPU, GPU and HPU
• Human understanding• Gesture and voice input
• Gaze tracking
• Spatial sound
• Uses wi-fi to connect and has no wires attached
• Design for comfort
Sensors, optics and speakers
Environment
camera
High definition
lenses
Spatial sound
with
speakers
Depth
camera
Video
camera
Fields of application•Enables the user to visualize
surrounding which is not directly
visible
•Get visual insight of an object or
model
•Modelling of non-existing objects
visualization
•Enrich real-world environment
objects with information
•Show information as part of the
real-world on a wall or table
•Show additional information to
objects, persons or activities
Provisioning
of information
•Support through linked view, voice
and gestures
•Meetings around Holographic
objects
•Working together on projects
whereby visualization is an
important part
•Working together when distance is
involved
Working
together
•Experience learning
•Getting more realistic teaching by
moving 2D presentations in books
to 3D holographic models
Education
•Holographic media
•Enrich media with additional
holographic objects
Media &
entertainment
•Play games as part of your real-
world environment
•Game interaction with your walls,
ceiling and floor
Gaming
• A company needs to add additional
electrical wiring.
• Detailed wiring plans are projected
over a wall or ceiling
• Based on detailed plans shown
through HoloLens he can decide
where to add
Example of visualization
Detecting
electrical wiring
• Detailed information about the elevator through a dashboard
• A 3D representation of parts of the elevator to allow quickly find the problem
• Ability to contact others throughphone and mail for help
Example of visualization
Elevator
maintenance
• Getting better insights of BI data
through 3D models
• Extend the desktop with 2D/3D
information screens
Example of provisioning of
informationEnhance your BI
desktop
• Sharing the same view, scenarios
and data of a car simulation with
sensors
• Getting help from each other and
make it easier to explain on what
you are working on
Example of working together
Working
together on
car safety
• Getting a realistic 3D projection of the human body
• It allows easy layering of the body and mark different body parts
• By walking around the projection and viewing and marking body parts will help students to learn more quickly
Example of education
Learning
about the
human body
• Become part of different squares in Rome
• Rome is viewed as a 360 degreeview around you in all directions
• Images are enhanced wihholographic additions like the inside of the church and the fountain in the front.
Example of media and
entertainment…
• HoloLens allows us to play games
which become part of our real-world
environment
• Fragments is a CSI game in which
you, together with virtual people
solve crime delicts
• Virtual people use your room as if
there are actually there
Example of gaming
Mixed reality
during
gameplay
Gaze
• The direction of the
HoloLens pointing is called
the gaze
• Hits an object or part of
the environment
• Uses a Vector3 based value
to define its so called
“forward” direction
Gestures
• Recognize input by tracking the position of either or both hands
• Gesture frame
• Recognized input from hands• Bloom
• Press, hold and release
• Follow the hand (my own )
• Gestures• Hold
• Manipulation
• Navigation
Voice
• Use words or sentences to
control your environment
• English language only at the
moment
• Phrase recognition using
Language Understanding
Intelligent Service (LUIS)
Spatial Sound
• Simulates 3D sound using direction, distance and environmental simulations
• Above, below, behind, to the side, etc.
• Attach sound to holographic objects
• Works also when object is not in line of sight
• Used to draw attention
• Audio engine in HoloLens
• CPU and memory considerations
• 10-12 spatial sound voices
Spatial Sound
• Gaze mixing• Highlighting objects
• Play a sound on the object to get the users attention
• Audio Haptics• Reactive audio for touchless
interactions
• Play a sound when user selects an object or when his hands appear inside the gesture frame
• Immersion• Ambient sounds surrounding the user
• Support your scene with background sounds or music
Cortana
• Start with “Hey Cortana”
• Some thing you can say
• Move <app name> here
• Take a picture
• Start or stop recording
• Call <contact>
• How much battery do i have left?
• Restart or Shutdown
• Cortana can be turned on/off
Spatial perception
• Spatial perception and
mapping
• Rooms
• Different ways of detecting
and working with spatial
perception
• Meshes & planes
Spatial perception is the
ability to be aware of your
relationships with the
environment around you
and with yourself
Spatial perception
• Device scans your
environment and builds a
digital model in real time
• It allows HoloLens to see
different surfaces, walls
and ceiling
• Possible to simulate a
physical space
• E.g. Projecting a terain
over your floor
Pinning
• Device can project a hologram into a room and keep it locked in position
• Called pinnnig
• Objects are not moving relative to you but you can move around the object
• Allows you to view the object from any angle
Vuforia
• Allows you to develop holographic apps which recognize specific things in the environment and attach experiences to them• Recognizes specific images and
objects in the environment
• Connects holograms to targets
• Vuforia engine• Extend the capabilities of your
holographic app
• Targets can be stored on Device or in the Cloud
Sharing holographic experiences
• Two or more people are
experiencing the same
holographic objects
• Accomplished to anchor sharing
HoloLens device editions comparison
• Kiosk mode• Limiting running apps to enable
demo or showcase experiences
• MDM• Manage Multiple HoloLens devices
through solutions like Microsoft Intune
• Business Store Portal• Private enterprise store for your
companies apps
• Secure boot• Only boot software which is trusted
for your organization
• Windows Update for Business• Controlled OS updates to device
Data storage
• Only allowed to store files to a number of local folders
• Save or open from another App like OneDrive
• HoloLens does not support connecting external hard drives or SD Cards
• There is no File Explorer application
• You could use the Windows Device Portal
Cloud
• Device supports connectivity to a broad set of service
• Azure services> Cognitive services
> AI / Bots
> Custom services
• Office 365> Microsoft Graph
> SharePoint Online
• Other> Bing
> Custom
User perspective
• Ensure the application is comfortable to the user
• Avoid abrupt movements
• Allow them to see the world in the background
• Animation from down, left or right or fade in
• Move content slowly and smoothly toward the user
• Keep a minimal distance of 3 meters
Design patterns for
mixed reality
User perspective
• Design your application from
an user’s unique point of view
• Determine if the user moves
around
• User is sitting down or
standing
• The user is the camera!
• Let them control the
movement
Design patterns for
mixed reality
Holographic frame
• Use the center of the holographic frame for your holographic objects
• Help the user through guideness
• Arrows pointing to a specific location
• Light trails
• Pointers
• Spatial sound
• Voice prompts
• Make sure your holograms fit the frame
• They will feel natural Design patterns for
mixed reality
top related