highly immersive telepresence research brief
TRANSCRIPT
www.HumanProductivityLab.com November 2011
Organizations are spending hundreds of
thousands of dollars per room for highly
immersive telepresence conferencing
environments.
What are the keys to creating highly im-
mersive telepresence environments?
What is the ROI for end-users? What is
the future of highly immersive confer-
encing?
By Howard Lichtman
This Human Productivity Lab Research Brief has been sponsored by Array Telepresence
Next
3. Background and history
4. Return on investment
4. What makes a Telepresence environment highly immersive
5. Most important components of creating immersive environments
5. Wide horizontal seamless displays
6. Absence of visible cameras
7. Absence of screen bezels
7. Stand-Up environment
8. Eye contact and eye lines
9. Augmented reality
9. High definition video with high frame rate
10. Frame rate
10. Life sized images
10. Engineered environments
10. Lighting
11. Audio
11. Future of highly immersive environments
12. Interim solutions
13. SurroundPresence
14. About the Author & Human Productivity Lab
Prev Next Table of Contents
November 2011 p.2
Contents Table
Of
Human Productivity Lab Research Briefs provide insight into telepresence and visual collaboration technologies for both business
line managers and IT professionals. Our principals, analysts, and Board of Advisors have real world experience in designing,
implementing, and managing telepresence and visual collaboration solutions in organi-zations ranging from Fortune 100 to Small
and Medium Businesses
The Human Productivity Lab works with both organizations looking to implement telepresence and visual collaboration and vendors looking to improve their offerings
and communicate complicated technologies to sophisticated audiences.
Our flagship publications the Telepresense Options website, the monthly Telepresence Options Telegraph, and
the Bi-Yearly Telepresence Options Mag-azine represent the largest identifiable au-
dience in the world interested in telepresence and visual collaboration tech-nologies.
You can follow Telepresence Options on Facebook, Twittter, RSS, and get our
articles emailed directly. Our industry as-sociation on LinkedIn, Telepresence Indus-try Professionals, is now over 2475+
telepresence industry pros and our German language group Telepresence Industry Pro-fessionals on Xing is over 260+
Background and History
My short definition of telepresence conferencing is “Visual collaboration solutions that address the human factors of participants and attempt to replicate, as closely as possible, an in-person experience”. Highly Immersive Telepresence is a sub-set of telepresence.
The father of immersive telepresence conferencing is TeleSuite (now the Polycom RPX) developer Herold Williams, the first visual collabora-tion architect to significantly address the human factors of the visual experience, hide the technology and integrate it into a format that would create immersion. The Result: usage went through the roof. While traditional videoconferencing systems averaged 5–15 hours per endpoint per month, TeleSuite systems were averaging 60, 70, 80, even 100+ hours per system per month! By focusing on the human fac-tors of participants and creating a highly immersive experience, Herold had tremendously improved the end users acceptance and prefer-ence over what could be achieved by a traditional, observant videoconferencing experience alone.
An early TeleSuite Enterprise 408 Telepresence Environment Circa 2004. Herold Williams sits center, to the right of Scott Allen, now CEO of telepresence managed service provider Iformata Communication
Early adopters included AOL, Cigna, 3COM, PriceWaterhouseCoopers, GlaxoSmithKline and Capital One, who proved that enterprises would pay hundreds of thousands of dollars for visual collaboration solutions that met their needs.
Polycom bought the TeleSuite intellectual property and manufacturing capabilities for over $50 million in 2007. Renamed the Polycom RPX, the offering has gone on to be one of the most successful visual collaboration solutions in history. Hundreds of Polycom RPX sys-tems are deployed globally, ranging from $299,999 for a four-seat environment to over $700,000 for a 28-seat environment.
Prev Next Table of Contents
November 2011 p.3
The Return on Investment
By greatly improving end-user acceptance, highly immersive telepresence dramatically increases usage, ROI and customer satisfaction over traditional videoconferencing. In surveys, employees consistently say they prefer the more humanistic, natural experience that highly immersive environments provide over traditional videoconferencing. They like the life-size body language and non-verbal cues that come across so clearly from remote participants, and how the comfortable environments let their meetings go longer without fatigue. Remote participants are represented more faithfully in the local space and are included in discussions as equal participants. They use the systems more willingly, without mandates. As a result, they travel less and produce more, giving organizations a time-to-market advantage from accelerated decision making, faster merger and acquisition, more flexible business models, and improved personal productivity of individual executives that can be more effectively leveraged around the world at the speed of light.
What Makes a Telepresence Environment Highly Immersive?
Anyone who has ever fallen in love on a first date can understand immersion. When immersion happens between two people, they connect so well they block out all other stimuli in their environments. This isn’t to say that executives need to fall in love to improve business communications, but they’ll produce better work in more immersive environments.
When the brain isn’t distracted by the Medium (the visible screen, the obvious camera, low-quality audio, space, etc.) it’s freed up to focus on the Message (what’s being said, body language and social cues). Immersive environments produce superior end-user ac-ceptance, compelling participants to stay longer without fatigue.
“In January 2008 I authored an article in the World Commerce Review for the IMCCA where we settled upon a specific definition for Telepresence: Telepresence represents the use of a number of technologies, aesthetics and acoustics that together allow a person or people in one location to meet and collaborate with a person or people in another location (or locations) where the experience simulates all people being in the same location. Implied in this experience is the understanding that the technologies, aesthetics and acoustics involved in the simulation are, or should be, practically invisible to the users.
By this strict definition, the only truly immersive products on the market today are Polycom’s RPX system and DVE’s TeleImmerison room. I understand many will debate the selection of only these two systems, but in my opinion one can’t call it an immersive experi-ence if you have mullions between displays that take up approximately two inches, have visible cameras the size of a breadbox, or most importantly don’t allow your face to be seen when you stand-up to make a point. All of these qualities would break the illusion of natural communications between people in the same room and therefore can’t be telepresence by the strict definition.”
- David Danto, Director of Emerging Technologies at the IMCCA & Principal Consultant for the collaboration, multimedia, video and AV disciplines at Dimension Data and former Vice President of Video and Telepresence Engineering at JPMorgan Chase where he built the largest commercial Cisco TelePresence ecosystem in the world outside of Cisco itself.
Quoted from the article “What is Telepresence?” which can be found at IMCCA.org
In telepresence and visual collaboration immersion is best thought of as a continuum where the graduations are not noticeably differ-ent from their adjacencies, although the ends or extremes are very different from each other. The more elements you address, the
greater the immersion you achieve.
Prev Next Table of Contents
November 2011 p.4
The most important components of creating immersive
environments include the following:
Wide horizontal seamless displays to address human’s wide horizontal field of view and peripheral vision
Peripheral vision is the part of vision that occurs outside the center of gaze (or foveal vision), providing the brain a sense of context and space in support of foveal vision. Human beings have fairly narrow vertical field of view, about 135 degrees, and a much wider
horizontal field of view of about 200 degrees.
Prev Next Table of Contents
November 2011 p.5
Visualization of the vertical vs. horizontal field of view and visual overview of the eye.
Absence of visible cameras
Another key aspect of immersion is hiding or removing cameras, screen bezels, and electronics that create a sense of artificiality in the envi-ronment. A visible camera leads to a phenomenon known as the “Documentarian’s Curse,” the tendency of people to behave differently on camera. Most people are familiar with this experience from family events where friends and relatives behave differently when being record-
ed with a video camera. In visual collaboration environments, participants may focus on how they look on camera or be less open as the brain registers the potential that the conference could be recorded. A camera out in the open also makes things seem artificial.
Solutions from Digital Video Enterprises (DVE) and TelePresence Tech hide the camera behind a piece of silvered glass called a beam splitter at eye-level. The Polycom RPX hides the camera behind the screen at eye-level.
Using a wide horizontal seamless display, telepresence architects can better immerse participants in the remote scene by duplicat-ing the usual and expected visual cues of peripheral vision.
The Polycom RPX 400 Series uses a 16' x 4' wide format display to supplement participants' peripheral vision and a hidden eye-level camera
Prev Table of Contents Next
November 2011 p.6
The author standing in a DVE Immersion Room with a 120-inch seam-less beam splitter and a hidden eye-level camera behind it.
Stand-Up Environment
Nothing destroys an immersive experience faster than head-less participants. Traditional videoconferencing rooms (and many telepresence group systems) can’t capture participants standing to enter, leave, stretch, make a point, or use a whiteboard.
Prev Next Table of Contents
November 2011 p.7
Absence of Screen Bezels
Human beings have been socialized to interact with visible screens in a very particular way since birth. Visible screen bezels around flat-panel dis-plays reinforce an observant experience. Dr. Ste-ve McNelley, the Co-CEO of DVE, is a clinical psy-chologist who did his doctoral research on vide-oconferencing. He calls the visible bezel, “the chief psychological cue that you are looking at a television screen.”
DVE’s telepresence solutions use seamless beam splitters to eliminate the visible bezel. Polycom’s RPX uses a rear projection screen.
The Digital Video Enterprises Huddle 70 eliminates the bezel, hides the camera
at eye-level and replicates architectural cues where possible.
Note the placement of the plant in both locations.
Eye contact and excellent eye lines
Eye-contact is chief among the body’s non-verbal cues. From infancy, we are biologically drawn to the gaze of our parents, establish-ing a preference for personal communication that continues throughout life. Eye contact between humans is physiologically powerful, eliciting changes in blood pressure and heart rate and increasing brain activity. The information transmitted through eye contact is
rich and varied:
· Eye gaze provides many communication fundamentals, including feedback, conversational regulation (turn taking) and
the expressions that punctuate emotion.
· Mutual eye gaze has been described by psychologists as “the key to the awareness of the thoughts of another.” People
with strong eye contact are perceived to be more honest, attractive and successful. Conversely, psychologists call people
with poor eye contact “gaze-avoidant personalities,” rated less favorably in the eyes of others.
Addressing Eye-Contact in Highly Immersive Environments
While beam-splitter solutions from DVE and TelePresence Tech can almost perfectly replicate eye contact between single participants
sitting directly in front of the display/hidden eye-level camera, it’s impossible to maintain truly perfect eye contact in multi-party con-ferences. However, considering that eye contact is made up of the vertical Y component and the horizontal X component, optimal
camera placement can go a long way in promoting eye contact. The most annoying aspect of eye contact is the mismatch of the Y component. Placing the camera at eye level behind a screen or beam-splitter display will significantly repose the gaze-angle differen-tial in the vertical Y component. But the horizontal X component is literally impossible to achieve. Some patents propose using multi-
ple cameras to capture different angles of the head and then averaging them so that the person appears to look straight ahead. This does not appear effective. You want to see people turn their heads to address someone in their room — our heads are perfect panning
mechanisms already, so the X component is much less important than the Y.
Significant gaze-angle differential using standard cameras mounted over flat panel
displays makes participants appear to look down.
Prev Next Table of Contents
Vertical “Y” gaze angle
November 2011 p.8
The TelePresence Tech TPT 4000 layers virtual images of remote conferees into the
physical space of the local environment while hiding the camera at eye-level.
Video conferencing resolutions
Augmented Reality
Another powerful technique to create a sense of immersion is augmented reality, which layers virtual images into a physical space to create the illusion that participants are in the same physical space. Digital Video Enterprises and TelePresence Tech use beam-splitter displays and an extreme low reflectance black velour background to achieve this effect. The black background turns invisible as it ab-sorbs light. With no visible background as a frame of reference, the remote participants appear as volumetric images layered into physical space with the right proportions.
High-Definition Video with a High Frame Rate
High definition video is classified as video with a screen resolution over 1280x720 pixels at a 16:9 ratio (typically referred to in industry parlance as 720p). The highest screen res-olutions offered in commercial telepresence and videoconferencing is 1920x1080 pixels at a 16:9 ratio (or 1080p). The higher the camera, codec, and screen resolution, the better the realism of the remote scene, especially with respect to facial features and minute nu-ances of individual expression. The larger the screen, the more important higher resolu-tions become in order to faithfully scale the image over a larger area. The videoconfer-encing industry’s move to high-definition images is one of the leading reasons visual col-laboration has caught on over the past five years.
Prev Next Table of Contents
November 2011 p.9
Frame rate
Frame rate is the frequency which a videoconferencing system produces unique consecutive images known as frames. The higher the frame rate, the smoother and more realistic the motion. The current standard for most videoconferencing systems is 30 frames per second (fps), though a significant percentage of newer systems are capable of 60fps.
Life-size Images of Remote Participants
Life-size images does not have to mean full human scale. In a typical conference room, we sit about five to six feet apart, which es-tablishes our perception of what is life size. In most immersive telepresence environments, we sit about eight to 10 feet from the screen plane. Accordingly, “life size” can be defined as 80 to 85 percent of human scale to maintain a sense of realism.
Engineered environments, including architectural elements, colors, furniture, etc.
Engineered environments address many of the subtle nuances that improve the quality of capturing video and audio including room color, lighting, and acoustics. Most environments precisely position participants so they are ideally captured for effective display on the other side. People feel more comfortable occupying a space that maintains attributes of a conference room environment.
Lighting Lighting has and will always be critical to creating immersive environments even with advances in sensor and camera technology. Low lighting will always add noise to the image—too much will add glare and washout parts of the scene. Direct overhead light will cast a shadow over the brow, darkening the eyes. Proper lighting includes key, fill, and back lighting. A balance of all three is critical with a subtlety of an office environment so it doesn’t look and feel like a television studio.
In this picture of a TeleSuite circa 2004, you can see all the elements of an engineered environ-ment: participants are properly lit and precisely
positioned for video capture, the furniture and ar-chitectural elements are mirrored on both sides, including attaching the table to the screen to cre-
ate the illusion that both sides share the same table and physical space.
Prev Next Table of Contents
November 2011 p.10
Audio Accurate voice capture is the most important aspect of telepresence, yet it often gets the least attention. Reverberation is the real kill-er. You can have the greatest audio compression and algorithms, but a lousy voice capture and poor replication are hard to overcome.
Microphones – Much of the problem boils down to microphone placement — where is the microphone in relation to the source voice. Lapel microphones achieve the best possible quality. Using today’s technology requires “micing up,” which has problems with naturalness, battery life, capacity and recharge. Steering microphone arrays are in the works. Because immersive rooms can become pretty active, table microphones can be problematic for standing participants. That leaves multiple overhead place-ments as the best option. This requires a high-quality DSP mixer, preferably with a de-reverberation algorithm. Spatial Acoustics – Another aspect of highly immersive environments is faithfully replicating the direction from which sound is produced. If a remote participant on the left-hand side of the screen speaks, the sound should come from that direction to ef-fectively mimic an in-person experience. Acoustics – Good acoustics lower reverberation. Immersive environments use acoustical materials to absorb sound and keep it from reflecting off the ceiling, walls and floor.
The Future of Highly Immersive Environments
The “Holy Grail” of immersive telepresence is creating an environment with photo-realistic 3D representations of remote participants projected into a physical space. Ideally, these spaces would be mirrored so both local and remote participants would be able to sit to-gether and collaborate on the same information. A participant in one location would work on a shared whiteboard, his digital persona replicating the content on an identically positioned whiteboard in realtime at the other location(s).
This requires capturing the image of the remote participants in three dimensions, compressing that information, sending it across a network, decompressing and then projecting that representation as an ultra-bright, high resolution three-dimensional photo-realistic image that appears solid. Here’s what makes the procedure even more complicated: to have a natural, humanistic interaction with the projected remote partici-pant, you must align the local camera, capturing the image to provide eye contact or the approximation of eye-contact to satisfy the innate expectations that human beings have with respect to inter-personal communications. A system might achieve this effect with vast arrays of low-cost, high-quality cameras ringing the space and adjusting their capture based on predictively tracking the position of participant’s heads in real-time. Suffice it to say, the technology isn’t there yet.
Prev Next Table of Contents
November 2011 p.11
Interim Solutions
Until photo-realistic 3D avatars are a reality, Herold Williams is at it again… His company, Array Telepresence, has filed patents on a
number of new technologies that capture and process the images pre-compression and has designed the SurroundPresence
telepresence environment to optimize the effect.
SurroundPresence uses many of the techniques listed above: high-definition life-size images, wide format seamless screens, hidden
eye-line camera, stand up presentation and an engineered environment that addresses acoustics, positioning, and lighting among oth-
er factors. The big difference is Herold has found a way to deliver the experience at 1/3 the cost of competing highly immersive solu-
tions with an environment that only requires a 13’ x 19’ room — a fraction of the space required from competing highly immersive en-
vironments. Herold’s patent-pending optical and image pre-processing technology, Equal-i, can also improve the experience of any
visual collaboration session done in a room with an elongated table with or without the other aspects of the SurroundPresence envi-
ronment. This problem was succinctly summed up by Cisco’s Chuck Stucki at the Wainhouse Research 2010 Summit, highlighting the
challenge of a typical boardroom configuration:
“How can manufacturers and installers solve the problem of supplying high-quality video, where facial gestures are clearly visi-
ble, across a long boardroom table? Assuming that the design of boardrooms is not going to change, then the equipment serv-
ing it will have to. One solution is cameras with zoom facilities, but what is not without its problems – who will control the
zoom, for example, and what happens if two people speak at once?”
Howard S. Lichtman is a productivity-focused technology technologist, author, publisher and consultant with specialties in telepresence and visual collabo-
ration to improve organizational and personal productivity. He is the founder and president of the Human Productivity Lab, an independent consultancy and
Cisco TelePresence did a product placement of
its vision for the future in the 2009 movie G.I.
Joe: The Rise of Cobra, in which photorealistic
3D representations of remote participants were
projected into physical space.
Prev Next Table of Contents
November 2011 p.12
Prev Next Table of Contents
The SurroundPresence SP-8 environment is optimized for the Equal-i system to create a highly immersive meeting experience for up to eight participants. The environment incorporates all of the techniques for creating high immersion: wide-format screen that takes up significant peripheral vision, high-definition, life-size remote participants, hidden eye-level camera,
stand up presentation, and an engineered environment with optimum lighting, acoustics, and architecture. SurroundPresence is also unique in that it doubles as a conventional conference room (albeit with enhanced data collabora-
tion capabilities) and can be deployed at 1/3 the cost and a fraction of the space required from other highly immersive telepresence environments.
The Equal-i system brings the
farthest participants in a con-
ference with an elongated
board room table up close and
personal (without pan/tilt/
zoom cameras) and distributes
them across either a wide-
format seamless screen or a
standard flat panel monitor.
This can help improve the con-
ference experience of organi-
zations that have non-
negotiable space or furniture
constraints like $100,000 ma-
hogany board room tables.
Williams is currently looking
for a distribution partner to
help take his products to mar-
ket by the time he showcases
SurroundPresence and Equal-i
at InfoComm 2012.
November 2011 p.13
Howard Lichtman is the President of the Human Productivity Lab, a telepresence consultancy and research firm
that helps organizations design telepresence and visual collaboration strategies and deploy and future-proof invest-
ments. He is also the publisher of Telepresence Options, the #1 website on the Internet covering telepresence and
visual collaboration and editor of the Telepresence Options Telegraph and Telepresence Options Magazine, the
world's most widely read publication covering telepresence technologies.
Mr. Lichtman is also the author and/or co-author of The Telepresence Options 2011 Yearbook, The Inter-
Company Telepresence and Videoconferencing Handbook (2009), The Telepresence and Videoconferencing
Exchange Review (2010), Telepresence, Effective Visual Collaboration and the Future of Global Business at
the Speed of Light (2006), and Emerging Technologies for Teleconferencing and Telepresence (2005). He is
currently working on Telepresence Options 2011.
Mr. Lichtman is a frequent commentator on telepresence, videoconferencing and effective visual collaboration
and his writings on and analysis of the industry have been featured by US News and World Report, Telephony
Magazine, CXO Magazine, The Chicago Tribune, Reuters, Pro AV Magazine, Killer App Magazine, ABA Banking
Journal, Bank Systems and Technology Magazine and CFO Magazine among others.
About the Author
Prev Table of Contents
November 2011 p.14
About the Human Productivity Lab
The Human Productivity Lab works with both organizations looking to implement telepresence and vis-ual collaboration and vendors looking to improve their offerings and communicate complicated technol-ogies to sophisticated audiences.
Our flagship publications the Telepresence Options website, the monthly Telepresence Options Telegraph, and the Bi-Yearly Telepresence Options Magazine represent the largest identifiable au-
dience in the world interested in telepresence and visual collaboration technologies.
You can follow Telepresence Options on Facebook, Twittter, RSS, and get our articles emailed di-rectly. Our industry association on LinkedIn, Telepresence Industry Professionals, is now over 2475+ telepresence industry pros and our German language group Telepresence Industry Professionals on Xing is over 260+.