Download - Sathu G Rajan Nov. 2006
-
8/9/2019 Sathu G Rajan Nov. 2006
1/26
Seminar Report
on
TELE - IMMERSION,
Submitted in partial fulfillment of th e requirements for the award of the degree of
Bachelor of Technologyy p t . .-. a. 2-
* ' .,in , - ir
- 7 ,
Computer Science and Engineering
by
SATHU G. RAJAN
b4
-C
November 2006
Department of Computer Science and EngineeringSree Narayana Gurukulam College of Engineering, Kolenchery
-
8/9/2019 Sathu G Rajan Nov. 2006
2/26
Tele - Immersion
ACKNOWLEDGEMENT
Firstly I would like to express my sincere gratitude to the Almighty for His
solemn presence throughout the seminar study.I would also like to exp ress my
special thanks to the Principal Prof.K. Rajendran for providing an opportunity
to undertake this seminar .I am deeply indebted to our seminar coordinator Mr.
Saini Jacob, Assistant Professor in the Department of Computer Science and
Engineering for providing me with valuable advice and guidance during the
course of the study.
I would like to extend my heartfelt gratitude to the Faculty of the Department
of Computer Science and Engineering for their constructive support and
cooperation at each and every juncture of the seminar study.
Finally T would like to express my gratitude to Sree Narayana Gurukulam
College of Engineering for providing me with all the required facilities
without which the seminar study would not have been possible.
Dept. of CSE SNGCE. K o l ~ r z I ~ t ~
-
8/9/2019 Sathu G Rajan Nov. 2006
3/26
TELE-I MMERSION
ABSTRACT
Tele-immersion is a technology to be implemented
together in a simulated environment to interact. Users will feel like they are actually
looking, talking, and meeting with each other face-to-face in the same room. This is
achieved using computers that recognize the presence and movements of individuals and
objects tracking those individuals and images and reconstructing them onto one stereo-
immersive surface. 3D reconstruction for tele-immersion is performed using stereo,
which means two or more cameras take rapid sequential shots of the same object,
continuously performing distance calculations, and projecting them into the computer-
simulated environment, as to replicate real-time movement.
Tele immersion is a technology that will be implemented
with Intemet2lt will enable users in different geographical locations to come together and
interact in a simulated holographic environment. Users will feel as if they are actually
1looking, talking and meeting with each other face to face in the same place, even though!they may be miles apart physically. In a tele immersive environment, computers!recognize the presence and movements of individuals as well as physical and virtual11I objects. They can then track these people and non-living objects, and project them in a
realistic way across many geographic locations.
It has varied applications and it will significantly affect the
educational, scientific and medical sectors.
Its main application is in video conferencing and it
takes video conferencing to the next level.
It is a dynamic concept, which will transform the way
human interact with each other and the world in general.
Dept-of CSE SNGCE,Kolenchery
-
8/9/2019 Sathu G Rajan Nov. 2006
4/26
TELE-IMMERSION
CONTENTS
Introduction ....1The H istory....What is tele- immersion? ....
Requirements of Tele immersion....
3D environment scanning ........econstruction in a holographic environm ent 6
Projective and display technologies....Tracking technologies ....7Moving sculptures ....
Audio technologies ....
Powerhl networking ....
Computational needs ....9Telecubicle .... 10Tele imm ersion studio.... 1
Applications....12
Challenges of Tele-Immersion.... 5
Solution....16About Internet2.... 16
....esktop Supercomputers 17....andwidth issues 17
....urrent developments 18Conclusion .... 0Future scope .... 1Bibliography .... 2
Dcpr.of CSE SKGCE.E;G!~~; :~~,
-
8/9/2019 Sathu G Rajan Nov. 2006
5/26
INTRODUCTIONI Tele-immersion, a new medium for human interaction
enabled by digital technologies, approximates the illusion that a user is in the sameI1 physical space as other people, even through the other participants might in fact beI
I .hundreds or thousands of miles away. It combines the display and interaction~
techniques of virtual reality with new vision technologies that transcend the traditional
limitations of a camera. Rather than merely observing people and their immediate
environment from one vantage point, tele- immersion stations convey them as "moving
sculptures,'' without favoring a single point of view. The result is that all theparticipants, however distant, can share and explore a life-size space.
I
Beyond improving on videoconferencing, tele-immersion~
was conceived as an ideal application for driving network-engineering research,
specifically for Tnternet2, the primary research consortium for advanced network
studies in the U.S. If a computer network can support tele-immersion, i t can probablyI
support any other application. This is because tele-immersion demands as little delay as~
possible from flows of information ( and as little inconsistency in delay ), in addition to'
the more common demands for very large and reliable flows.
Tele-immersion car? be of i~nmense use i n rnedict~l
industry and i t also finds its application i n the field of edi~cation
TH E HJSTORY
It was i11 1965 that, Ivan Sutherland, proposed the conceptI
of the 'Ultimate Display'. It described a graphics display that would allow the user to
experience a completely computer-rendered environment. 'The term Tele-immersion
\\as first used in October 1996 as the title of a workshop organized by EVL and'
sponsored by Advanced Network & Services, Inc. to bring together researchers in,
distributed computing, collaboration, VR, and networking. At this workshop. specific
attention was paid to the future needs of applications in the sciences. engineering- an d
education. In 1998 Abilene, a backbone research project was launched and no\\ senes
as the base for Internet-2 research. Tele-immersion is the application that \ \ i l l dr i \e
forward the research of Internet-2.
Dept.of CSE 1 sh(;c~-x :-- -_ - -.
-
8/9/2019 Sathu G Rajan Nov. 2006
6/26
TELE-IMMERSION
WHAT IS TELE- IMMERSION ?
I
!
Tele-immersion enables users at geographically distributed
sites to collaborate in real time in a shared, simulated, hybrid environment as if they wereI
in the same physical room.
It is the ultimate synthesis of media technologies:
J 3D environment scanning.
J Projective and display technologies.Tracking technologies.
J Audio technologies.
J Powerful networking.
The considerable requirements for tele-imm ersion system,
such as high bandwidth, low latency and low latency variation make it one of the most
challenging net applications. This application is therefore considered to be an ideal driverfor the research agendas of the Internet2 community.
Tele-immersion is that sense of shared presence with
distant individuals and their environments that feels substantially as if they were in one's
own local space. This kind of tele-immersion differs significantly from conventional
video teleconferencing in that the use's view of the remote environment changes
dynamically as he moves his head.
Dept.of CSE 2 SNC;CE.Kolencher>
-
8/9/2019 Sathu G Rajan Nov. 2006
7/26
TELE-IM MERSION
REQUIREMENTS OF TELE-IMMERSION
Tele-immersion is the ultimate synthesis of media
technologies. It needs the best out of every media technology. The requirements are
given below.
3D ENVIRONMENT SCANNING
For a better exploring of the environment a stereoscopic
view is required. For this, a mechanism for3D environment scanning method is to be
used. It is by using multiple cameras for producing two separate images for each ofeyes. By using polarized glasses we can separate each of the views and get a 3D view.
The key is that in tele-immersion, each participant must
have a personal view point of remote scenes-in fact, two of them, because each eye
must see from its own perspective to preserve a sense of depth. Furthermore,
participants should be free to move about, so each person's perspective will bei n
constant motion. Tele-imm ersion demands that each scene be sensedi n a manner that is
not biased toward any particular viewpoint (a camera, in contrast, is locked intoI portraying a scene from its own position). Each place, and the people and things in it,I
has to be sensed from all directions at once and conveyed as if it were an animated
three-dimensional sculpture. Each remote site receives information describing the
whole moving sculpture and renders viewpoints as needed locally. The scanning
process has to be accomplished fast enough to take place in real time at most within a
small fraction of a second.
The sculpture representing a person can then be updated
quickly enough to achieve the illusion of continuous motion. This illusion starts to
appear at about 12.5 frames per second (fps) but becomes robust at about 25 fps and
better still at faster rates.
Measuring the moving three-dimensional contours of the
inhabitants of a room and its other contents can be accomplished in a variety of ways.
In 1993, Henry Fuchs of the University of North Carolina at Chapel Hill had proposed
'one method, known a s the "sea o f cameras" approach, in w hich the viewpoints o f many
cameras are compared. In typical scenesi n a human environment, there will tend to be
Dept.of CSE 3 SNGC E.Kolencher!
-
8/9/2019 Sathu G Rajan Nov. 2006
8/26
TELE-IMMERSION
Ivisual features, such as a fold in a sweater, that are visible to more than one camera. By
comparing the angle at which these features are seen by different cameras, algorithmscan piece together a three- dimensional model of the scene.
This technique had been explored in non-real-time
'configurations, which later culminated in the "Virtualized Reality "demonstration at
Carnegie Mellon University, reported in 1995. That setup consisted of 51 inward-
looking cameras moun ted on a geodesic dom e. Because it was not a real- ime device,it could no t be used for tele-immersion.
Ruzena Bajcsy, head of GRASP ( General Robotics,Automation, Sensing and Perception) Laboratory at the University of Pennsylvania,
was intrigued by the idea of real-time seas of cameras. Starting in 1994 , small scale
,"puddles" of two or three cameras to gather real-world data for virtual- realityapplications w as introduced.
But a sea of cameras in itself isn't complete solution.
Suppose a sea of cameras is looking at a clean white wall. Because there are no surface
futures, the cameras have no information with which to build a sculptural model. Aperson can look at a white wall without being confused. Humans don't worry that a
wall might actually be a passage to an infinitely deep white chasm, because we don't
I rely on geometric cues alone- we also have a model of a room in our minds that can
'
rein in errant mental interpretations. Unfortunately, to today's digital cameras, a
person 's forehead or T-shirt can present the same challenge as a white wall, and
today's software isn't smart enough to undo the confusion that results.
Researchers at Chapel Hill came with a novel method that
has show n prom ise for overcoming this obstacle, called" imperceptible structured light
-' r ISL. Conventional light bulbs flicker 50 or60 times a second, fast enough for the
.flickering to be generally invisible to the human eye. Similarly, ISL appears to the
human eye as a continuous source of white light, like an ordinary light bulb, but in fact
it is filled with quickly changing patterns visible only to specialized, carefullyI synchronized cameras. These patterns fill in voids such as white wall with imposedI
features that allow a sea of cameras to complete the measurements. If imperceptible
structured light is not used, then there may be holes in reconstruction data that result
-
8/9/2019 Sathu G Rajan Nov. 2006
9/26
from occlutions, areas that aren't seen by enough cameras, or areas that don't provide
distinguishing surface features.To accomplish the simultaneous capture and display an
office of the fbture is envisioned where ceiling lights are controlled cameras and
"smart". projectors that are used to capture dynamic image-based models with
imperceptible structured light techniques, and to display high-resolution images on
designated display surfaces. By doing simultaneously on the designated display
surfaces, one can dynamically adjust or auto calibrate for geometric, intensity, and
1 resolution variations resulting from irregular or changing display surfaces, or
overlapped projector images.
Now the current approach to dynamic image-based
.modeling is to use an optimized structured light scheme that can capture per-pixel depth
and reflectance at interactive rates. The approach to rendering on the designated
(potentially irregular) display surface is to employ a two-pass projective texture scheme
to generate images that when projected onto the surfaces appear correct to a moving
head-tracked observer.
-
8/9/2019 Sathu G Rajan Nov. 2006
10/26
1 RECONSTRlJCTION IN A HOL OGR APH lC EN VIIIONMBNrl'1k1
The process of reconstruction of image occursi n
a/ holographic environment. At the translnitting end. the3d imagc scanned i s generatedusing two techniques. The reconstri~ction process is different fhr sharedtable and ic3d
I approach.
I Sharer1 Table App roachI
Here, the depth of the 3d image is calculated usirig 3d wireI
ffames. this technique uses various calncraviews and con~plexnlage analqsis algorithms
to calci~latche depth.~
1 Assulning that the geonletrical paralnetcrs o f the multi-I
Iview c apture device, the virtual gcene and thc virtual cam era are n ell fittcdt o each otl~er.
I .~t is cnsurcd that thc scenc is viewed in thc right pcrspectivc view, m e n wl~ilc. hanging
I the vien ing position.
r Ic3d(incornplete 3d) ApproachI
In this case, a colnmon teuturc surfact: is extracted from the
alailable camera vielvs and thc depth information is codcd in an associated disparit~
nidp. This representation can be encoded into a mpeg-3 vidcool~ jcc t .which is then
transmitted.
The decoded disparities are scaled according to the uscr's
3d viewpoint in the virtual scene, anda disparity-contt.olled prq jec tiot~ s carried out. 'I'l~c
36 perspective of the person cha nges Lvith the ~n ov eni en t f the virtual cam era.
In both the approaches, at the receiving end the entirely
composed 3d scene is rendered onto the 2d display of the terminal by using a virtual
a m e r a . the position of the virtual camera coincides with the current position of the
conferee's head. for this purpose the head position is permanently registered by a head
tracker and the virtual ca mera is moved w ith the head.
-
8/9/2019 Sathu G Rajan Nov. 2006
11/26
PROJECTIVE & DISPLAY TECHNOLOGIES
By using tele-immersion a user must feel that he is
immersed in the other person's world. For this, a projected view of the other user's
world is needed. For producing a projected view, big screen is needed. For better
projection, the screen must be c u ~ e dnd spe cial projection cameras are to be used.
TRACKING TECHNOLOGIES
It is great necessity that each of the objects in the
imm ersive environment be tracked so that we get a real world experience. This is done
by tracking the move ment of the user and ad justing the came ra accordingly.
Head & Hand tracking
The UNC and Utah sites collaborated on several joint
design-and-manufacture efforts, including the design and rapid production of a head-
tracker component (HiBall) (now used in the experimental UNC wide-area ceiling
.tracker). Precise, unencum bered tracking of a u ser's head and hands ov er a room sized
working area has been an elusive goal in modern technology and the weak link in mostvirtual reality systems. Currant commercial offerings based on magnetic technologies
perform poorly around such ubiquitous, magnetically noisy computer components as
CR Ts, while optical-based products have a very small working volume and illuminateda beacon.targets (LEDs). Lack of an effective tracker has crippled a host of augmentedI
reality applications in which the user's views of the local surroundings are augmented
by synthetic data (e.g., location of a tumorin the patient's brain or the removal path of
a part from w ithin a complicated piece of machinery).
MOVING SCULPTURES
It combines the display and interaction techniqu es o f virtual
reality with new vision technologies that transcend the traditional limitations of a
camera. Rather than merely observing people and their immediate environment from
one vantage point, tele-immersion stations convey them as" moving sculptures",
without favoring a single point of view. The result is that all the participants, however
distant,'can share and explore a life size space.
Dept.of CSE 7 SNC;CE.Kolencher!
-
8/9/2019 Sathu G Rajan Nov. 2006
12/26
AUDIO TECHNOLOGIES
For true immersive effect the audio system has to beextended to another dimension, i.e., a 3 D sound capturing and reproduction method has
to be used. This is necessary to track each sound source's relative position.
POWERFUL NETWORKING
The considerable requirements for tele-immersion system,
such as high bandwidth, low latency and low variation (jitter), make it one of the mostI
'Ichallenging net applications.
I P Internet 2 -the driving force behind Tele-immersion'
It is the next generation internet. Tele-immersion wasI conceived as ideal application for driving network engineering research. Internet2 is a
consortium consisting of the US government, industries and around 20 0 universities
: and colleges.
It has high bandwidth and speed. It enables revolutionary
internet applications.
i-. Need for speed:
If a computer network can support tele-immersion, it can
probably support any other application. This is because tele-immersion demands as
little delay as possible from flows of information (and as little inconsistency in delay),
in addition to the more common demands for very large and reliable flows.
-train to Network :
In tele-immersion not only participant's motion but also the
entire surface of each participant had to sent. So it strained a network very strongly.
Our demand for bandwidth varies with the scene and application; a more complex
scene requires more bandwidth.
i Network backbone:
A backbone is a network within a network that lets information//t
Itravel over exceptionally powerful, widely shared connections to go long distances
/ more quickly. Each of earlier net played a part in inspiring new applications for the
Dspt-of CSE
-
8/9/2019 Sathu G Rajan Nov. 2006
13/26
TELE-IMMERSION
Internet, such as the World Wide Web. Another backbone research project, called
Abilene, began in 1998, and it was to serve a university consortium called Internet2.
Abilene now reache s more than 170 Am erican research
universities. If the only goal o f Internet2 were to offer a high level o f bandwidth (that
is, a large number of bits per second), then the mere existence of Abilene and related
resources be sufficient. But Internet2 research targeted additional goals, among them
the development of new protocols for handling applications that demand very high
.bandwidth and very low, controlled latencies (delays imposed by processing signals
and route).
COMPUTATIONAL NEEDS
Beyon d the scene-capture system, the principal com ponents
of a tele-immersion setup are the computers, the network services, the display and
interaction devices. Literally dozens of processors are currently needed at each site to
keep up with the dem ands of tele-immersion. R oughly speaking, a cluster of eight two-
gigahertz Pentium processors with shared memory should be able to process a triowithin a sea of cameras in approximately real time. Such processor clusters should be
available in the later year.
Bandwidth is a crucial concern. Our demand for bandwidth
varies with the scene and application; a more complex scene requires more bandwidth.
We can assume that much of the scene, particularly the background walls and such, is
unchanging and does not need to be resent with each frame.
Conveying a single person at a desk, without thesurrounding room, at a slow frame rate of about two frames per second has proved to
require around 20 megabits per second but with up to 80-megabit-per-second peaks.
With time, however, that number will fall as better compression techniques become
established. Each site must receive the streams from all the others, soin a three-way
conversation the bandw idth requirement m ust be m ultiplied accordingly.
Dept.of CSE 9 SN(;CE.Kolenchery
-
8/9/2019 Sathu G Rajan Nov. 2006
14/26
'IELE-IMMERSION
TELE CUBICLE
The tele-cubiclc represents the next gerieration iriimcrsive
interface. It can also be seen as a subset of all possible imrricrsive interfaces. An oftice
appears as one quadrant in a larger shared virtual office space. 'l'he canvases onto which
the imagery can be displayed are a stero-immersive desk surface as \veil as at least two
stereo. Such a system represents the unitication of Virtual Reality and vidcoconfcrencing.
and it provides an opportunity for the full integration of VR into the workflow. Physical
and virti~al nvironments appear united for both input and display. 'l'liis combination, we
bclievc, offers a new paradigm for human communications and collaboration .
2 obliquefmnt stereo _ A
Tele cubicle which consists of two w-all surfaces and a desk surfacc wl~icli prqjects 3 D
images.
It consists of a stereo irnniersivc desk surface and two
btereo-imrncrsive wall surfaces. These three display surfaces join to form a corner desk
mit . The \.\ialls appear as windows to the otlier users' environment while thc desks join
3cpt.of CSE
-
8/9/2019 Sathu G Rajan Nov. 2006
15/26
together to form a virtual conference tablc in the centre. This will a l l o ~ hc realistic
inclusion of tele-immersion into the work environment. as it \vill takc up the usualamount of desk space.
'Today's tcle-immersion conibincs the superior displa) of
CAVE and ImmersaDesk display systcnis ~ i t h dvaliced ~letuorl< apabilities .The
CAVE (Cave Automatic Virtual Environment) is a ~nulti-display virti~al eality device
comprised of three projection screcn,two "walls" and a *'floor" ~\hicI=i projects rcal-time
images in response to the user's eye and/or head movements. I'o ensure thc qualit) of the
picture and timeliness of response, the CAVE must be controllcd by a po~zerfi~l achine.
or supcrcompuier. I n some cases, CAVE proccssing units contain lip to sixteen
processors .CAVE is an example of 3D disply system which i~nplements ' l'elecubicle ".
TELE IMME RSION STIJDIO
It's a room uith all array of vidco cameras to provide
multiple viewpoints and a group of computers to process tlie digitized images. Tlie
people. who appeared as 3-D images. cberc tracked wit11 an array of eight ordinar) video
cameras whilc three other video cameras captured real liglit patterns in room lo calculate
distances. 'This enables tlie proper depth to be recreated on the 3 - D space.
In a remote location, a vicwcr sits In front of a screen.
\\earin:: polarized glasses like those used for 3-T) movics. The screen shows uhat or c ~ h o
's i n front of the arra) of video cameras. If the obserker rnovzd his or her head to the
'zfr. he/slie could see the corresponding images rliat \\auld bc seal if she were actually in
:5e rooni u ith the person on tlie screen.
I12~1 .~ffSE 1 1 SN(iC't:.Kolenchery
-
8/9/2019 Sathu G Rajan Nov. 2006
16/26
APPLICATIONS
I) collaborative Engineering WorksTeams of engineers might collaborate at great distances on
computerized designs for new machines that can be tinkered with as through they were
real models on a shared workbench. Archaeologists from around the world might
experience being present during a crucial dig. Rarefied experts in building inspection or
engine repair might be able to visit locations without losing time to air travel.
2) Video Conferencing
Although few would claim that tele-immersion will be
absolutely as good as "being there" in the near term, it might be good enough for
business meetings, professional consultations, training sessions, trade show exhibits
and the like. Business travel might be replaced to a significant degree by tele-
immersion in I0 years. This is not only because tele-immersion will become better and
cheaper but because air travel will face limits to growth because of safety, land use and
environmental concerns.
3) Irnrnersive Electronic Book
Applications of tele-immersion will include immersive
.electronic books that in effect blend a "time machine" with 3D hypermedia, to add an
additional important dimension, that of being able to record experiences in witch a
viewer, immersed in the 3D reconstruction, can literally walk through the scene orj1 move backward and forward in time. While there are many potential application areasI'
for such novel technologies (e.g., design and virtual prototyping, maintenance and
repair, paleontological and archaeological reconstruction), the focus here will be on a
socially important and technologically challenging driving application, teaching
surgical management of difficult, potentially lethal, injuries.
4) Collaborative mechanical CAD
A group of designers will be able to collaborate from
remote sites in an interactive design process. They will be able to manipulate a virtual
'model starting from the conceptual design, review and discuss the design at each stage,
perform desired evaluation and simulation, and even finish off the cycle with the
production of the concrete part on the milling machines.
Dept-of CSE 13 SN(;GE.Kolenchery
-
8/9/2019 Sathu G Rajan Nov. 2006
17/26
5) Entertainment
Tele-inimersive holographic environments have a number
of applications. Imagine a video game free of joysticks. i n which you become a
participant in the game, fighting rnonsters or scoring touchdowns.
6) Live chat
Instead oftraveling hundreds of miles to visit your relatives
during the holidays, you can simply call thein up and join them in a shared holographic
room.
7) Medicine
Tele immersion can be of immense use to the field of
medicine. The way medicine is taught and practiced has always been very hands-on. I t is
impossible to treat a patient over the phone or give i~lstructions for a tilmour to be
removed without physically being there. With the help of tele-immersion. 3 D surgical
learning for virtual operations is now in place and, i u the future. the hope is to he able to
carry out real surgery on real patients. A geographically distanced surgeon could be tele-immersed into an operation theatre to perform an operation. This could potentially be
lifesaving if the patient is in need of' special care (either a technique or a piece of
equipment), which is not available at that particular location.Tele-immersion 'will give
surgeons.the ability to superimpose anato~nic mages right on their patients \while they are
being operated on'.
8) Uses in education
I11 education, tele-immersion can be used to bring together
students at remote sites in a single environment. Kelationships among educational
institutions could improve tremendously in the future with the use of tele-immersion.
Already, the academic world is sharing information on research and development to
I better the end results. Doctors and soldiers could use tele-immersion to train in ai
simulated environment. This will be a distinct advantage in surgical training. While it
will not replace the hands-on training, this technology will give surgeons a chance toP learn complex situations before they treat their patients. With teleinlmersion in schools.
Dept.of CSE 13 SNGCE.Kolencher1
-
8/9/2019 Sathu G Rajan Nov. 2006
18/26
students could have access to data or control a telescope from a reniote location. or meet
with students from other countries by projecting themsclvcs into a foreign space.
Internet2 will provide access to digital librarics and virtual labs, opening up the lil!es of
communication for students. Tele-immersion will bring to the111 places, equipnient and
situations earlier not available, helping tl-iem expcrience what they could have on11
watched. read or heard about earlier.
9) Future office
In years to come, instead of asking for a colleagi~e on the
phone you will find it easier to instruct your computer to find him or her. Once yo11 do
that, you'll probably see a flicker on one ol' your office walls arid lind that your
colleague, who's physically present in another city, is sitting right across you as if' he or
she is right there. The person at the other end will experience the sanlc inimersivc
connection. With tele-immersion bringing t ~ o r more distant people together in a single.
si~nulatcd ffice setting, business travel will become quite redundant.
g Other applications
Building inspectors could tour structures without leaving
their deslts. Automobile desigmers from different continents coi~ld meet to develop the
next generation of vehicles. In the entertainment industry, ballroo~n dancers could train
together from separate physical spaces. lnstead of cornmutirig to ~ o r k or a board
meeting, businesspersons could attend i t by projecting Lhemselves into the conferelice
room. The list of' applications is large atid varied. and one thing is crystal clear thistechnology \ w i l l signiiicantly affect the educational, scientific and nledical sectors.
-
8/9/2019 Sathu G Rajan Nov. 2006
19/26
CHALLENGES OF TELE-IMMERSION
Tele-immersioii has emerged as a high-end driver for tlle Quality of Servicc (QoS),
bandwidth. and reservation efforts envisioned by the NGI and Internet2 leadership.
From a networking perspective, tele-immcrsion is a very challenging technology for
several reasons.
The networks must be in place and tuned to support high-bandwidth
applications.
Low latency, needed for 2-way collaboration, is hard to specify and guarantee
given current middleware.
The speed of light in fiber itself is a limiting factor over transcontinental and
transoceanic distances.
Multicast, unicast, reliable and unreliable data transmissions (called "flows")
need to be provided for and managed by the networks and the operating systems
of supercomputer-class workstations.
. Real-time considerations for video and audio reconstruction ("streaming") are
I critical to achieving the feel of telepresence, whether synchronous or recordedI and played backs1 The computers, too, are bandwidth limited with regard to handling very large11;11
data for collaboration~
Simulation and data mining are open-ended in computational and bandwidth
needs-there will never be quite enough computing and bitslsecond to fullyanalyze, and simulate reality for scientific purposes.
In Layman's language the realization of tele-imniersion is impossible today due
to
1. The non-availability of high speed networks
2. The non-availability of supercomputers
3. Large network bandwidth requirement reasons
-
8/9/2019 Sathu G Rajan Nov. 2006
20/26
SOLUTION
The ilrst two basic problc~ns can be overcome Iwhen Internet-2 will come into picture later and third problem can be overcorue by the Ifast development of image compression techniques.
ABOUT INTERNE T3
Internet2 is not a separate physical ne~work and will not replace the current IInternet. It is not for profit consortium consisting of 200 I J S universities. 1Industries and is directly under the control of US govt..
Internet2 is for developing and deploying advanced network applica~ions and Itechnology, accelerating the creation of tomorroc\rfs nternet.
Jnternet2 enables completely new applications such as digital libraries. cirtual
laboratories, distance-independent learning and tele-immersion.
A key goal of this effort is to accelerate the diffusion of advanced Internet
technology, in particular into the commercial sector.
Internet2 is the second generation internet, helps to develop
advanced network applications and technologies for research and higher education, by
recreating the partnerships among academia, industry, and government.
Another backbone research project, called Abilene, begun
in 1998, and it was to serve Internet2. Abilene now reaches more than 170 American
research universities. Internet2 research targeted in the development of new protocols for
handling applications that demand very high bandwidth and very low, controlled
Iatencies (delay is reduced by processing signals along their travel through the network).
We need a po\verful network with high speed and high
bandwidth to transfer the large a~nounts of data that tele-immersion will
psoduce.lnternet2 will replace the current Internet infrastructure. l'his new network will
hale a higher bandwidth and speeds that are 1000 times faster than today's Intenlet. 1 his
. .hgh-bandwidth, high-speed provided by Internet2 is sui'licient to transfkr the largeamounts of data that tele-immersion will produce.
I$c;.r.of CSE 16 SNGCE.liolenchery
-
8/9/2019 Sathu G Rajan Nov. 2006
21/26
TELE-IMMERSION
Internet 2 had a peculiar problem : no existing applications
that requires the high levcl of performance provided by internet 2 except teleinimersion
Desktop supercomputers
The Grid will use distributed cotnputiug. There are not
enough supercomputers to deal with the enorlnous amounts of data that n i l l rush through
the Net in the future. As a solution, new networks will connect their PCs so they can
share processing power and hard disk space. 'l'hey nil1 be locked in toa
grid-effectivelyci-eating one supercomputer. About a dozen American universities are doing research 011
various aspects of immersive technologies, including USC, the University of North
Carolina. the University of Pennsylvania and Brown University Mainly two institutions
called P E N N and UNC (University of north Carolina) are doing researches in tele
immersion.
Bandwidth issues
Network bandwidth required to make tclc-irnniersio~~ ork
is one of the main concerns of this new technology. It is estimated that a s much as 1.2
gigabits per second will be needed for future high-quality effects. This is much higher
than the average home connection bandwidth. l'he exact amount of bandwidth needed for
each scene depends on the complexity of the background. With time, the number of
megabits used for transmitting a scene will rcducc as advanced compsession techniques
are established. Initially, bandwidth-intensive applications will have to be limited to the
larger organizations that can afford high c,onnection speeds
-
8/9/2019 Sathu G Rajan Nov. 2006
22/26
'I'ELE-I MMERSlON
CURRENT DEVELOPMENTS
Haptic sensors: Miniaturized forceltorque sensors
There is an increasing need for measuring forces acting
between human hands and the environment. External finger forces are measured by
placing force sensing pads at the fingertips. A wide variety of such pads have been
developed in the past for applications in robotics and medicine, using resistive,
11 capacitive, piezoelectric, or optical elements to detect force. A critical problem with these11
I force sensors is that they are often bulky and inevitably deteriorate the human's haptic
sense, since the fingers cannot directly touch the environment surface.1~
I Recently, much research has focused on rcducing this
problem by inventing thinner and morc tlexiblc force-sensing pads. a new approach to the
detection of finger forces is presented i n order to completely climinate any irnpedinient to
the natural haptic sense and hence thc name 'haptic sensors'. (Haptic mcans that '
relating to or based on the sense of touch ' ). An optical sensor mounted on the fingernailI detects the force. This allows the human to touch the environment with barc fingers andI
perform fine, delicate tasks using the full range of haptic sense. Miniaturized optical
components and circuitry allow the sensor to be disguised as a decorative fiiigernail
covering.
.l-laptic sensor is a new type of touch sensor for detecting
contact pressure at human fingertips. Ijence the sensor is rnounted on the fingernail rather
:ban on the fingertip. Specifically, the fingernail is instrumented with miniature light
emitting diodes (LEDs) and photo detectors in order to measure changes in the reflection
intensity whcn the fingertip is pressed against a surface. I'hc changes i n intensit), are then
~ s e do determine changes in the blood volume under the fingernail. a tcchniquc termed
--reflectance photoplethysmography." A ho~nodynalnic model is used to investigate the
d~namics f the blood volume at two locations under the fingernail.A
miniaturizedprototype nail sensor is de-signed. built, and tested. 'T'he theoretical analysis is verified
through experiment and simulation.
-
8/9/2019 Sathu G Rajan Nov. 2006
23/26
TELE-IMMERSION
Fig :Implementation of tingernail touch sensors
Figure shows the implementation of fingernail touch sensors. For the prototjrpe s h o ~ nherc. two photodiode arrays of dimension4 mm 1 mm are attached end to cnd on the
bottom sidc. Up to 8 of the 32 total photodiodcs canbe m~iredup at once, rcsulting i n up
to cight sensing locations along the Icn@h of thc fingernail. IJp to three LEDs of
dimension 0.25 mm 0.25 min can be placedi n flexible locations beside the photodiode
arrays.
Haptic sensors would allow people to touch prc),jectionsas
if they were real. A 3D sensor and supporting software has been developed and patented
that ena bles the real-time visualization of the haptic sense of pressure. Haptic sen sors can
be used in tele immersion systems to sense the pressure and reconstruct the feeling of
touch in com bination with other devices
Dept.of CSE 19 SNC;CE.Kolencliery
-
8/9/2019 Sathu G Rajan Nov. 2006
24/26
CONCLUSION
Tele-immersion is a fast developing technologq and it isgoing to benefit the common man once Internet-2 comes into picture. i t is of i~nmense
use in thc field of
J MedicineJ It helps in reducing business travel
J Education and numerous other fields
Tele immersion is a dynamic concept, which will
transform the way humans, interact with each other and the world in general.Tele-lmmersion is a technology that is certainly going to
bring a new revolution in the world and let us all hope that this technolog) reaches the
world i n its f i l l 1 flow as quickly as possible.
Dept.of C SE
-
8/9/2019 Sathu G Rajan Nov. 2006
25/26
FUTURE SCOPE
The tele-immersion system of 20 10 would ideally:
Support one or more flat panels/projectors with ultra-high color resolution (say
5000x5000)
Be stereo capable without special glasses
Have several built-in micro-cameras and microphones
Have tether-less, low-latency, high-accuracy tracking
Network to teraflop computing via multi-gigabit optical switches with low
latency
Have exquisite directional sound capability
Be available in a range of compatible hardware and software configurations
Have gaze-directed or gesture-directed variable resolution and quality of
rendering
Incorporate AI-based predictive models to compensate for latency and anticipate
user transitions
Use a range of sophisticated haptic devices to couple to human movement and
touch
Accommodate disabled and fatigued users in the spirit of the Every Citizen
Interface to the NTlI (National Telc- Immersion Initiative ).
Dept.of CSE 2 1 SNC;Ck~.I.;olenchcry
-
8/9/2019 Sathu G Rajan Nov. 2006
26/26
TELE-!MM ERSION
Dept.of CS E 22 SNGCE.Kolcr.z:- L -