computing architectures for virtual reality electrical and computer engineering dept

Computing Architectures for Virtual Reality

Electrical and Computer Engineering Dept.

System architecture

ComputerComputer(rendering (rendering

pipeline)pipeline)

Computing Architectures

Definition:

A key component of the VR system which reads its input devices, accesses task-dependent databases, updates the state of the virtual world and feeds the results to the output displays.

It is an abstraction – it can mean one computer, several co-located cores in one computer, several co-located computers, or many remote computers collaborating in a distribute simulation

The VR EngineThe VR Engine


The real-time characteristic of VR requires a VR engine which is powerful in order to assure:

fast graphics and haptics refresh rates (30 fps for graphics and hundreds of Hz for haptics);

low latencies (<100 ms to avoid simulation sickness);

at the core of such architecture is the rendering pipeline.

within the scope of this course rendering is extended to include haptics

ApplicationApplication GeometryGeometry RasterizerRasterizer


The process of creating a 2-D scene from a 3-D model iscalled “rendering.” The rendering pipeline has three functional stages.

The Graphics Rendering PipelineThe Graphics Rendering Pipeline

Modern pipelines also do anti-aliasing for points, lines or the whole scene;

Aliased polygons(jagged edges)

Anti-aliased polygons

How is anti-aliasing done? Each pixel is subdivided (sub-sampled) in n regions, and each sub-pixel has a color;

The anti-aliased pixel is given a shade of green-blue (5/16 blue + 11/16 green). Without sub-sampling the pixel would have been entirely green – the color of the center of the pixel (from Wildcat manual)

More samples produce better anti-aliasing;

8 sub-samples/pixel8 sub-samples/pixel

16 sub-samples/pixel16 sub-samples/pixel

From Wildcat “SuperScene” manual http://62.189.42.82/product/technology/superscene_antialiasing.htm



The Rendering PipelineThe Rendering Pipeline

The application stageThe application stage Is done entirely in software by the CPU; It reads Input devices (such as gloves, mouse); It changes the coordinates of the virtual camera; It performs collision detection and collision

response (based on object properties) for haptics; One form of collision response if force feedback.

Higher resolution model 134,754 polygons.

Low res. Model ~ 600 polygons

Application stage optimization…Application stage optimization… Reduce model complexity (models with less polygons – less to feed down the pipe);

Application stage optimization…Application stage optimization… Reduce floating point precision (single precision

instead of double precision) minimize number of divisions Since all is done by the CPU, to increase

speed a dual-processor (super-scalar) architecture

is recommended.

Rendering pipeline




The geometry stageThe geometry stage Is done in hardware; Consists first of model and view transforms Next the scene is shaded based on light models; Finally the scene is projected, clipped, and

mapped to the screen coordinates.

The lighting sub-stageThe lighting sub-stage It calculates the surface color based on: type and number of simulated light sources; the lighting model; the reflective surface properties; atmospheric effects such as fog or smoke. Lighting results in object shading which makes

the scene more realistic.

The lighting sub-stage optimization…The lighting sub-stage optimization… It takes less computation for fewer lights

in the scene; The simpler the shading model, the less

computations (and less realism): Wire-frame models; Flat shaded models; Gouraud shaded; Phong shaded.

The lighting modelsThe lighting models Wire-frame is simplest – only shows polygon

visible edges; The flat shaded model assigns same color to all

pixels on a polygon (or side) of the object; Gouraud or smooth shading interpolates colors

Inside the polygons based on the color of the edges; Phong shading interpolates the vertex normals

before calculating the light intensity based on the

model described – most realistic shading model.

Computing architectures

Gouraud shading modelGouraud shading model

Flat shading modelFlat shading model

Wire-frame modelWire-frame model

The Rasterizer StageThe Rasterizer Stage Performs operations in hardware for speed; Converts 2-D vertices information from the

geometry stage (x,y,z, color, texture) into pixel

information on the screen; The pixel color information is in color buffer; The pixel z-value is stored in the Z-buffer (has

same size as color buffer); Assures that the primitives that are visible from

the point of view of the camera are displayed.

The Rasterizer Stage - continuedThe Rasterizer Stage - continued The scene is rendered in the back buffer; It is then swapped with the front buffer which

stores the current image being displayed; This process eliminates flicker and is called

“double buffering”; All the buffers on the system are grouped into the

frame buffer.

Distributed VR architecturesDistributed VR architectures Single-user systems: multiple side-by-side displays; multiple LAN-networked computers; Multi-user systems: client-server systems; pier-to-pier systems hybrid systems;

Distributed VR architecturesDistributed VR architecturesClient-server systems•Clients : service requesters•Server: acts as distributed application •Often clients and servers communicate over a computer network on separate hardware, but both client and server may reside in the same system.•A server machine is a host that is running one or more server programs which share their resources with clients.

Distributed VR architecturesDistributed VR architecturesPeer to peer•Peers are equally privileged•Peers make a portion of their resources, such as processing power, disk storage or network bandwidth, directly available to other network participants.• Peers are both suppliers and consumers of resources.

Distributed VR architecturesDistributed VR architecturesHybrid Technology•Consist of more than one technologies• Integration of Fuzzy logic, Neural Networks and Genetic Algorithms nature's problem solving strategies.• Neural Networks are highly simplified model of human nervous system which mimic our ability to adapt to circumstances and learn from past experience. • Fuzzy logic addresses the imprecision or vagueness in input and output description of the system.• Genetic algorithms are inspired by biological evolution, can systemize random search and reach to optimum characteristics.

(3DLabs Inc.)

Single-user, multiple displaysSingle-user, multiple displays

Side-by-side displays.Side-by-side displays.

Used is VR workstations (desktop), or in large volume displays (CAVE or the “Wall”); One solution is to use one PC with graphics accelerator for every projector; This results is a “rack mounted” architecture, such as the MetaVR “Channel Surfer” used in

flight simulators or the Princeton Display Wall

Side-by-side displays.Side-by-side displays.

Another (cheaper) solution is to use one PC only; with several graphics accelerator cards (one for every monitor). Windows 2000 allows this option, while Windows NT allowed only one accelerator per system; Accelerators need to be installed on a PCI bus;

Genlock..Genlock..

If the output of two or more graphics pipes is used to drive monitors placed side-by-side, then the display channels need to be synchronized pixel-by-pixel; Moreover, the edges have to be blended, by creating a region of overlap.

(Courtesy of Quantum3D Inc.)

Problems with non-synchronized displays...Problems with non-synchronized displays...

CRTs that are side-by-side induce fields in each other, resulting in electronic beam distortion and flickers – need to be shielded; Image artifacts reduce simulation realism, increase latencies, and induce “simulation sickness.”

Synchronization of displays:Synchronization of displays:

software synchronized – system commands that frame processing start at same time on different rendering pipes; does not work if one pipe is overloaded – one image finishes first


BufferCRT


BufferCRTSynchronization command

Synchronization of displays:Synchronization of displays:

frame buffer synchronized – system commands that frame buffer swapping starts at same time on different rendering pipes; does not work because swapping depends on electronic gun refresh - one buffer will swap up to 1/72 sec before the other.


BufferCRT


Buffer

CRTSynchronization command

Synchronization of displays:Synchronization of displays: video synchronized – system commands that CRT vertical beam starts at same time; one CRT becomes the “master” does not work if horizontal beam is not synchronized too (one line too many or too few).


BufferMaster CRT


Buffer Slave CRT

Synchronization command

Synchronization of displays: Synchronization of displays:

Best method is to have software + buffer + video synchronization of the two (or more) rendering pipes


BufferMaster CRT


Buffer Slave CRT

Synchronization commandSynchronization commandSynchronization command


PC ClustersPC Clusters

multiple LAN-networked computers; used for multiple-PC video output; used for multiple computer collaboration (when computing power is insufficient on a single machine) – older approach.

Chromium cluster of 32 rendering servers and four control servers

Princeton display wall using eight LCD rear projectors (1998)

Princeton display wall: eight 4-way Pentium-Pro SMPs with E&S graphics accelerators. They drive 8 Proxima 9200 LCD projectors. (1998)


Multi-User distributed remote system Multi-User distributed remote system architecture:architecture: Multiple modem-networked computers; multiple LAN-networked computers; multiple WAN-networked computers;

computing architectures for virtual reality electrical and computer engineering dept

Documents

vr engine slide

vr system

computer engineering

realtime characteristic

colocated computers

colocated cores

fast graphics

remote computers