jReality — a thread-safe Java scene graph for mathematics

Peter Brinkmann1, Charles Gunn, Tim Hoffmann2,Holger Pietsch, Markus Schmies, and Steffen Weißmann

Institut für Mathematik, MA 3–2, Technische Universität Berlin, Germany1The City College of the City University of New York, New York, U.S.A.

2Mathematisches Institut der Universität München, Germanymathvis@mmath.tu-berlin.de

June 29, 2006

Abstract

We introduce jReality, a thread-safe Java scene graph de-signed with mathematical experiments in mind, includingsupport for rendering noneuclidean geometries.

The design provides a clean separation between scenegraph frontend and rendering backends. Available back-ends include interactive viewers for workstations as wellas CAVE-like environments, a PostScript backend, anda RenderMan backend. The CAVE viewer synchronizesseveral copies of a scene graph across a network.

The tool system of jReality transparently matches in-teractive aspects of a scene to available input devices.For instance, a jReality application will accept input fromkeyboards and mice as well as the tracking devices of aCAVE, without requiring any changes to the application.GUI widgets may be part of the scene.

jReality offers support for all phases of content devel-opment, such as rapid prototyping at a workstation, de-ployment to various environments as an applet or applica-tion, and the presentation of results in publications.

Introduction

jReality1 is a thread-safe Java scene graph that grew outof separate 3D viewers that Holger Pietsch and Tim Hoff-mann developed independently. The design of jReality

1www.jreality.de

began as an amalgam of these early viewers. Maintenanceand development of jReality is currently taking place atthe Technical University of Berlin as well as the City Col-lege of the City University of New York.

Part of the motivation behind jReality was the need todeploy software across various different platforms, withdifferent input/output devices. The visualization group atTU Berlin routinely presents its work on workstations, on-line via Java Web Start2, as embedded applets in presenta-tions3, in portable as well as stationary GEOWALL-typesetups4, and in the PORTAL, a CAVE-like immersive vir-tual reality theater [CNDD93]. Other needs include high-quality rendering for publications and video clips. jReal-ity transparently covers these usage scenarios and more.

The design of jReality provides a clean separationbetween scene graph frontend and rendering backends.Available backends include three interactive viewers (apure Java viewer for portability, an OpenGL/JOGL5

viewer for speed [BS04], and a distributed CAVE viewer),a PostScript backend for publications, and a RenderManbackend for high-quality output [Ups90]. In addition torendering backends, there are backends for managing ascene, such as a tree viewer that allows users to interac-tively view and modify the structure of a scene.

Interactivity of jReality applications is achieved by

2java.sun.com/products/javawebstart3www-sfb288.math.tu-berlin.de/jdvi4geowall.geo.lsa.umich.edu5jogl.dev.java.net

1

means of thetool concept, which separates the seman-tics of an interaction from the physical input device usedto effect the interaction. When developing an interactivejReality application, the programmer will define interac-tive features by adding abstract tools to the scene. Forinstance, if users are supposed to be able to rotate a ge-ometry, then the programmer will simply add a rotationtool to the scene graph node to which the geometry isattached. At runtime, jReality will automatically matchtools in the scene graph to available input devices, so thatthe same code will work with input devices as diverse asmice, joysticks, trackers, etc.

Available tools range from basic geometric operationssuch as rotations and translations to sophisticated naviga-tion capabilities including terrain following. The tool sys-tem also provides support for using Swing GUI widgets ina scene.

Having been designed with mathematical experimentsin mind, jReality provides support for rendering non-euclidean geometries [Gun93, Wee02]. jReality comeswith a basic linear algebra package (which, too, is awareof noneuclidean geometry), but the API is general enoughto work with any other mathematical package and doesnot require programmers to go through the included math-ematics libraries.

The purpose of this paper is to announce jReality, butnot to provide a complete documentation or discussionof technical details. We present the main ideas behindjReality; details will appear in separate papers.

1 Related work

There is no shortage of scene graphs out there. We brieflyreview a representative sample in order to place jRealityin a greater context.

VTK, The Visualization Toolkit, is a flexible, OpenGL-based software package for 3D computer graphics6.It consists of a graphics library and bindings for anumber of languages, including Python and Java. Itssize and use of a C++ library, however, complicateinstallation and preclude the deployment of VTK ap-plications through the web.

6public.kitware.com/VTK

OpenSceneGraphis a high performance 3D graphicstoolkit7. Like VTK, it is OpenGL-based and writ-ten in C++, with bindings for Python and Java. LikeVTK, it takes some effort to install and is not easy todeploy via the web.

JavaView is a 3D geometry viewer and mathematical vi-sualization package in Java8. It is easy to set up andcomes with JavaView-Lite, a small viewer intendedto be embedded in web pages. It does not, how-ever, employ a scene graph; its hierarchies are flat.Moreover, it does not provide support for hardware-accelerated graphics, and its interactive features areless flexible than the tool concept of jReality.

Java 3DTM [Sel02] offers a complex scene graph andfast graphics via OpenGL, but it is not well-suited forsystems that modify a scene arbitrarily because onehas to specify modifiable parts in advance. Modifi-cations of a live scene graph are not supported.

Syzygy [SG03,SBF∗05] offers a scene graph frameworkthat is a thin overlay of OpenGL/GLUT functions.Written in C++, with Python bindings, it is highlyoptimized for use in distributed, CAVE-like installa-tions. Syzygy expects input from tracking devices; ifno tracking devices are available, users have to com-municate with Syzygy through a tool that simulatestrackers.

2 Design

A jReality scene graphis a directed graph without cycles.A scene graph is not necessarily a tree because an instanceof a scene graph node may appear in different locations ofthe graph at the same time. The location of a node in thescene graph is specified by a path to the node.

Rendering backends are derived from the abstract classViewer . Viewers take two parameters: a scene graphroot node and a path that specifies the currently activecamera. Note that the root node is not an intrinsic propertyof the scene graph but a choice for rendering purposes.

7www.openscenegraph.org8www.javaview.de

2

In jReality, all nodes of the scene graph are derivedfrom a common abstract superclass:SceneGraphN-ode . SceneGraphComponent is the only one sub-class ofSceneGraphNode whose instance can have de-scendants. All other nodes are leaves. The leaves in detail:

Transformation : a leaf node of the scene graph thatspecifies a local coordinate system via a 4x4 matrix.This class supports transformations in euclidean,spherical and hyperbolic geometries. The transfor-mation from object coordinates to world coordinatesis the composition of transformations along a pathfrom the root node to an object node.

Appearance : a leaf node of the scene graph that rep-resents shading attributes such as colors and texturesas a key-value dictionary. Keys must be strings; val-ues can be one of several standard types or an ar-bitrary Java class instance. There is no fixed list ofattributes; different viewers may honor different setsof attributes. Shader names themselves are part ofthis dictionary, so that different shaders can be spec-ified.

While each rendering backend is responsible for im-plementing a standard set of shaders not all backendsare expected to implement all shaders. For exam-ple, the OpenGL backend supports shaders written inthe OpenGL Shading Language9, and shader param-eters for RenderMan, will only make sense for cer-tain viewers. A backend that encounters an unknownshader falls back on one of the standard shaders.

When traversing a scene graph, viewers are expectedto maintain a stack of appearances encountered onthe way from the root node. The value of the dictio-nary at a specific node in the scene graph is then con-trolled by an inheritance mechanism which dependson the collected appearances along the path from theroot through the scene graph to this node.

Geometry : a leaf of the scene graph that holds geom-etry information, such as face sets, Bezier patches,etc. Instances of this class can store attributes beyondthe usual coordinates and connectivity information.

9www.opengl.org

As in the case of appearances, the set of possibleattributes is not fixed, and again stored in dictio-naries (seperate ones for vertex, edge, and face at-tributes). This way data for non-standard shaders(like a per-vertex temperature information) can bestored directly on the geometry using the appropriatekey. There is no inheritance for geometry attributes.

Camera: a leaf that holds a camera. Every jRealityviewer contains a scene graph root node, and a cam-era path. The latter is a scene graph path that beginswith the root and ends at a camera node. Renderingfor this viewer is always with respect to this camera.Cameras are general enough to support rendering ina variety of environments, including stereo and off-axis alignment (such as CAVEs).

Light : a leaf that holds a light source. Lights can beglobal or local, with standard distant, point, and spotlight support. Backends collect lights before render-ing.

Attributes of an instance ofSceneGraphCompo-nent include a geometry, a transformation, an appear-ance, a camera, and a light, each of which may benull ,as well as a list of descendants of typeSceneGraph-Component. For every leaf in a connected scene graphthere may be more than one path from the selected rootnode. Each such path defines one copy of the leaf whenthe graph is traversed.

Further attributes of an instance ofSceneGraph-Component are a flag indicating whether geometries at-tached to this instance and its descendants are visible, anda list of tools that provide interactive features associatedwith this instance.

2.1 Internals

The clean separation of frontend and backend is achievedby means of thevisitor pattern [GHJV95]. The visitorpattern allows for the addition of different backends with-out the need to modify or extend the classes that are ofconcern to application programmers.

Thread-safety is achieved by means of a read/write lockmechanism and that allows users to wrap several sequen-cial changes into a Runnable and execute them as if theywhere atomic.

3

Notification of changes to the scene graph are com-municated by means of a system of events (Appear-anceEvent, CameraEvent, GeometryEvent,etc.). Any change of the scene graph is reported byfiring the appropriate events. Some backends use theseevents to maintain copies of scene graphs. The OpenGLbackend does so for rendering reasons, while the CAVEviewer uses this mechanism to keep several copies of ascene graph synchronized across a network.

We will discuss further details of the design of jRealityin a separate paper.

3 Tools and virtual devices

Virtual devicesprovide an abstraction layer between phys-ical input devices and jReality applications. An applica-tion takes its input from virtual devices. At runtime, jRe-ality matches available input devices such as mice, key-boards, and trackers to the virtual devices desired by ascene.

Virtual devices come in two varieties:

Axes take integer values. They are meant to convey twokinds of information, one-dimensional continuousvalues such as joystick axes or mouse coordinates,and on/off values from keys and buttons.

Transformations are 4-by-4 matrices, meant to conveyinformation from trackers.

The tool systemmanages virtual devices. An XMLconfiguration file maps physical input devices to virtualdevices. Moreover, the configuration file may define newvirtual devices constructed from existing ones. For in-stance, a navigation tool may use a joystick if one is avail-able. If no joystick is available, then pairs of keys (suchas W/S for forward/backward and A/D for left/right) maybe used as a substitute for a joystick. Other constructionsof new virtual devices use matrix operations in order tocreate new transformations.

Another virtual device turns mouse motions intopointer transformations (such as those returned by a track-ing device in a CAVE). In addition to virtual devices cor-responding to actual input devices, a number of propertiesof the scene (such coordinate transformations) are avail-able as virtual devices.

Figure 1: Swing widgets in a jReality scene

Tools encapsulate user interactions in a device-independent fashion. Some tools are always active, whilemost tools will require an activation. Each tool declaresthe set of virtual devices it requires, typically an activa-tion device, and more devices that control it when active.In most cases, the configuration file will define more vir-tual devices than most scenes require. The tool systemdiscards any virtual devices that the scene does not use.

When an input device provides new data, the tool sys-tem updates the status of the virtual devices that dependon it. Then the tools are updated in three steps:

1. Tools are activated as necessary. Typically, an ac-tivating event will be a pick on a geometry. Whena pick on a geometry occurs, the tool system willcheck whether any tools attached to this geometryare activated by this event. If no matching tool is at-tached to the picked geometry, then the tool systemwill move up the scene graph towards the root nodeuntil it finds a tool activated by this event.

2. All active tools are updated.

3. Tools are deactivated as necessary.

A scene may also contain Swing GUI widgets (Fig-ure 1). Such widgets appear as textures on planes in ascene, and the tool system will generate the appropriateSwing events corresponding to user actions. For instance,when using Swing widgets in a CAVE, users will interact

4

Figure 2: A discrete minimal surface exported from jRe-ality

Figure 3: A discrete Kühn surface rendered in the soft-ware renderer and as SVG

by means of trackers and buttons on a wand, and the toolsystem will generate mouse motions and clicks.

The details of the tools of jReality are quite subtle andwill be the subject of a separate paper.

4 Examples

jReality has become the main workhorse of the Mathe-matical Visualization Group at TU Berlin, where we typi-cally use it in conjunction with the Oorange developmentenvironment10 and JTEM11, a collection of mathematical

10www.oorange.de11www.jtem.de

Figure 4: Non-euclidean tessellations of spherical (left)and hyperbolic space (right), using regular dodecahedra,shown in the OpenGL backend.

Figure 5: jReality in the Portal

Figure 6: A still from the Mars Flight animation (Marstextures courtesy of ESA/DLR/FU Berlin (G. Neukum))

5

Figure 7: A still from the MATHEON video project

Figure 8: The MATHEON Buddy Bear in jReality

Figure 9: The finished bear

Figure 10: Crossing maps in jReality

6

tools and algorithms.We use jReality at TU Berlin to present results

from discrete differential geometry (Figure 2, Figure 3)[BHS03] in a variety of settings, including fixed as wellas portable GEOWALL-type and the Portal, a CAVE-likeenvironment with three walls (Figure 5).

jReality has also been used for interactive explorationof the classical 3-dimensional non-euclidean geometriesof spherical and hyperbolic space (Figure 4).

jReality was used in a joint project with the MappingDepartment of TU Berlin to create a fly-over of the planetMars based partly on the latest image and elevation datafrom the Mars Express High-Resolution Stereo Camera(HRSC) (Figure 6).

The RenderMan backend was put to good use in threeanimated sequences for the MATHEON video project (Fig-ure 7). A fourth animation, illustrating recent work onembedding triangulated surfaces [HLZ], is in the works.

jReality played a central role in the design of theMATHEON Buddy Bear12, a seven-foot statue that nowadorns the entrance of the Mathematics Building of TU.Figure 8 shows the design in jReality; Figure 9 shows thefinished bear [BGKAC∗06].

The Virtual Math Labs of the Geometry Group atTU Berlin13 include a number jReality applications, suchasjTantrix (Figure 10), an example of a mathematical ap-plication deployed via Java Web Start. It computes andinteractively displays crossing maps [AOS∗02], an impor-tant tool in geometric knot theory.

There are also jReality applications to courseware, suchasSpaceCurve ExplorerandSurfaceExplorer, Web Startapplications for a first semester course in differential ge-ometry of curves and surfaces, also accessible through theVirtual Math Labs.

5 Outlook and conclusion

Several extensions of jReality are in progress, such sup-port for moving textures (i.e., using textures drawn fromvideo content rather than just static pictures), and real-time simulations of smoke using a hardware-acceleratedintegrator (Figure 11) [Wei06]. We intend to add sup-

12www.united-buddy-bears.com13www.math.tu-berlin.de/geometrie/lab

Figure 11: Realtime hardware-accelerated smoke ringsimulation

port for spatialized audio, using OpenAL14 and the JOALProject15, in the future.

jReality has grown to support all phases of content de-velopment, ranging from initial development at a work-station over application deployment to across differentplatforms to the creation of high-quality output for pub-lications. Combined with Oorange and JTEM, jRealityprovides a powerful environment for experimental math-ematics and mathematical visualization. jReality is nowapproaching its official release.

Acknowledgments

We gratefully acknowledge the support of MATHEON

Project F5 under the direction of Ulrich Pinkall and JohnSullivan. Nils Bleicher and Bernd Gonska designed andimplemented the jReality animations for the MATHEON

video project.

References

[AOS∗02] ADAMS C., OTHMER J., STIER A.,LEFEVER C., PAHK S., TRIPP J.: An

14www.openal.org15joal.dev.java.net

7

introduction to the supercrossing index ofknots and the crossing map.Journal ofKnot Theory and Its Ramifications 11, 3(2002), 445–459.

[BGKAC∗06] BRINKMANN P., GUNN C., KHADEM-AL-CHARIEH S., PINKALL U.,SCHMIES M., SPRINGBORN B., SULLI -VAN J., WEISSMANN S.: The MATHEON

Buddy Bear. Preprint, 2006.

[BHS03] BOBENKO A., HOFFMANN T., SPRING-BORN B.: Minimal surfaces from cir-cle patterns: Geometry from combina-torics. to appear in Ann. of Math.(2003).math.DG/0305184.

[BS04] BOARD O. A. R., SHREINER D.:OpenGL Reference Manual: The Offi-cial Reference Document to OpenGL.Addison-Wesley Professional, 2004.www.opengl.org .

[CNDD93] CRUZ-NEIRA C., D.J.SANDIN , DE-FANTI T.: Surround-Screen Projection-Based Virtual Reality: The Design andImplementation of the CAVE. InProceed-ings of SIGGRAPH(1993), ACM SIG-GRAPH, pp. 135–142.

[GHJV95] GAMMA E., HELM R., JOHNSON R.,VLISSIDESJ.: Design Patterns: Elementsof Reusable Object-Oriented Software.Addison-Wesley Professional, 1995.

[Gun93] GUNN C.: Discrete Groups and the Vi-sualization of Three Dimensional Mani-folds. InSIGGRAPH Proceedings(1993),vol. XX, pp. 255–262.

[HLZ] H OUGARDY S., LUTZ F. H., ZELKE M.:Polyhedra of genus 2 with 10 vertices andminimal coordinates. math/0507592.

[SBF∗05] SCHAEFFERB., BRINKMANN P., FRAN-CIS G., GOUDESEUNEC., CROWELL J.,KACZMARSKI H.: Myriad: Scalable VRVia Peer-to-peer Connectivity, PC Clus-tering, and Transient Inconsistency. In

Virtual Reality Software and Technology(2005), ACM.

[Sel02] SELMAN D.: Java 3D Programming.Manning Publications, 2002.java3d.dev.java.net/ .

[SG03] SCHAEFFER B., GOUDESEUNE C.:Syzygy: Native PC Cluster VR. InVirtualReality Conference(2003), IEEE.

[Ups90] UPSTILL S.: RenderMan Companion:A Programmer’s Guide to Realistic Com-puter Graphics. Addison-Wesley Pro-fessional, 1990. renderman.pixar.com/ .

[Wee02] WEEKS J.: Real-Time Rendering inCurved Spaces. InComputer Graphicsand Applications 22 (6)(2002), IEEE,pp. 90–99.

[Wei06] WEISSMANN S.: Realtime simulation offluid flow. Master’s thesis, 2006. in prepa-ration.

8