testbed for mobile augmented battlefield visualization: summing up may 10, 2006

36
Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Post on 21-Dec-2015

219 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Testbed for Mobile Augmented Battlefield Visualization:

Summing UpMay 10, 2006

Page 2: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

• Xin Zhang, Tazama Upendo St Julien, Ramesh Rajagopalan, William Ribarsky, Pramod Varshney, Chilukuri Mohan, and Kishan Mehrotra . Dynamic Decision Support for Mobile Situational Visualization. AppliedVis 2005.

• William Ribarsky, co-editor, Special Issue on Haptics, Telepresence, and Virtual Reality, IEEE Transactions on Visualization and Computer Graphics (November, 2005).

• Justin Jang, Peter Wonka, William Ribarsky, and C.D. Shaw. Punctuated Simplification of Man-Made Objects. To be published, The Visual Computer.

• Tazama St. Julien, Joseph Scoccinaro, Jonathan Gdalevich, and William Ribarsky. Sharing of Precise 4D Annotations in Collaborative Mobile Situational Visualization. Submitted to IEEE Symposium on Wearable Computing.

• Remco Chang, Thomas Butkiewicz, Caroline Ziemkiewicz, Zachary Wartell, Nancy Pollard, and William Ribarsky. Using Urban Legibility to Produce Completely Navigable Large Scale Urban Models. To be published, ACM SIGGRAPH 2006 Short Papers.

• Remco Chang, Thomas Butkiewicz, Caroline Ziemkiewicz, Zachary Wartell, Nancy Pollard, and William Ribarsky. Hierarchical Simplification of City Models to Maintain Urban Legibility. Submitted to IEEE Transactions on Visualization and Computer Graphics.

• Xin Zhang, Tazama Upendo St Julien, Ramesh Rajagopalan, William Ribarsky, Pramod Varshney, Chilukuri Mohan, and Kishan Mehrotra . An Integrated Path Engine for Mobile Situational Visualization. To be submitted.

Publications in the Last Year

Page 3: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Matrix of Project Activities and ResultsProposed Tasks

What Was Done Done This Year

Multimodal 3D Interaction

Development and evaluation of gesture and voice interface; implementation and use of new interface in mobile environment.

Mobile Visualization

Implementation and initial evaluation; scenario development and evaluation.

Integrated mobile sitvis & decision support

4D Modeling Automated tree modeling; initial modeling of large collections; further modeling of heterogeneous collections.

Dynamic, Universal Data Structures

Paged object data structure for simple buildings; scalable structure; multiresolution buildings from multiple sources.

Interactive Rendering and Visualization

Simple building LODs; view dependent, appearance preserving methods; new punctuated simplification approach.

Urban legibility approach for very large collections of buildings

Collaboration and Integration

Initial implementation of mobile augmented testbed; use and evaluation of testbed.

Technology Transfer

Work with Army, Sarnoff Corp., NRL; DHS.

New projects with Army and DHS; possible new project with DTO

Page 4: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Transitions

•Presented mobile situational visualization and its relation to homeland security at invited talks at a special session of the AAAS meeting on the National Visualization and Analytics Center (February, 2005), at AppliedVis 2005 (May, 2005), and at an invited presentation for the DHS Regional Visualization and Analytics Centers (January, 2006).

•Using work in urban terrain analysis begun here as a foundation, began work on a project funded by ARO for eye-point dependent models applied to terrain analysis and applications such as line-of-sight.

•Made a proposal for the DTO ARIVA project that will use, among other things, urban infrastructure visualization. The proposal is now in Phase 2 evaluation.

•Established the Southeastern Regional Visualization and Analytics Center, funded by DHS. Among other things, the SRVAC will be looking at critical infrastructure simulations for disaster relief planning and emergency response. The terrain visualization and modeling capabilities developed here will be used.

Page 5: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Visualization and Analytics Centers

RVACUniv. of North Carolina Charlotte,Georgia Tech

RVACPenn. State

DHS

GVAC

NVAC: Pacific Northwest National Laboratory

RVACStanford University

Scholars

http://nvac.pnl.gov/www.pnl.gov/infoviz

A Partnership with Academia, Industry,Government Laboratories

Detecting the Expected -- Discovering the UnexpectedTM

RVACUniversity of Washington

RVACPurdueUniversity

IVAC

Consortium

Page 6: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

RVACUniversity of Washington

RVACPurdueUniversity

RVACUniv. of North Carolina Charlotte,Georgia TechBank of America

RVACPenn. State

DHS

GVAC(s)

NVAC: Pacific Northwest National Laboratory

RVACStanford University

Scholars

Consortium

IVAC

http://nvac.pnl.gov/www.pnl.gov/infoviz

Detecting the Expected -- Discovering the UnexpectedTM

Alaska

NewZealand

Australia

Hawaii

Europe

Canada

PacificRim

IndianaUniv.Schoolof Medicine

DrexelUniversityNY EmergencyManagementPort of Authority

NSF

Visualization and Analytics Centers

A Partnership with Academia, Industry,Government Laboratories

Page 7: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Mobile Situational Visualization

User equipped with mobile situational visualization system

Mobile Situational Visualization: An extension of situation awareness that exploits and integrates interactive visualization, mobile computing, wireless networking, and multiple sensors:

• Mobile users with GPS, orientation sensing, cameras, wireless

• User carries own 3D database• Servers that store and disseminate information

from/to multiple clients (location, object/event, weather/NBC servers)

• Location server to manage communications between users and areas of interest for both servers and users

• Ability to see weather, chem/bio clouds, and positions of other users

• Accurate overviews of terrain with accurately placed 3D buildings

• Ability to mark, annotate, and share positions, directions, speed, and uncertainties of moving vehicles or people

• Ability to access and playback histories of movement

Page 8: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Drawing Area

Buttons Pen Tool

Mobile Team

Collaboration Example

collaborators

Shared observations of vehicle location, direction, speed

Mobile Situational Visualization

Page 9: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

• Everybody has a location in space and time in the Virtual World

• Geographic server lookup approach– Users– Location Servers– Data Servers

Weather Server

UserUser

User

Location Server

Traffic ServerAnnotation Server

GeoData Server

Mobile Sitvis Collaborative Environment

Page 10: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

• Everybody has a location in space and time in the Virtual World

• Geographic server lookup approach– Users– Location Servers– Data Servers

UserUser

User

Location Server

Traffic ServerAnnotation Server

GeoData Server

Weather Server

Mobile Sitvis Collaborative Environment

Page 11: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Testbed Development

Accurate placement of modeled buildings and trees from multiple sources

Georgia Tech campus -Navigable environment

Detailed urban componentRealistic urban buildings

x

Page 12: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Scenarios and ResultsScenario 1A commander, out of sight of his unit, directs it. He creates waypoints and paths through the mobile sitvis system that individuals in the unit move to.

Scenario 2Three individuals in a unit track a moving subject. They must keep it in sight and coordinate their tracking activities.

In both scenarios, working with the full mobile sitvis system is compared with a traditional method (individuals with only radio communication and maps). Tracked subjects followed paths (set for about equal length and number of turns) that were not known to unit members.

Page 13: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Scenarios and ResultsScenario 2: History of all users’ locations and annotations over a 45 minute session. (Tracked subject is in red.)

Page 14: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Scenarios and Results

What did we find?•Mobile sitvis works!•It does as well as traditional method (radio + map) for tracking a moving subject.

•It is better than traditional method for command operations that direct multiple units.

•It provides significant new capabilities:-Significantly more accurate location than GPS alone.-Specific digital annotations in space-time that can be shared immediately.

-Overviews of several moving, annotated entities that can be understood all at once

-Histories for tracking and analysis•This work suggest new scenarios of greater impact.

Page 15: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Mobile Situational Visualization +Dynamic Decision Support

3D Interaction

Visualization

Decision Support Module

4D Geospatial Server

SensorsUser Input

Collaboration with the Syracuse team

Page 16: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Mobile Situational Visualization +Dynamic Decision Support

Page 17: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

We have fully integrated the dynamic decision support engine with mobile situational visualization, providing the following capabilities:

•a structure for shared interaction and collaboration among mobile users,

•general methods for heterogeneous spatiotemporal sensor organization and display,

•a decision support module supporting activity recognition, response planning, and behavioral modeling that is integrated with the mobile visualization structure and will accept the mobile users as collaborating agents,

•a prototype mobile situational visualization system that employs the decision engine to produce meaningful responses in one or more urban scenarios.

Mobile Situational Visualization +Dynamic Decision Support

Page 18: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Dynamic Decision Support: Grid-Based Approach

•The region to be traversed is laid out in a grid, balancing computation cost (no. of grid cells) versus accuracy (edges or nodes in the same grid cell).

•Edge relaxation is used to choose vertices. Vertices connected by valid edges are considered, and those with the best value of a quality metric, Q, are chosen.

•A probability risk model is applied. A simple zero mean Gaussian distribution with a finite range is used to model point risks. Default values are given for grenades, rifles, etc.

•If multiple risks overlap in a grid cell, the multiple threat is computed as [1 - (1 - P1) x (1 – P2) x …… x (1 - Pn)] where there are n threats.

Djikstra’s single source, single destination shortest path algorithm is used.

Page 19: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Dynamic Decision Support: Grid-Based Approach

Red boundary shows selected area in dense urban area (mid-town Atlanta).

Page 20: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Green path (highlighted with green dots): optimal (in this case low risk) path for selected balance between risks and path length.

Our initial urban scenario is route planning under dynamic threats. Threats of different extents and risks are sighted, placed, and shared by mobile users. The decision engine provides a real-time path that balances on a continuous scale between risk and shortest path (where the mobile user can select the balance).

Mobile Situational Visualization +Dynamic Decision Support

Page 21: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

The mobile decision engine produces fast, accurate, and usable results

Mobile Situational Visualization +Dynamic Decision Support

Safe route Some risk

Shortest route

Page 22: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Future Directions

•Combine dynamic route planning with line-of-sight to take into account obstructions in determining risk.

•Scale up to larger areas.•Take into account moving risks or risks that change in other ways.

•Support other decisions.

Page 23: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Interactive Visualization of Very Large Urban Spaces

•How can one freely navigate very large urban spaces? - A medium size city can have several hundred

thousand buildings; a large city can have millions of buildings.

- Even if the simplest building models are rendered, there could still be an overwhelming amount of geometry and textures.

View of Xinxiang, China with over 26,000 buildings.

What should be rendered? Apply knowledge from urban planning: urban legibility.

Page 24: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

•Urban legibility embodies concepts from urban planning about what makes an urban space understandable and more easily navigable. (For example, depict the city around the concepts of paths, edges, districts, nodes and landmarks.)

•Can we find an automated way to embody these concepts and thus keep the city legible (and recognizable) at all scales?

Interactive Visualization of Very Large Urban Spaces

Page 25: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Yes, we can shape our automated urban analysis to embody the urban legibility principles.

Original (textured) District Simplification with our method

Simplification with Qslim Our simplified model with textures applied

Interactive Visualization of Very Large Urban Spaces

Page 26: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Skyline at full resolution

Skyline with 7% polygonsLandmark preservation

Full resolution Interactive view with 18% polygons and greatly simplified textures

View-dependent rendering

Perceptual errors are not very noticeable because conceptual structure (i.e., what’s important) is retained.

Interactive Visualization of Very Large Urban Spaces

Page 27: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

View-Dependent Rendering ofVery Large Collection

Bounding box

Selected LOD

•Hierarchical multiresolution organization

•View-Dependent LOD for large collections of 3D models

Q

Q QQ Q

Q

Q QQ Q

Q

Q QQ Q

Q

N LevelsLinked Global Quadtrees

Viewpoint

Screen

Page 28: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Knowledge Visualization: Very Large Urban Spaces

Video

Page 29: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Organizing Large Collections of 3D Models for Interactive Display

•Merging of different types and formats•Automated replacement of lower resolution duplicate structures

Common format and organization for different types

Q

Q QQ Q

Q

Q QQ Q

Q

Q QQ Q

Q QQ Q

Linked Global Quadtrees

Page 30: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Paging, Culling, and Fast Rendering

Quadcell

Block

Block

QQQQ

Linked global quadtree

Block

BlockBlock

Block

Out-of core Storage

Page 31: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Hierarchical, Multiresolution Organization

Quadtrees

LODs

Page 32: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Collections of simple geometry

Quadtree interior nodes Quadtree leaf nodes

collection1 collection2 collection3

model1 tree2 sg3

Hierarchical, Multiresolution Organization

Urban Legibility Detailed Hierarchical Simplification

Page 33: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Questions?

www.viscenter.uncc.edu

Page 34: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Publications from Previous Years

• Ernst Houtgast, Onno Pfeiffer, Zachary Wartell, William Ribarsky, and Frits Post. Navigation and Interaction in a Multi-Scale Stereoscopic Environment. IEEE Virtual Reality 2004.

• Nickolas Faust and William Ribarsky. Integration of GIS, Remote Sensing, and Visualization. Invited paper, Proc. Remote Sensing 2003 (Barcelona, 2003).

• William Ribarsky, editor (with Holly Rushmeier). 3D Reconstruction and Visualization of Large Scale Environments. Special Issue of IEEE Computer Graphics & Applications (December, 2003).

• David Krum, Olugbenga Omoteso, William Ribarsky, Thad Starner, and Larry Hodges. Evaluation of a Multimodal Interface for 3D Terrain Visualization. pp. 411-418 IEEE Visualization 2002.

• Justin Jang, William Ribarsky, Christopher Shaw, and Peter Wonka. Appearance-Preserving View-Dependent Visualization. IEEE Visualization 2003, pp. 473-480.

• William Ribarsky, Zachary Wartell, and Nickolas Faust. Precision Markup Modeling and Display in a Global Geospatial Environment. Proceedings SPIE 17th International Conference on Aerospace/Defense Sensing, Simulation, and Controls (2003).

• William Ribarsky. Virtual Geographic Information Systems. The Visualization Handbook, Charles Hanson and Christopher Johnson, editors (Academic Press, New York, 2003).

• Zachary Wartell, Eunjung Kang, Tony Wasilewski, William Ribarsky, and Nickolas Faust. Rendering Vector Data over Global, Multiresolution 3D Terrain. Eurographics-IEEE Visualization Symposium 2003, pp. 213-222.

• Peter Wonka, Michael Wimmer, Francois Sillion, and William Ribarsky. Instant Architecture. Siggraph 2003, pp. 669-678 (2003).

• Tony Wasilewski, William Ribarsky, and Nickolas Faust. From Urban Terrain Models to Visible Cities. Vol. 22(4), pp. 10-15, IEEE CG&A (2002).

• David Krum, Rob Melby, William Ribarsky, and Larry Hodges. Isometric Pointer Interfaces for Wearable 3D Visualization. ACM CHI 2003.

• William Ribarsky, “Towards the Visual Earth,” Workshop on Intersection of Geospatial Information and Information Technology, National Research Council (October, 2001).

• William Ribarsky, Christopher Shaw, Zachary Wartell, and Nickolas Faust, “Building the Visual Earth,” to be published, SPIE 16th International Conference on Aerospace/Defense Sensing, Simulation, and Controls (2002).

Page 35: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Publications from Previous Years

• David Krum, William Ribarsky, Chris Shaw, Larry Hodges, and Nickolas Faust. Situational Visualization. pp. 143-150, ACM VRST 2001 (2001).

• David Krum, Olugbenga Omoteso, William Ribarsky, Thad Starner, and Larry Hodges. Speech and Gesture Multimodal Control of a Whole Earth 3D Virtual Environment. Eurographics-IEEE Visualization Symposium 2002. Winner of SAIC Best Student Paper award.

• “Acquisition and Display of Real-Time Atmospheric Data on Terrain,” T.Y. Jiang, William Ribarsky, Tony Wasilewski, Nickolas Faust, Brendan Hannigan, and Mitchell Parry, Proceedings of the Eurographics-IEEE Visualization Symposium 2001, pp. 15-24.

• “Client-Server Modes of GTVGIS,” Nick Faust, William Ribarsky, and Frank Jiang, Vol. 4368A, SPIE 15th Annual Conference on Aerosense (2001).

• “Hierarchical Storage and Visualization of Real-Time 3D Data,” with Mitchell Parry, Brendan Hannigan, William Ribarsky, T.Y. Jiang, and Nickolas Faust, Proc. SPIE 15th Annual Conference on Aerosense 2001, Vol. 4368A.

• “Semiautomatic Landscape Feature Extraction and Modeling,” Matthew Grimes, Tony Wasilewski, Nickolas Faust, and William Ribarsky, Proc. SPIE 15th Annual Conference on Aerosense (2001), Vol. 4368A.

• “Real-Time Global Data Model for the Digital Earth,” William Ribarsky, Nickolas Faust, William Ribarsky, T.Y. Jiang, and Tony Wasilewski, Proceedings of the INTERNATIONAL CONFERENCE ON DISCRETE GLOBAL GRIDS (2000).

• Development of Tools for Construction of Urban Databases and Their Efficient Visualization,” Nickolas Faust and William Ribarsky, Modeling and Visualizing the Digital Earth, Mahdi Abdelguerfi, Editor (Kluwer, Amsterdam, 2001).

• Computers & Graphics, Special Issue on Data Visualization (Vol. 24, no. 3, June, 2000), Editors Eduard Groeller, William Ribarsky, and Helwig Loeffelmann.

Page 36: Testbed for Mobile Augmented Battlefield Visualization: Summing Up May 10, 2006

Students Who Worked on Project

• Remco Chang• Tom Butkiewicz• Caroline Ziemkiewicz• Xin Zhang• Justin Jang• Tazama St. Julien• David Krum• Olugbenga Omoteso • Jaeil Choi• Weidong Shi• Guoquan (Richard) Zhou• Eunjung Kang• Brendan Hannigan• Mitchell Parry• Matthew Grimes• Ernst Houtgast• Onno Pfeiffer• Joseph Scoccinaro• Jonathan Gdalevich