nguyen - science of information, computation and fusion - spring review 2013

1 DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution 15 February 2013

Integrity Service Excellence

Tristan Nguyen Program Officer

AFOSR/RTC Air Force Research Laboratory

Science of Information, Computation and Fusion

Date: 06 03 2013

2 DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution

2013 AFOSR SPRING REVIEW

NAME: Tristan Nguyen BRIEF DESCRIPTION OF PORTFOLIO: Research new techniques that enable or facilitate extracting, assembling, and understanding of information collected from multiple sources. Challenges:

1. Dealing with information at different levels of abstraction 2. Mechanizing patterns of reasoning in terms of computation.

LIST SUB-AREAS IN PORTFOLIO: Sub-Areas Objectives Bottom-up Low-level Data Analytics

• Discover structures in data and shape them into information • Formulate models to describe different data sources • Find efficient and provable computational algorithms

Top-down High-level Information Processing

• Develop expressive, computable representation of information • Synthesize contextual information with observed data through reasoning

Presenter

Presentation Notes

The bottom-up procedures are predominantly correlational in nature. As a result, they tend to discover patterns and structures based on previously established features or markers, either directly or indirectly. Correlation can be measured by statistical means or some quantifiable metrics and it helps answer the identification or classification problems. Finding robust, accurate markers in different applications is a challenging problem. The top-down procedures offer causal explanation, help construct new hypotheses based on observed data and domain knowledge, verify these hypotheses via reasoning, probabilistically and/or logically. With domain knowledge, we can assemble different pieces of information extracted from the bottom-up process to form a global picture from which inferences can be made. The challenge here is to find a formal framework that can mechanize conceptual-level tasks while dealing with observed data of different types, NOT to apply these tools to different applications. In practice, the top-down and bottom-up steps may not be in locked steps. Several iterations may require. Furthermore, different pieces of information may have to be fused or integrated through different levels of abstraction. After all, information manifests itself in and is represented by different levels of abstraction. Understanding these levels of abstraction can advance our understanding of what information really is.


Bottom-up Data-driven

Focus On:

New Data Structures Information Extraction Procedures Constructive Computational Algorithms Provable Performance Guarantees

Stay Away From:

Data Provenance Information Management Cloud Computing Radar, Communications, Signal Processing

Important but being funded elsewhere

NEW TRENDS (Dealing with Big Data)

Few data samples in high dimensions

Nonlinear high-dimensional data Fast approximation algorithms Integration of multiple models/

techniques

Presenter

Presentation Notes

Although there are many algorithms have been developed over the years to process data and extract information, most of them are driven by common-sense thinking or heuristic approaches. Many of these algorithms lack performance guarantees. They may work well for some particular dataset but do not generalize. We need algorithms or computational procedures that are guaranteed to yield results under some hypotheses or constraints. Data provenance is a growing subject that cuts across many scientific applications. It helps keep track of data history, i.e., where or what data are input and output, and the steps that are implemented to process data. Data provenance, information management, and cloud computing are very well sponsored in the industry, particularly, by big companies with a research arm like Microsoft Research, IBM, Google Research, Yahoo Research, etc. Radar, communications, and signal processing have traditionally been funded by Jon Sjogren in his program. The communications portion is now in Bob Bonneau’s Complex Networks Program, which thoroughly covers communication channel, coding, and networks.


Top-down Concept-driven

Focus On:

Construction of rich data types Models of computation New programming language Connection with data analytics

Stay Away From

Cognitive modeling Decision analysis and modeling Current semantic technologies Various database models

NEW TRENDS Higher-order structures Constructive techniques

Synthesis of reasoning & computing Merging qualitative & quantitative

models

Important but being funded elsewhere

Presenter

Presentation Notes

Finding meaningful representation of data types, data structures for digital data and logical propositions, is an important subject. At AFOSR, we focus on finding a framework that can be used to represent data of different types, to manipulate data via computational procedures, and to mechanize reasoning or inference simultaneously. This would allow computing and reasoning in the same framework. Another focus is on higher-order structures, e.g., relation of relations or object of objects as in the case of nonparametric hierarchical Bayesian models. Constructive techniques, together with a new language for describing data structures and computational procedures, would enable mechanization. That is, we need much more than just information or knowledge representation and reasoning. Current semantic technologies and relational database are mature for integrating information. However, they are very far from yielding optimal performance. First, they need a great deal of manual curation. Second, they intrinsically lack expressive higher-order structures that can capture complex data structures and logical propositions. Third, they don’t naturally interact with the bottom-up data analysis, i.e., data analytics are typically carried out separately, whose results are subsequently merged with database or semantic technologies.


Similar Programs But with Different Emphases and Approaches

Mathematics of Sensing, Exploitation, and Execution

Network-based Hard/Soft Information Fusion Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation Revolutionizing High-Dimensional Microbial Data Integration

Information Integration and Informatics EarthCube Algorithms for Threat Detection (NSF-DTRA-NGA)

Information Integration Intelligent and Autonomous Systems

Presenter

Presentation Notes

Below are two examples of what other agencies are pursuing in different areas that are closely related to this AFOSR program. - DARPA’s MSEE focuses on a unified representation of sensor data. However, the program focuses mainly on videos and images. The approaches are also different: composition models; stochastic grammars; BLOG programming for describing graphical models; closed-loop sensing-inference for computer vision. ARO’s counterparts consist of two separate programs. One deals with lower-level sensor processing, namely, Value-centered Information Theory for Adaptive Learning. This program emphasizes closed-loop sensing inference. In fact, one of its key PIs is also a PI in DARPA’s MSEE. This explains why control theory is featured in sensing. The other program (Network-based Hard/Soft Information Fusion) focuses on higher-level information processing. The latter uses graphs, graph matching, dimension reduction (PCA), and Dempter-Shafer possibility theory to model and fuse information.


Collaborations & Transitions

NSF-DTRA-NGA – Algorithms for Threat Detection

OSD/AFRL/RH – Autonomy

DARPA/AFRL/RY – Mathematics of Sensing, Exploitation and Execution (MSEE)

ASD R&E/JCTD – Advanced Mathematics for DoD Battlefield Challenges

STTR – Space-time Signal Processing for Detecting and Classifying Distributed Attacks in Networks, Numerica Corporation


Air Force Relevance

Technological Applications:

Information Triage Automated Reasoning Human-Machine Interface Formal Verification

Autonomy

Intelligence, Surveillance, Reconnaissance

Cyber Domain

Space Situational Awareness

Presenter

Presentation Notes

The basic research projects in the program will contribute to the development of technologies in Information Triage, Automated Reasoning, Human-Machine Interface, and Formal Verification. Of course Distributed Intelligence and Information Fusion, the official name of the program, is a subset of the first two technologies mentioned above. These four technologies will support four key missions of the AF: ISR, Cyber Domain, Autonomy, and SSA.


Highlights of Research


Image Articulation Manifolds R. Baraniuk et al., Rice University

Motivation: Data are “sparsely” collected by a moving platform Challenges: How to integrate the motion information with data Lack of mathematical tools Objective: To develop a mathematical foundation for Image Articulation Manifolds New Techniques: Generalization of group action Generalization of many ideas in differential geometry

Applications: Beyond videos

Presenter

Presentation Notes

Many current applications have assumed that data points lie on manifolds and leverage differential-geometric techniques to carry out computational procedures. Examining real datasets, however, has revealed that this assumption is invalid. Image Articulation Manifolds arise from such class of datasets. So a different suite of tools is needed to tackle this type of problems. This project is a first step in finding new mathematical techniques that can alleviate this deficiency. Moreover, this project has both bottom-up and top-down flavors. The bottom-up analysis leverages geometric information to identify relevant features in images or to sample data points in Image Articulation Manifolds. The top-down analysis uses motion information to estimate geometric quantities that generalize their differential-geometric counterparts. In fact, the PIs has formulated optical flow metrics, parallel flow fields, flow curvature, etc. The team will continue to investigate this topic for future research directions, besides their work related to video compressed sensing.


Bayesian Nonparametric Modeling of Data E. Fox et al. – University of Washington, Seattle

Objective: Relating multiple time series for analysis and fusion Main Theme: The time-series collection encodes shared dynamic behaviors (features)

Notes: This generative model provides a probabilistic, top-down model for data sources. The PI is collaborating with Ed Zelnio’s group in AFRL/RY.

New Idea: Using Beta and Bernoulli processes to

Allow for infinitely many features Induce sparsity with shared features.

Presenter

Presentation Notes

This is a joint work with A. Willsky, M. Jordan, and E. Sudderth, notable names in non-parametric hierarchical Bayesian mixture modeling. The particular model used in this work belongs to the class of Markov Switching Processes. This project provides a generative-model-based method to assemble different streams of time-series data which are interdependent but encode unknown shared dynamic behaviors. As such, this work provides a top-down statistical model for fusing interdependent time-series data when their explicit forms of dynamics are not given. The goal is to be able to discover these latent behaviors using a Beta-process prior. This approach allows for possibly infinitely many features but assumes that only a small number of nodes have shared features. The assumption results in a sparse mixture model that facilitates computation. In order to compute the posterior conditional distributions, the authors resort to an MCMC technique that samples potentially infinitely many features, dynamic parameters, hyperparameters, and transition variables in the Markov switching process.


Graph-structured Activation A. Singh et al., Carnegie Mellon University

Objectives: Detection and localization of weak structured patterns in large graphs from a small number of compressive measurements. Achievements: Determined the minimum number of measurements and weakest SNR required for detection and localization in lattices. Adaptivity of measurements and structure of activation can improve localization but not detection. Future Research: Consider more general structures of activation on any graphs.

Presenter

Presentation Notes

The PI has worked in signal detection and identification/localization on graphs. The main objectives are: (1) to be able to give a necessary condition for distinguishing the null and alternative hypotheses; (2) to find a statistical testing procedure that can perform quickly with some theoretical guarantee, based on the number of samples or the structure of the graph. -The PI and her collaborators recently constructed two testing procedures, Spanning-Tree Wavelet Bases and Spectral Scan Statistics, which provide tractable substitutes for NP-hard computational procedures of the traditional testing methods like the generalized likelihood ratio test and scan statistic. Under mild assumptions on the class of clusters of activation, the PI established theoretical guarantees on the performance of these two tests.


Objective: Develop reasoning and computing mechanisms across different domains of information to support information fusion. Key Ingredients: Reasoning - Making inference, generating hypothesis, verifying hypothesis based on observed data Computing - Manipulating data or its mathematical structures and connecting with bottom-up data analytics Motivation: Dependent Type Theory was recently applied (2012) to information fusion and situation awareness. Types allows for expressive data structures and properties (logical, mathematical, etc.) Technical Approach: Develop Homotopy Type Theory to mechanize reasoning, computing, and constructing data types in the same framework.

Homotopy Type Theory for Reasoning & Computing

S. Awodey, R. Harper, J. Avigad, Carnegie Mellon University

Presenter

Presentation Notes

Recently, the classical Dependent Type Theory (DTT) was applied to the problem of information and situational awareness (see Formal Foundations for Situation Awareness Based on Dependent Type Theory, Information Fusion, accepted for publication 2012). The authors described how this approach supersedes the (first-order) semantic technology. Moreover, they showed how DTT interacts with sensor outputs to enable information fusion and situation awareness. So, DTT provides a valuable top-down approach to information fusion. However, DTT has many known drawbacks due to its simplistic nature. -This project studies a new type theory, called Homotopy Type Theory (HTT), which has many desirable properties that can overcome DTT’s deficiency. First, HTT has richer mathematical structures and can hence deal with a variety of data types and logical constructs. Having expressive mathematical constructs will facilitate computational procedures on data. Second, HTT solves the intractable computation of types that have beset DTT. Third, one can now describe computational properties of types with HTT. Since HTT refines DTT, it is expected that HTT will have significant impacts in information integration.


Directed Information & Graphical Models N. Kiyavash – University of Illinois, Urbana

Objective: To model networks of coupled random processes with causal dependence structures.

Technical Approaches: Discovered two new equivalent graphical models –

Minimum Generative Model Graph

Directed Information Graph

Constructed efficient algorithms to identify these graphs.

Future Research: Can the time-invariance hypothesis on the causal dependence structures be weakened or removed?

Presenter

Presentation Notes

Directed information is a notion that was formalized in communication theory in the early 90’s with the goal to capture the directivity of information flow. Unlike mutual information, which describes statistical correlation, directed information describes statistical causation. Directed information is defined in terms of the conditional Kullback-Leibler divergence that is, in turn, defined by “causally” conditioned distributions. These causally conditioned distributions are conditional distributions that are factored in a certain way. Directed information has been applied to communication with feed back and other problems in which causal relationships between (nonlinear) processes are assumed. Many applications fall in these categories. This project combines directed information with graphical models to analyze complex aggregated behaviors of interacting processes. Such integrated models provide a statistical framework for fusing together several interdependent agents who exchange information among themselves. More precisely, the PI came up with two graphical models to represent causal statistical dependence structure between a group or network of interacting stochastic processes, Minimal Generative Model Graphs and Directed Information Graphs. She showed that these two models are equivalent under a rather weak assumption of the joint distributions. In addition, she derived the relationships between these graphs and the well-known causal Markov chain and Bayesian networks. Finally, she presented a method for identifying these two types of graphical models.


Sensor Scheduling for Tracking Resident Space Objects

I. Clarkson, University of Queensland

Objective: To improve the current tracking system, Tasking Autonomous Sensors in a Multiple Application Network (TASMAN).

Current Technical Approaches: Unscented Kalman Filter for updating objects’ states and setting up scheduling.

Challenge: Objects can be out of field of view in a scheduling period

New Approach: Integrated searching and tracking via particle-filtering (in collaboration with AFRL’s AMOS)

Simulated Results

Presenter

Presentation Notes

-This project is aimed at improving tracking of resident space objects (RSO) in orbit to support space situational awareness and to avoid collision with space flights. The PI is collaborating with AFRL researchers in Maui, HI. The chief difficulty is to track multiple moving objects with multiple sensors when these objects can be out of view. -The current scheduling system uses a metric, computed from a sub-matrix of the update term in the Kalman update equation, to select when to use a particular sensor to observe a specific RSO for maximum effectiveness. There is no predictive capability implemented in scheduling. In fact, the system only plans for sensor scheduling within 24 hours. So an object may be seen in one period of observation but we don’t know when to expect to see it again. - To improve sensor scheduling, the PI explored a collaborative sensor scheduling scheme that evaluates when and which RSO should be observed by a sensor or when a combination of sensors could yield more accurate measurement update. Next, he devised a method for predictive scheduling in which future observation times are considered. Finally, he proposed an integrated search strategy that can be adapted to missed observations and guide a limited search to reacquire the object for tracking.


Some Intramural Projects

J. Culbertson (RY) & K. Sturtz (Universal Mathematics) o Categorical Formulation of Probability and Bayesian o Collaborating with D. Koditschek (U Penn) and MURI Team

W. Sakla (RY) & T. Klausutis (RW)

o Manifold Learning and Sparse Representation o Will collaborate with a new PI

R. Ilin & L. Perlovsky (RY)

o Integration of text with sensor data via parametric models

W. Curtis (RW) o Map-based Particle Filtering for Target Tracking


FY10 MURI - Control of Information Collection and Fusion

Objective: To formulate a new perspective on the joint control of heterogeneous information sources to simultaneously achieve quantified informational and physical objective.

• RCA.1 Unified Mathematical Representation – for sensor, control, mission objectives – incorporating multiple scales of resolution and

uncertainty • RCA.2 Joint Physical-Information State Descriptors

– capturing physical state of the information gathering system and the state of the information

– include formal expression of constraints limiting state transitions

• RCA.3 Control-Information Linkage to: – robustly link control actions to information states – support feedback to enable simultaneous control of

physical and information states

• Jadbabaie (control) + 1PhD • Koditschek (robotics) + 2PhD + 1.5PD • Kumar (robotics) + 2PhD • Ribeiro (sig. proc.) + 3PhD

Berkeley • Ramachandran (info. thry) + 1PhD • Sastry (control) + 1PhD • Tomlin (control) + 1PhD + 1PD

Illinois • Baryshnikov (math) + 1PhD

Minnesota • Giannakis (sig. proc.) + 4PhD • Roumeliotis (robotics) + 4PhD

Melbourne • Howard (comm., radar) • Moran (appl. math)



Consistent Vision-aided Inertial Navigation System (VINS) – S. Roumeliotis et al. (U Minnesota)

Challenge: VINS is a nonlinear estimation problem; Linearized estimators (e.g., Extended Kalman filter (EKF), Unscented (U)-KF) become inconsistent.

Solution: o Determined unobserved directions of the nonlinear system using finitely many Lie derivatives o Linearized states using the observability matrix o Identified cause of inconsistency o Used the computed unobserved directions to improve consistency and accuracy



Multi-robot Team To Find Targets and Avoid Hazardous Areas – V. Kumar et al. (U Penn)

Challenge: Sensing, communication, and coordination are coupled.

Solution: Distributed algorithms for detection and multi-target detection

and localization using o a recursive filter based on Finite Set statistics o approximated gradient of mutual information between

sensor readings & target locations. Complexity reduction by

o clustering robots into groups o adding access points connected to a central server.

Next Step: Empirical validation and merge with VINS in complex environments.


Summary

Bottom-up data-driven analysis can discover structures in data.

Top-down conceptually driven processing can integrate these structures.

These two directions may not align nicely. So recursion may require.

There are several layers of abstraction in information processing.

Different technical tools are needed to treat various layers of abstraction.

Symbols, Magnitudes, etc.

Semantics, Logics, etc.

Mor

e A

bstr

act


Science of Information, Computation and Fusion

nguyen - science of information, computation and fusion - spring review 2013

Technology

distribution statement

data samples

big data

observed data

contextual information

motion information

different data sources

datadriven focus