improving nest performance using surrogates
DESCRIPTION
Improving NEST Performance Using Surrogates. Project Status: Dec 16, 2003. David W. Etherington Andrew J. Parkes Matt Ginsberg. University of Oregon. C I R L. Administrative. Project Title: Improving NEST Performance Using Surrogates Program Manager: Vijay Raghavan - PowerPoint PPT PresentationTRANSCRIPT
Improving NEST Performance
Using Surrogates
C I R LC I R L
David W. EtheringtonAndrew J. Parkes
Matt Ginsberg
Project Status: Dec 16, 2003
University of Oregon
C I R LC I R L
Administrative
• Project Title: Improving NEST Performance Using Surrogates
• Program Manager: Vijay Raghavan
• PI Names: David W. Etherington, Matthew L. Ginsberg, Andrew J. Parkes
• PI Phone Numbers: 541-346-{0472, 0471, 0434}
• PI E-Mail Addresses: {ether, ginsberg, parkes}@cirl.uoregon.edu
• Company/Institution: CIRL/University of Oregon
• Contract Number: F33615-02-C-4032
• AO Number:
• Award Start Date: 9/12/2002
• Award End Date: 9/11/2005
• Agent Name/Organization: Ed DePalma, AFRL
C I R LC I R L
Subcontractors and Collaborators
• Subcontractors– none
• Collaborators– none
Problem and Challenge New Ideas
FY04 Schedule
Application of surrogates to distributed reasoningeasy-to-measure standins for properties of interest
Transition behaviors can be used to justify use of surrogates for system behavior prediction/control
Extend notion of relaxation to approximate relaxationsExplore the interaction between structure and transitions
Improving NEST PerformanceUsing Surrogates
Avoidance of dangerous regions; develop tractable mechanisms for design/control/prediction
Find manageable surrogates for critical properties
1QFY04• Demonstrate use of local surrogates in random NESTs• Determine thresholds for static global surrogates
2QFY04• Discover local surrogates for static properties of interest• Use static global surrogates to predict/control structured NESTs
3QFY04• Develop temporally structured NEST testbed/generator• Determine thresholds for static local surrogates
4QFY04• Identify functional dynamic properties in temporally structured NESTs• Use static local surrogates to predict/control structured NESTs
Impact
Simplified design and control of performanceSimplified performance prediction/modeling with
uncertain configuration informatione.g., prediction of capabilities under various
attrition/failure modelsPredict/determine maintenance/replenishment
requirements under uncertain field conditionse.g., model tradeoffs of capabilities vs costs
Q1 Q2 Q3 Q4
Etherington, ParkesCIRL, University of Oregon
cost
quality
approx-imation
full propertyparam
estimation
pure surrogates
estimation + phase xition
C I R LC I R L
Problem Description/Objective
• Problem: NEST systems induce difficult design and control problems– these impair our ability to achieve expected
benefits like robustness and speed
• Goal: develop the ability to predict, analyze, and bound NEST performance “in the large”
• Approach: use surrogates: properties that are easy to measure/control yet strongly correlated with real objectives– identify “hard” properties of interest– exploit structure to find surrogates
C I R LC I R L
Current Technical Approach
• Identify hard problems:– model technical problems of interest
• experiment to determine transition points/control variables• discrete/continuous analysis of identified parameters
– predict scaling behavior
• Apply appropriate tools to find surrogates– simple control behavior
• thresholds & experimental bounds on control variables
– complex behavior• group theory (ZAP)• constraint weakening/strengthening
C I R LC I R L
Changes in Technical Approach
• Deëmphasized development of synthetic generators/surrogates to more quickly connect to NEST demonstration platforms.
• Reëmphasized computational hardness of target problems, as well as utility for NEST.
• Reëmphasized general modeling as opposed to search for specific surrogates.
C I R LC I R L
Progress Since Last PI Meeting
• Built NEST-generating system– parameterized distribution, connectivity, etc.
• Identified “pseudo-density” connectivity surrogate in structured NESTs– covers realistic distributions, interesting properties– identified thresholds controlling connectivity
• ‘Twisted tree’ disjoint spanning tree surrogate– demonstrated on various, non-idealized, distributions
• Discovered group-theoretic structure exploitation– potential exponential compaction of discovery process
Details in technical presentation, following.
C I R LC I R L
Deliverables and Publications
Generalizing Boolean Satisfiability I: Background and Existing Work. Heidi E. Dixon, Matthew L. Ginsberg, and Andrew J. Parkes: to appear in JAIR, 2004.
Generalizing Boolean Satisfiability II: Theory. Heidi E. Dixon, Matthew L. Ginsberg, and Andrew J. Parkes: in preparation, to be submitted to JAIR.
Generalizing Boolean Satisfiability III: Implementation. Heidi E. Dixon, Matthew L. Ginsberg, and Andrew J. Parkes: in preparation, to be submitted to JAIR.
Scaling Properties of Pure Random Walk on Random 3-SAT. Andrew J. Parkes. Proceedings of the Eighth International Conference on Principles and Practice of Constraint Programming (CP2002). Published in Lecture Notes in Computer Science, LNCS 2470. Pages 708--713.
Easy Predictions for the Easy-Hard-Easy Transition. Andrew J. Parkes. Eighteenth Nat’l Conference on Artificial Intelligence (AAAI-02)
Likely Near-Term Advances in SAT Solvers. Heidi E. Dixon, Matthew L. Ginsberg, Andrew J. Parkes, at MTV-02.
Inference methods for a pseudo-Boolean satisfiability solver. Heidi E. Dixon and Matthew L. Ginsberg. AAAI-02.
Six further papers are currently in preparation for submission to AAAI-03, titles will be added after the blind review period has expired.
C I R LC I R L
Success Criteria
• Metrics– predictive accuracy of discovered surrogates – utility of the control surrogates that are discovered– design process simplification through surrogates– generality/reusability of methodology
• Decision points– ability to generate predictive/control models for static
problems of interest– ability to demonstrate transitions implying existence
of, and to identify, surrogates for those problems
C I R LC I R L
Project Plans: Collaboration
• Collaborate in extreme scaling effort– help identify hard, make-or-break, issues– model problematic aspects of “ideal” approach– develop surrogates that enable minimal functionality
Metric: impact of models, problems, surrogates found
• Assist in exploitation of existing surrogates– twisted-path implementation, etc.
Metric: degree of successful exploitation
• Develop new surrogates– exfiltration, sentry/relay/sleep issues, scalability, etc.
Metric: utility for design or modeling
C I R LC I R L
Project Plans: Modeling
• Study problem-distribution’s influence on surrogates– Payoff: help find useful, reliable, surrogates– Risk: open ended; difficult to determine right distributions
• Strengthen/weaken constraint-set to identify surrogates– Payoff: provide systematic methods– Risk: unproven; may not provide effective mechanism
• Off-line search (characterize search space, and identify useful gradient indicators that aid convergence)– Payoff: improved ability to find surrogates automatically– Risk: required understanding of the underlying search space
may be slow in coming
Metrics: in all of these, the metrics will be predictive accuracy and the ability to find useful surrogates
C I R LC I R L
Random NESTs
Project Schedule and Milestones
Structured NESTs
Temporally structured NESTs
Upcoming milestones:• demonstrate use of local, static surrogates in structured NESTs• discover local, static surrogates in structured NESTs• determine local and global static thresholds• elaborate network generator to produce more sophisticated NESTs
C I R LC I R L
Specific Milestones
1. Random NESTs/synthetic properties1. develop parameterized network
generator
2. Identify static properties of interest
3. Discover surrogatesa) Identify globally observable surrogates
b) Identify local surrogates
4. Determine thresholdsa) for global surrogates
b) for local surrogates
5. Build demonstration systemsa) use global surrogates to predict
behavior
b) use of local surrogates
Legend:
On schedule
Partially completed
Delayed
2. Structured NESTs/synthetic properties1. Develop network generator based on
community specifications/needs
2. Identify functional properties
3. Discover surrogatesa) Discover global surrogates
Notes: • As described previously, efforts on
1.3.b, 1.4.b, and 1.5.b were suspended in order to more quickly move the project to domains of practical interest to the NEST community.
• Continuing work is expected to be done to make the system developed in 2.1 more widely useful to the community.
C I R LC I R L
Technology Transition/Transfer
• N/A
C I R LC I R L
Program Issues
• N/A
C I R LC I R L
Technical Progress Summary
• Modeling of realistic sensor deployments– developed simulator for experiments– identified transition behavior in big-brother problem
• Surrogates for connectivity– pseudo-density controls many measures of interest
• Fairly robust routing– twisted trees: minimal disjoint spanning trees surrogate
• Group-theoretic structure exploitation (ZAP)– exponential simplification of certain search problems– potential for automatic generation of surrogates
• by simplification of generators for relevant groups
C I R LC I R L
The Overall Plan
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
C I R LC I R L
The Overall Plan
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: satisfiability in random 3-SAT– Must be possible to evaluate surrogate quickly
C I R LC I R L
The Overall Plan
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: satisfiability in random 3-SAT– Must be possible to evaluate surrogate quickly– Quality/feature being modeled must have real
operational use
C I R LC I R L
The Overall Plan
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: satisfiability in random 3-SAT– Must be possible to evaluate surrogate quickly– Quality/feature being modeled must have real
operational use– Feature being modeled must be beyond the reach
of existing capabilities
C I R LC I R L
The Overall Plan
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: satisfiability in random 3-SAT– Must be possible to evaluate surrogate quickly– Quality/feature being modeled must have real
operational use– Feature being modeled must be beyond the reach
of existing capabilities
C I R LC I R L
The Overall Plan
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: satisfiability in random 3-SAT– Surrogate is clause/variable ratio– Quality/feature being modeled must have real
operational use– Feature being modeled must be beyond the reach
of existing capabilities
C I R LC I R L
The Overall Plan
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: satisfiability in random 3-SAT– Surrogate is clause/variable ratio– Satisfiability is the coin of the realm in this domain
ffff– Feature being modeled must be beyond the reach
of existing capabilities
C I R LC I R L
The Overall Plan
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: satisfiability in random 3-SAT– Surrogate is clause/variable ratio– Satisfiability is the coin of the realm in this domain
ffff– 3-SAT is NP-complete
C I R LC I R L
Multipath Routing: July
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: satisfiability in random 3-SAT– Must be possible to evaluate surrogate quickly– Quality/feature being modeled must have real
operational use– Feature being modeled must be beyond the reach
of existing capabilities
C I R LC I R L
Multipath Routing: July
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: biconnectedness– Must be possible to evaluate surrogate quickly– Quality/feature being modeled must have real
operational use– Feature being modeled must be beyond the reach
of existing capabilities
C I R LC I R L
Multipath Routing: July
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: biconnectedness– Density as a surrogate (V2/N)– Quality/feature being modeled must have real
operational use– Feature being modeled must be beyond the reach
of existing capabilities
C I R LC I R L
Multipath Routing: July
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: biconnectedness– Density as a surrogate (V2/N)– Quality/feature being modeled must have real
operational use– Not hard! Vijay: linear-time algorithms exist
C I R LC I R L
Multipath Routing: July
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: biconnectedness– Density as a surrogate (V2/N)– Not useful, either!
– Not hard! Vijay: linear-time algorithms exist
C I R LC I R L
Multipath Routing: July
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: biconnectedness– Density as a surrogate (V2/N)– Not useful, either! Long paths
– Not hard! Vijay: linear-time algorithms exist
C I R LC I R L
Multipath Routing: July
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: biconnectedness– Density as a surrogate (V2/N)– Not useful, either! Long paths, impossible to find
– Not hard! Vijay: linear-time algorithms exist
C I R LC I R L
Multipath Routing: July
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: biconnectedness– Density as a surrogate (V2/N)– Not useful, either! Long paths, impossible to find
and who cares about disjointedness anyway?– Not hard! Vijay: linear-time algorithms exist
C I R LC I R L
Multipath Routing: December
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: ability to find multiple, short, nearly disjoint routings
C I R LC I R L
Multipath Routing: December
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: ability to find multiple, short, nearly disjoint routings– computationally viable surrogate
C I R LC I R L
Multipath Routing: December
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: ability to find multiple, short, nearly disjoint routings– density (V2/N) still works
C I R LC I R L
Multipath Routing: December
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: ability to find multiple, short, nearly disjoint routings– density (V2/N) still works– problem has to be hard
C I R LC I R L
Multipath Routing: December
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: ability to find multiple, short, nearly disjoint routings– density (V2/N) still works– problem is known to be NP-hard (disjoint case)
C I R LC I R L
Multipath Routing: December
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: ability to find multiple, short, nearly disjoint routings– density (V2/N) still works– problem is known to be NP-hard (disjoint case)– has to be useful
C I R LC I R L
Multipath Routing: December
• Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly
• Example: ability to find multiple, short, nearly disjoint routings– density (V2/N) still works– problem is known to be NP-hard (disjoint case)– clear impact on both robustness and power
consumption
C I R LC I R L
Multipath Routing: December
• Biconnectedness is too easy• Robust routing is too hard• Fairly robust routing is just right
• And curiously enough, V2/N is a surrogate for all of them
C I R LC I R L
Fairly Robust Routing
• Goal: “approximately disjoint” spanning trees– low cost to construct (time/communication)– short– well dispersed
• Approach: exploit weak localization information– augment standard flood fill with directional bias– produce a loose “spiral” in toward root
• Results: near-shortest-path trees– good spatial dispersion– low cost to construct– no spanning tree needed (extreme scalability)
C I R LC I R L
Twisted Trees
• Angular bias applied to flood fill algorithm– based on angle between arcs to neighbor and root– for a 30º left bias, choose neighbor bearing closest
to 30º left of the bearing to the root
• Works in high surrogate density (V2/N) regions• Serial complexity is O(V+E)
– makes efficient use of distribution of sensors– conjecture: O(log(V)) parallel complexity
• Experimentally validated, proofs in progress– good spatial separation (not perfect!)– short paths (small multiplier of optimal)
C I R LC I R L
Gauss network graph
C I R LC I R L
Shortest path routing tree
C I R LC I R L
Boundary edges for router
C I R LC I R L
Twisted routing tree
C I R LC I R L
Multipath route
C I R LC I R L
Crop-duster network graph
C I R LC I R L
Shortest-path routing tree
C I R LC I R L
Boundary edges for router
C I R LC I R L
Twisted routing tree
C I R LC I R L
Multipath route
C I R LC I R L
A Common Surrogate
• Biconnectedness• Robust routing• Fairly robust routing
• V2/N is a surrogate for all of them
C I R LC I R L
A Common Surrogate
• Biconnectedness• Robust routing• Fairly robust routing
• V2/N is a surrogate for all of them• How come?
C I R LC I R L
• Continuous analysis yields order parameters:– critical clustering density:
– fraction of reachable nodes:
– surrogate density that controls reachability:
Predicting Coverage
€
€
ρC
= 1− e−R2 /2V 2 ⎛
⎝
⎜ ⎜ ⎜
⎞
⎠
⎟ ⎟ ⎟
€
f = 1 −2 πρCV
2
N
€
V 2
N
C I R LC I R L
Experimental Validation
Connected fraction tracks prediction well
C I R LC I R L
Sensor coverage predictions
C I R LC I R L
Sensor coverage predictions
C I R LC I R L
A Common Surrogate
• Biconnectedness• Robust routing• Fairly robust routing
• V2/N is a surrogate for all of them• Why?
C I R LC I R L
A Common Surrogate
• Biconnectedness• Robust routing• Fairly robust routing
• V2/N is a surrogate for all of them• Because Gaussian distributions are benign
C I R LC I R L
A Common Surrogate
• Biconnectedness• Robust routing• Fairly robust routing
• V2/N is a surrogate for all of them• Because Gaussian distributions are benign• Suggests existence of other surrogates here:
C I R LC I R L
A Common Surrogate
• Biconnectedness• Robust routing• Fairly robust routing
• V2/N is a surrogate for all of them• Because Gaussian distributions are benign• Suggests existence of other surrogates here:
– what is needed for exfiltration?
C I R LC I R L
A Common Surrogate
• Biconnectedness• Robust routing• Fairly robust routing
• V2/N is a surrogate for all of them• Because Gaussian distributions are benign• Suggests existence of other surrogates here:
– what is needed for exfiltration?– how to manage sentry/relay tradeoff?
C I R LC I R L
A Common Surrogate
• Biconnectedness• Robust routing• Fairly robust routing
• V2/N is a surrogate for all of them• Because Gaussian distributions are benign• Suggests existence of other surrogates here:
– what is needed for exfiltration?– how to manage sentry/relaysleep tradeoff?– extreme scalability: what will work?
C I R LC I R L
A Common Surrogate
• Biconnectedness• Robust routing• Fairly robust routing
• V2/N is a surrogate for all of them• Because Gaussian distributions are benign• Suggests existence of other surrogates here:
– what is needed for exfiltration?– how to manage sentry/relay tradeoff?– extreme scalability: what will work?– surrogates avoid guesswork
C I R LC I R L
Next Steps
• Assist in exploitation of existing surrogates– twisted path implementation, etc.
• Participate in the extreme scaling effort– develop new surrogates in specific areas as needed– exfiltration, sentry/relay/sleep issues, scalability, etc.
• Develop tools to find surrogates in general case– as originally proposed, but harder than expected– general tools for exploiting structure (ZAP)