universal laws and architecture: challenges for sustainable infrastructure

Universal laws and architecture:Challenges for Sustainable Infrastructure

John DoyleJohn G Braun Professor

Control and Dynamical Systems, EE, BioECaltech

“Universal laws and architectures?”

• Universal “conservation laws” (constraints)• Universal architectures (constraints that deconstrain)• Mention recent papers*• Focus on broader context not in papers• Lots of theorems• Case studies: evolution, physiology, bacterial biosphere, ,

glycolytic oscillations, Internet/IT, neuroscience, smartgrid, aerospace, wildfire ecology, turbulence, stat mech, earthquakes, heart rate variability

*try to get you to read them?

Collaborators and contributors(partial list, out of date,…)

Theory: Parrilo, Carlson, Murray, Vinnicombe, Paganini, Papachristodoulou, Prajna, Goncalves, Fazel, Liu, Lall, D’Andrea, Jadbabaie, Dahleh, Martins, Recht, many more current and former students, …

Biology: Chandra, Buzi, Csete,Yi, El-Samad, Khammash, Tanaka, Arkin, Savageau, Simon, Gross, Kitano, Hucka, Gillespie, Petzold, F Doyle, Stelling, Caporale,…

Web/Internet: Chen, Low, Lavaei, Sojoudi, Li, Alderson, Willinger, Kelly, Zhu,Yu, Wang, Chandy, Trossen, Griffin,…

Turbulence: Gayme, McKeon, Bamieh, Bobba, Gharib, Marsden, …Physics: Sandberg, Delvenne, Barahona, Carlson, Asimakopoulos,

Matni,…Disturbance ecology: Moritz, Carlson,…Neuroscience: Lamperski, Grafton, Gazzaniga, Mitra,…

Current Caltech Former Caltech OtherLongterm Visitor

Thanks to

• NSF• ARO• ONR • Braun family• Lee Center for Advanced Networking (Caltech)• Philips

• NIH/NIGMS? AFOSR? DARPA?

• Special thanks to Hiroaki Kitano (ERATO)

Happy families are all alike; every unhappy family is unhappy in its own way.

Leo Tolstoy, Anna Karenina,

Chapter 1, first line

• What does this even mean? • Given incredible diversity of people and environments?• It has to be a statement about organization.

• Happy family = empathy + cooperation + simple rules?• Constraints on components and architecture

wasteful

fragile

efficient

robust


Want robust and efficient systems and architectures

Are robust, efficient systems/architectures

“all alike”?

accessibleaccountableaccurateadaptableadministrableaffordableauditableautonomyavailablecredibleprocess

capablecompatiblecomposable configurablecorrectnesscustomizabledebugabledegradabledeterminabledemonstrable

dependabledeployablediscoverable distributabledurableeffectiveefficientevolvableextensiblefailure transparentfault-tolerantfidelityflexibleinspectableinstallableIntegrityinterchangeableinteroperable learnablemaintainable

manageablemobilemodifiablemodularnomadicoperableorthogonalityportableprecisionpredictableproducibleprovablerecoverablerelevantreliablerepeatablereproducibleresilientresponsivereusable robust

safety scalableseamlessself-sustainableserviceablesupportablesecurablesimplicitystablestandards

compliantsurvivablesustainabletailorabletestabletimelytraceableubiquitousunderstandableupgradableusable

Requirements on systems and architectures

happy?

accessibleaccountableaccurateadaptableadministrableaffordableauditableautonomyavailablecredibleprocess

capablecompatiblecomposable configurablecorrectnesscustomizabledebugabledegradabledeterminabledemonstrable

dependabledeployablediscoverable distributabledurableeffectiveefficientevolvableextensiblefailure transparentfault-tolerantfidelityflexibleinspectableinstallableIntegrityinterchangeableinteroperable learnablemaintainable

manageablemobilemodifiablemodularnomadicoperableorthogonalityportableprecisionpredictableproducibleprovablerecoverablerelevantreliablerepeatablereproducibleresilientresponsivereusable robust

safety scalableseamlessself-sustainableserviceablesupportablesecurablesimplicitystablestandards

compliantsurvivablesustainabletailorabletestabletimelytraceableubiquitousunderstandableupgradableusable

Requirements on systems and architectures

wasteful

fragile

efficient

robust

wasteful

fragile

efficient

robust


Want robust and efficient systems and architectures

In what sense are robust, efficient systems/architectures

all alike?

inefficientwasteful

weak fragile

efficient(slow)

strongrobust

Biology

Human evolution Apes

feetskeletonmuscleskinguthands

inefficientwasteful

weak fragile

efficient(slow)

strongrobust

Biology

Hard tradeoffs?

Apes

Architecture?

inefficientwasteful

weak fragile

efficient(slow)

strongrobust

Biology

sticksstonesfire

+Technology

inefficientwasteful

weak fragile

efficient(slow)

strongrobust

Biology+Technology

++Technology

wasteful

fragile

efficient

robust Hard tradeoffs?

Architecture?

Constraints(that deconstrain)

wasteful

fragile

efficient

robust

Next 3 speakers

Biology

sticksstonesfire

+Technology

feetskeletonmuscleskinguthands

Human complexity?

wasteful

fragile

efficient

robust

Robust Fragile

Human complexity

Metabolism Regeneration & repair Healing wound /infect

Obesity, diabetes Cancer AutoImmune/Inflame

Start with physiology

Lots of triage

Robust Fragile

Mechanism?


Fat accumulation Insulin resistance Proliferation Inflammation



Robust Fragile

What’s the difference?




ControlledDynamic

UncontrolledChronic

ControlledDynamic

UncontrolledChronic

Low meanHigh variability

High meanLow variability


Death

Robust Fragile

Restoring robustness?




ControlledDynamic

UncontrolledChronic

Low meanHigh variability

High meanLow variability


Robust Fragile Metabolism Regeneration & repair Healing wound /infect



• Fragility Hijacking, side effects, unintended… • Of mechanisms evolved for robustness • Complexity control, robust/fragile tradeoffs• Math: robust/fragile constraints (“conservation laws”)

Accident or necessity?

Both

Human complexity?

wasteful

fragile

efficient

robust

Robust Yet Fragile

Human complexity

Metabolism Regeneration & repair Immune/inflammation Microbe symbionts Neuro-endocrine Complex societies Advanced technologies Risk “management”

Obesity, diabetes Cancer AutoImmune/Inflame Parasites, infection Addiction, psychosis,… Epidemics, war,… Disasters, global &!%$# Obfuscate, amplify,…

Accident or necessity?

In the real (vs virtual) world

What matters:

• Action

What doesn’t:

• Data

• Information

• Computation

• Learning

• Decision

• …

Don’t worry ...• “Like, dude, like, chill…” • “There’s an app for that.”• “The ‘new sciences’ of …”• “There’s a gene…• “The market will...”• “Order for free…”• “The rapture is near.”

Don’t worry ...• “Like, dude, like, chill…” • “There’s an app for that.”• “The ‘new sciences’ of …”• “There’s a gene…• “The market will...”• “Order for free…”• “The rapture is near.”

Come back to this later

IEEE TRANS ON SYSTEMS, MAN, AND CYBERNETICS, JULY 2010, Alderson and Doyle

Csete and Doyle

Feathers and

flapping? Or lift, drag, propulsion, and control?

The dangers of naïve biomemetics

Getting it (W)rightGetting it (W)right,, 1901 1901• “We know how to construct airplanes...” (lift and drag)• “… also know how to build engines.” (propulsion)• “Inability to balance and steer still confronts students of the flying problem.” (control)• “When this one feature has been worked out, the age of flying will have arrived, for all other difficulties are of minor importance.”

Wilbur Wright on CWilbur Wright on Control,ontrol, 1901 1901

Getting it rightGetting it right,, 2011 2011• “...know how to construct sustainable infrastructures...”• “… also know how to build their components.”• “Inability to control and manage fragilities ....” • “When this one feature has been worked out, the age of sustainability will have arrived, for all other difficulties are of minor importance.”

Fragilities? • Unintended crashes, hijacking, parasitism, evolution• Need robust, efficient, evolvable architectures• Policy trumps technology (next talks)• Aligning incentives (next talks)

wasteful

fragile

efficient

robust

Hard tradeoffs?

Chandra, Buzi, and Doyle

simple enzyme

Fragility

Metabolic Overhead

complex enzyme

lnz p

z p

2 20

1ln ln

z z pS j d

z z p

Theorem!

Glycolytic “circuit” and oscillations

• Perfect circuit case study – Every cell (1030), heavily studied– Experiments, models, simulation, …, all “well-known”

• Oscillations?– Remain persistent mystery (decades,…?)– Frozen accident? Edge of chaos? Emergulence?

• New insight: constraints and tradeoffs– “Universal” robustness/efficiency tradeoff – Evolution + physiology + “CDS” theory– Issues & theory: broadly relevant and “universal”

• Extreme responses typical

ubiquitous


• End of an old story (why oscillations)– no purpose per se– side effect of hard robustness/efficiency tradeoffs– just needed a theorem

• Beginning of a new one – robustness/efficiency tradeoffs– complexity and architecture– need more theorems and applications

robust

efficient wasteful

fragile

Tradeoffs?

Hard limit

Robust=maintain energy level w/fluctuating demand

Efficient=minimize metabolic overhead

Want robust and efficient

Control, OR Comms

Compute Physics

ShannonBode

TuringGodel

EinsteinHeisenberg

Carnot

Boltzmann

Theory?Deep, but fragmented, incoherent, incomplete

Nash

Von Neumann

KalmanPontryagin

Control Comms

Compute Physics

ShannonBode

TuringGodel

Einstein

Heisenberg

Carnot

Boltzmann

wasteful?

fragile?

slow?

?

• Each theory one dimension• Tradeoffs across dimensions• Assume architectures a priori• Progress is encouraging, but…

.1%

1%

10%

100%

http://phe.rockefeller.edu/Daedalus/Elektron/

F Efficiency

When will steam engines be 200% efficient?

Exponential improvement

Note: this is real data!

.1%

1%

10%

50%

http://phe.rockefeller.edu/Daedalus/Elektron/

When will steam engines be 200% efficient?

1

F

F

F Efficiency

Oops… never.

wasteful

fragile

robust

efficient

At best we get one

Technology?

wasteful

fragile

robust

efficient

Often neither

???

Bad theory?

???

?

?

Bad architectures?

wasteful

fragile

gap?

robust

efficient

Case studies

wasteful

fragile

Sharpen hard bounds

Hard limit

Conservation “laws”?

simple enzyme

Fragility

Overhead

complex enzyme

lnz p

z p

2 20

1ln ln

z z pS j d

z z p

Theorem!

z and p functions of enzyme complexity and

amount

What reviewers say• “The approach to establish universality for all biological and

physiological systems is simply wrong. It cannot be done…” • “…does not seem to have an understanding or appreciation of

the vast diversity of biological and physiological systems…” • “… a mathematical scheme without any real connections to

biological or medical problems…” • “…desire to develop rigorous framework is understandable, but

usually this can be done only by imposing a high degree of abstraction, which would then make the model useless …”

• “While the notion of universality is well justified in physics, it is perhaps not so useful in biological sciences and medicine. To develop a set of universal principles for biological and physiological systems is mostly likely a dream that will never be realized, due to the vast diversity in such systems.”


• End of an old story (why oscillations)– no purpose per se– side effect of hard robustness/efficiency tradeoffs– just needed a theorem

• Beginning of a new one – robustness/efficiency tradeoffs– complexity and architecture– need more theorems and applications

wasteful

fragile

efficient

robust

Hard tradeoffs?

Architecture?

TCPIP

Physical

MACSwitch

MAC MACPt to Pt Pt to Pt

Diverse applications

Layered architectures

Proceedings of the IEEE, Jan 2007

Chang, Low, Calderbank, and Doyle

TCPIP

Physical

Diverse applications

Too good?

Diverse

TCPIP

Deconstrained(Hardware)

Deconstrained(Applications)


ConstrainedNetworks

“constraints that deconstrain” (Gerhart and Kirschner)

OS




ConstrainedPCs

“constraints that deconstrain” (Gerhart and Kirschner)

OS




Constrained Control, share, virtualize, and manage resources

ProcessingMemoryI/O

Few global variablesFew global variables

Don’t cross layersDon’t cross layers

TCP/IP





Processing?Memory?I/OCommsLatency?

Few global variables?Few global variables?

Don’t cross layers?Don’t cross layers?

Cata

bolis

m

AA

Ribosome

RNA

RNAp

transl.Proteins

xRNA transc.

Prec

urso

rs

Nucl.

AA

DNA

DNAp

Repl. Gene

ATP

ATP

Enzymes

Building Blocks

Shared protocols

Deconstrained (diverse)

Environments

Deconstrained (diverse) Genomes

Bacterial biosphere

Architecture =

Constraints that

Deconstrain


Cat

abol

ism

AA

Ribosome

RNARNAp

transl.Proteins

xRNAtransc.

Pre

curs

ors

Nucl.

AA

DNADNAp

Repl. Gene

ATP

ATP

Enzymes

Building Blocks

Crosslayer autocatalysis

Macro-layers

Inside every cellalmost

Cata

bolis

m

AA

Ribosome

RNA

RNAp

transl.Proteins

xRNA transc.

Prec

urso

rs

Nucl.

AA

DNA

DNAp

Repl. Gene

ATP

ATP

Enzymes

Building Blocks

Core conserved constraints facilitate

tradeoffs

Deconstrained phenotype

Deconstrained genome

What makes the bacterial biosphere so adaptable?

Active control of the genome (facilitated variation)

Environment

Action

Layered architecture

This paper aims to bridge progress in neuroscience involving sophisticated quantitative analysis of behavior, including the use of robust control, with other relevant conceptual and theoretical frameworks from systems engineering, systems biology, and mathematics.

Doyle and Csete, Proc Nat Acad Sci USA, online JULY 25 2011

Meta-layers

Physiology

Organs

Prediction GoalsActions

errorsActions

Cor

tex

Fast,Limited scope

Slow,Broad scope

Which blue line is longer?

“Seeing is dreaming?”

“Seeing is believing?”

sense

move Spine

delay=death

sense

move Spine

Reflex

Reflect

sense

move Spine

Reflect

Reflex

Layered

Physiology

Organs

Neu

rons

Neu

rons

Neu

rons

Cor

tex

Cel

ls

Cor

tex

Cor

tex


Cells

Physiology

Organs

Meta-layers

Prediction

GoalsActions

errors

ActionsCo

rtex

Simulation

Seeing is dreaming

Consciousperception

Consciousperception

Physiology

Organs

Prediction

GoalsActions

errors

Actions

Seeing is believing

Consciousperception

Prediction

Goals

Consciousperception

Seeing is dreaming

sourcereceiver

signalinggene expression

metabolismlineage

Biological pathways

sourcereceiver

control

energy

materials

signalinggene expression

metabolismlineage

More complex

feedback

sourcereceiver

control

energymaterials

Physiology

Organs

Prediction

GoalsActions

errors

Actions

Prediction

Goals

Consciousperception

fast

fast

Meta-layers

Physiology

Organs


errorsActions

Cor

tex

Fast,Limited scope

Slow,Broad scope

Unfortunately, we’re not sure how this all works.

Meta-layers

Physiology

Organs


errorsActions

Cor

tex

Fast,Limited scope

Slow,Broad scope




Meta-layers

Physiolog y

Organs

Pre

dic t io

n

Goal s

Actio

ns

er rors

Actio

ns

Cortex

Fast,Limited scope

Slow,Broad scope

UAV

Com

ms

Meta-layers

Physiolog y

Organs

Pre

dic t io

n

Goal s

Actio

ns

er rors

Actio

ns

Cortex

Fast,Limited scope

Slow,Broad scope

Dis

turb

ance

Plant

RemoteSensor

Sensor

Actuator

Interface

Control


Com

ms D

istu

rban

ce

Plant

RemoteSensor

Sensor

Actuator

Interface

Control


Com

ms D

istu

rban

ce

Plant

RemoteSensor

Sensor

Actuator

Interface

Control

?



Next layered architectures

Constrained

Control, share, virtualize, and manage resources

Other examples

Clothing

Lego

Money

Cell biology

OutfitBody Environment

Shirt

Slacks

JacketTie T-Shirt

Socks

Shoes CoatShorts


• Complexity Robustness • Layers must be hidden to be robust• Choice (management and control) is more complex than assembly

Mgmt/ctrl

Assembly

Outfit

Outfit

Cloth

Thread

Fiber

Garment

Cloth

Thread

Fiber

Garment

Cloth

Thread

Fiber

Garment

Layering within garments (textiles)

Cloth

Thread

Fiber

Garments

Cloth

Thread

Fiber

Garments

Weave

Sew

Spin

Universal strategies?

Prevents unraveling of lower layers

Cloth

Thread

Fiber

Garments

Xform

Xform

Xform

Universal strategies?

Garments have limited access to

threads and fibers

constraints on cross-layer interactions

quantization for robustness

Even though garments seem analog/continuous

Prevents unraveling of lower layers

Cloth

Thread

Fiber

Garments

Xform Ctrl Mgmt

Networked, universal,

layeredXform Ctrl Mgmt

Xform Ctrl Mgmt

Xform Ctrl Mgmt

Co

ntro

l

Su

pp

lyComplexity?

Fiber

Geographically diverse sources

Diverse fabric

Functionally diverse garments

General purpose machines Diverse Thread

sew

knit, weave

spin

OS





ProcessingMemoryI/O


Don’t cross layersDon’t cross layersDirect

access to physical

memory?

Cat

abol

ism

AA

Ribosome

RNA

RNAp

transl.Proteins

xRNA transc.

Pre

curs

ors

Nucl.

AA

DNA

DNAp

Repl.Gene

ATP

ATP

Enzymes

Building Blocks

Shared protocols

Deconstrained (diverse)

Environments

Deconstrained (diverse) Genomes

Bacterial biosphere

Architecture =

Constraints that

Deconstrain



Meta-layers

Physiology

Organs

Prediction

Goals

Actions

errors

Actions

Cortex

Fast,Limited scope

Slow,Broad scope






Com

ms

Meta-layers

Physiolog y

Organs

Pre

dic t io

n

Goal s

Actio

ns

er rors

Actio

ns

Cortex

Fast,Limited scope

Slow,Broad scope

Dis

turb

ance

Plant

RemoteSensor

Sensor

Actuator

Interface

Control




Problems with leaky layering

Modularity benefits are lost• Global variables? @$%*&!^%@& • Poor portability of applications• Insecurity of physical address space• Fragile to application crashes• No scalability of virtual/real addressing

• Limits optimization/control by duality?

Fragilities of layering/virtualization

• Hijacking, parasitism, predation– Universals are vulnerable– Universals are valuable

• Breakdowns/failures/unintended/… not transparent

• Hyper-evolvable but with frozen core

TCP/IP


Deconstrained(Applications)Original design challenge?

Constrained

• Trusted end systems• Unreliable hardware

Facilitated wild evolutionCreated

• whole new ecosystem• complete opposite

TCP/IP





Processing?Memory?I/OCommsLatency?

Few global variables?Few global variables?

Don’t cross layers?Don’t cross layers?

App AppIPC

Global and direct access to

physical address!

Robust?• Secure• Scalable• Verifiable• Evolvable• Maintainable• Designable• …

DNS

IP addresses interfaces

(not nodes)

IP addresses interfaces

(not nodes)

Physical

IP

TCP

Application

Naming and addressing need to be • resolved within layer• translated between layers• not exposed outside of layer

Related “issues”• VPNs• NATS• Firewalls• Multihoming• Mobility• Routing table size• Overlays• …

?



Next layered architectures


CommsMemory, storageLatencyProcessingCyber-physical



Every layer has

different diverse graphs.

Architecture is least graph topology.

Architecture facilitates arbitrary graphs.

Persistent errors and confusion (“network science”)

Physical

IP

TCP

Application

Notices of the AMS, 2009

wasteful

fragile

slowGood case studies

Hard limit

bad

worse

Fix bugs

“New sciences” of “complexity” and “networks”?

D. Alderson, NPS 122

“New sciences” of “complexity” and “networks”? worse

• Edge of chaos• Self-organized criticality• Scale-free “networks”• Creation “science”• Intelligent design• Financial engineering• Risk management• “Merchants of doubt”• …

Not today

Science as • Pure fashion• Ideology• Political• Evangelical• Nontech trumps tech

IEEE TRANS ON SYSTEMS, MAN, AND CYBERNETICS, JULY 2010, Alderson and Doyle

Statistical physics

Complex networks

edge of chaos, self-organized criticality, scale-free,…

Complex systems?

Fragile

• Scale• Dynamics• Nonlinearity• Nonequlibrium• Open• Feedback• Adaptation• Intractability• Emergence• …

Even small amounts can create bewildering complexity

Complex systems?

Fragile



Robust

Complex systems?

• Resources• Controlled• Organized• Structured• Extreme• Architected• …

Robust complexity


New words

Fragile complexity

Emergulent

Emergulence at the edge of

chaocritiplexity


“New sciences” of complexity and networks

Statistical physics

Complex networks

edge of chaos, self-organized criticality, scale-free,…

IEEE TRANS ON AUTOMATIC CONTROL, FEBRUARY, 2011Sandberg, Delvenne, and Doyle

http://arxiv.org/abs/1009.2830

Stat physics, fluids, QM

Complex networks

“orthophysics”

J. Fluid Mech (2010)

Transition to Turbulence

FlowStreamlinedLaminar Flow

Turbulent Flow

Increasing Drag,

Fuel/Energy Use and

Cost

Turbulence and drag?

Physics of Fluids (2011)

wU

z x

y

uzx

yFlow

upflowhigh-speed

region

downflowlow speed

streak

Blunted turbulent velocity profile

Laminar

Turbulent

wU3D coupling

Coherent structures and turbulent drag

wasteful

fragile Laminar

Turbulent

efficient

robust


Laminar

Turbulent

wU

?

Transition to Turbulence

FlowStreamlinedLaminar Flow

Turbulent Flow

Increasing Drag,

Fuel/Energy Use and

Cost

Turbulence and drag?

uzx

yFlow

Coherent structures

wU

z x

y

wU

z x

y


Laminar

Turbulent

wU

0u

1uu u p u

t R

“turbulence is a highly nonlinear

phenomena”

0u

1uu u p u

t R

Small Large

RobustSimple

2d, linearOrganizedComputer

Fragilechaocritical3d, nonlinear

Irreducibile?

Complexity?

mildly nonlinear

highly nonlinear

Model

wasteful

fragile Laminar

Turbulent

efficient

robust

Laminar

Turbulent

wU

?Control?

Supplementary materials has a demo.

Doyle and Csete, Proc Nat Acad Sci USA, online JULY 25 2011

m

M

L

Fragility

up + eyes

lnz p

z p

2 20

1ln ln

z z pS j d

z z p

Theorem

up, no eyes

This is a cartoon, but can be made

precise.

L

hopeless

down

lower focus

L

u

x

m

M

m

M

Linearized pendulum on a cart

0

1ln 0S j d

Easy, even with eyes closedNo matter what the length

0

ln 0

Gratuitous fragility

Fragile robustness

S j d

Gratuitous fragility versus

fragile robustness

0

1ln S j d p

1 1g

p z r rL

small largep L

Up is hard for shorter lengths

Down easy, even with • eyes closed• all lengths

Fragility

complex


precise.

L

0

1ln S j d p

1p

L

L

Too fragile Why oscillations?

Side effects of hard tradeoffs

L

1

1 1

1 1

g mz p z r r

L M

p z r

p z r

m

M

Eyes closed

2 20

1ln ln

z z pS j d

z z p

Want r and z large (but p small).

Fragility

up + eyes

lnz p

z p

2 20

1ln ln

z z pS j d

z z p

Theorem

up, no eyes


precise.

L

hopeless

down

lower focus

universal laws and architecture: challenges for sustainable infrastructure

Documents

unhappy family

dynamical systems

bioecaltechuniversal

wildfire ecology

efficient systemsarchitectures

disturbance ecology

architecturesare robust

stat mech