Risk, Control and Complex

Systems

Dr. Paul Nightingale

CoPS/SPRU

Risk and System Failures

Thule (Greenland) Nuclear Alert

October 5th 1960 Colorado Springs

– North American Air Defense headquarters

– Ballistic Missile –Early Warning System - Thule

Greenland

5* - 99.9% certainty of ballistic missile attack

– Strategic Air Command Nebraska Readied

Risk and System Failures

But, Khruschchev in New York?

No ground bursts?

Software upgraded 4 days ago?

Radar detected moon rising over Norway

– Rather than one slow moving object

– Radar signals bounced 250,000 miles

– Detected as multiple fast moving objects - launches

Cuban Missile Crisis

1962 Soviet submarine B-59 vs USS Beale

– Depth charges vs nuclear torpedoes

– Crew thought the war had started

Officer Valassy

Risk and Scientific System

CERN - Martin Rees

1. Create a black hole

– Destroy earth

2. Create a „strangelet‟ of uncompressed quarks

– Earth reduced to hyper-dense sphere 100m across

3. Present vacuum might be unstable

– Universe undergoes a phase change

In the history of universe - happened naturally?

Risk and Scientific System

Trinity 1945 – Oppenheimer‟s chain reaction

Dual Use

– Cello et al („02) polio cDNA synthesis,

– Rosengrad et al („02) variola immune,

– Jackson et al („01) mousepox immune evasion,

– Parkhill et al (‟04) pathogen genomes on web

Structure of Lecture

Unit of analysis – Large Technical Systems

Crises of control

– Increase: scale, energy, complexity, & speed

Controlling LTS - the management of risk

– Efficiency-reliability tradeoff

– Normal accidents vs. high reliability

Large Technical Systems

Infrastructure Large technicalsystem

Physical &informationalflows

Power Electric power, gas,oil pipeline, nuclearpower, etc.

Transmission ofpower

Transportation Railways, urbantramways, ships,road, air, elevators

Movement of goods,people andmessages

Communication Postal, telegraph,telephone, radio,TV, computers,internet

Processing &transmission ofinformation

Key LTS characteristics

Components

– systemically related components

– technical & institutional

Network structure

– components are systemic - connected

– „change one component effects others‟

Control systems

– Systems often centrally controlled to meet system goals:

• a) optimize system performance (efficiency)

• b) maintain system reliability (e.g. safety)

Efficiency & Capacity Utilisation

Capital Intensive

High fixed costs, low variable costs

Profits depend on capacity utilisation ( ROI)

Capacity - how much produced in time t

Utilisation - percent of capacity used

Load factor

Speed up (increase energy) (David Landes)

Expand to fill gaps in demand - growth

Control Systems

Increase complexity & speed

Control - change actual to desired behaviour

Closed Loop Control:

– monitor outputs, compare actual with desired,

feedback to change system

– model embodies theory about inputs-outputs link

Innovation in Control Systems

Accuracy of Model

– Better match between desired and actual behaviour

Speed of calculation and control

– Systems‟ change during control - feedback

Scope of coverage

– More optimal but takes longer

Reliability - critical systems

– Preventative maintenance, redundancy, backup systems

Modernity, Systems and Control

Mid-19th century - systemness of society

increased

– railways, telegraph, electricity broke spatial barriers

„Suddenly, in a matter of decades, goods began to

move faster,…reliably and in mounting volume, through

factories, across continents, and around the world. For

the first time in history,…material flows threatened to

exceed in both volume and speed the system‟s capacity

to contain them. Thus was born the crisis of control’

(Beniger, p219, 1986)

Crises of control induced IT innovation

increasing scale &

volume/speed of flows

crisis of control

innovation in

control technology

increasing scale &

volume/speed of flows

crisis of control

Dynamic tension between crisis and control

Control innovation to cope with increasing size and

complexity of systems, and speed and volume of traffic flows

Safety crisis 1840s - Railways

Western Railroads collision 5 October 1841 – 2 dead, 8 seriously injured, Worcester & Massachusetts line

Accident blamed on failure of control – despite precise scheduling & contingency procedures

– the conductor failed to examine his timetable

Control of Western line centralised in company HQ – lines of authority & command linked to 3 regional divisions

– data collection; formal rules; standardised communication

Distributed control of fast-moving system‟s flows – conductor now controlled train from origin to destination

– first time people used as distributed decision makers

Efficiency crisis 1850s

Control problems shifted from safety to efficiency

– But „as systems grew larger, per-mile operating costs actually

increased‟ because of problem of „keeping track of trains, cars, and

personnel in increasingly large and busy systems‟

Control crisis of Erie Railroad 445 mile trunk line

– 1851 how to manage 123 trains, 68 passenger cars & 1,373 freight

and baggage cars (av. Speed at 24-29 miles per hour)

From increasing inefficiencies to improved utilisation

– Solved by organisational changes - hierarchical communication:

reporting system - lines of feedback & control – telegraph (1844)

Crises of Production 1860s

1860s: Increased energy use – (Bessemer)

Innovations in the internal co-ordination of materials

1870s “... unless the movement of trains and goods were carefully

monitored and co-ordinated, accidents occurred, lives were lost and

goods moved slowly and with uncertainty” – Chandler

New control metrics:

ton-mile, operating ratio, stock-turn (turnover to investment) earnings-

to-sales (effectiveness of operations) turn-over (flow of materials)

Crises of Consumption 1880s

Now had safe, efficient mass production and

distribution

Crises in consumption and marketing

– Innovations in mass media and advertising

– Radio adverts, department stores, mail order,

brochures, construction of demand

Modern crises of control

Crises of control continued in modern systems

– air traffic control; high-speed trains; electric power; digital

telecoms; intelligent buildings; investment banking; internet

Infrastructures upgraded with high-speed signaling and

computer control systems

Improvements in control involve

– balance between centralisation and decentralisation

– a tradeoff between efficiency and reliability

Software improves control but has its own problems

The problem of control - reliability

„increased control brought increased reliability and hence

predictability of processes and flows‟ (Beniger)

Risks high-volume, Tightly coupled complex systems

– no slack or buffer between components

– when what happens in one affects another

– little or no time for recovery from failure

„As systems grow in size and the number of diverse functions they

serve…they experience more and more incomprehensible or

unexpected interactions. They become more vulnerable to

unavoidable system accidents‟ (Perrow 1986)

Complex systems: non-linear interactions &

tight-coupling (Perrow)

Complex interactions „Complex systems tend to have

elaborate control centres…because components must

interact in more than linear, sequential ways, and

therefore may interact in unexpected ways‟

System accidents

– accident - failure in subsystem or system as a whole that disrupts

the output of the system

– system accident - multiple failures in components that interact in

unanticipated ways

– 1998 satellite failure – loss of pagers for police, doctors etc.

Figure 9. Airline network of Flyveselkap 1999

Figure 7. Airline network of Swissair 1999

System accidents in telecoms

„Modern network systems are very reliable, but when something fails, it fails spectacularly‟ (BT research manager)

Global telecom networks becoming unmanageable – mixing telephone, data and images, coupled with higher

usage will overload systems

– increasingly complex & tightly-coupled due to introduction of software-based controls

Software failure in AT&T network‟s (1990, 91 & 92) – Financial markets closed and planes were grounded

BT learnt by developing distributed control system – spread intelligence through the network

– identify and solve problems within fractions of a second

Debate: normal accidents vs high-

reliability organisations

Pessimistic view: Normal Accidents Theory

However much much you introduce improved controls,

complexly interactive & tightly-coupled systems will be

subject to system accidents (Perrow)

Optimistic view: High-Reliability Theory

Specifies the actions that organisations can take to

achieve high reliability (culture to promote learning,

shared experiences, stories & simulation)

Conclusions

Large Capital Intensive Systems

– Change Division of Labour in Society

– Increase output - generate economic returns

Economics driven by increases in

– Speed, Complexity, Size, Energy intensity……..

– All of which increase risk of failure

Control Technologies allow limits to be pushed

and have their own problems (software)

High reliability vs Normal Accidents vs Scale

Free

Conclusions

Systems growth

1) Concentration time/space of causes (plural)

2) Leverage time/space of effects

3) Co-ordinated (information) transfer

– i.e., explosives, production, drugs, systems

Specific - ‘Inert’ - CFCs, DDT, DNA

Complex, complicated - new DoL (Mode 2)

Change in the distribution of goods & bads

– Systemic & Technical

– Consensus on Goods and Bads?