gaura, brusey

Gaura, Brusey ISWC, Pittsburgh, 01/10/2008

Sensing and Actuation:End-to-end systems design for

safety critical applications

Dr. Elena Gaura, Reader in Pervasive ComputingDirector of Cogent Computing Applied Research Centre,

Coventry University,[email protected]

Dr. James Brusey, Senior Lecturer, [email protected]

mailto:[email protected]

mailto:[email protected]

Gaura, Brusey

Cogent Staff and PhD studentswww.cogentcomputing.org

Tessa Daniel [email protected]:Applicative Query Mechanisms; Information Extraction in Wireless Sensor Networks.

Mike Allen [email protected]:Design and Deployment of Wireless Sensor Networks; Distributed Embedded Sensing.

John Kemp [email protected]:Advanced Sensing; Sensing Visualisation Systems.

Dan Goldsmith [email protected]:Middleware design and test-beds for WSNs

Dr Elena Gaura [email protected]:Advanced Sensing; Advanced Measurement Systems; Ambient Intelligence; Design and Deployment of Wireless Sensor Networks; Distributed Embedded Sensing; Intelligent Sensors; Mapping Services for Wireless Sensor Networks; MEMS Sensors

Dr James Brusey [email protected]:Industrial Robotics and Automation; Machine Learning; RFID; Sensing Visualisation Systems.

Dr. James [email protected]:3D Graphics; data fusion and feature extraction, information visualization

Dr. Fotis [email protected]:Mixed reality systems; mobile computing, virtual reality for entertainment and education

Tony [email protected]:Wireless sensing for gas turbine engines

Ramona [email protected]:Body sensor networks,Posture

Michael [email protected]:3D CFD Modelling

Costa Mtagbe

Expertise:Environmental monitoring

Bremen, February 2009.

http://www.cogentcomputing.org/

http://www.cs.berkeley.edu/~culler/AIIT/print1-lectures/M1-intro.pdf


Talk Scope

• development cycle for a multi-modal wearable instrument

• system design decisions• embedding actuation and its

consequences• hurdles encountered….


Pointers• Timeliness: BSNs and WSNs are becoming

commercial in their simpler forms; also coming out of research labs in elaborate versions;

- Task Difficulty: Designing such systems needs teams of applications specialists, electronics engineers (most often) and definitely Computer Scientists;

- Usefulness: proven, but, apart from being very useful, BSNs are a lot of fun to develop!


Talk Structure

• Part 1: Introduction and overview of the application

• Part 2 : The deployment environment - a physiological perspective

• Part 3 : System design• Part 4 : Enabling actuation - on-body processing• Part 5 : Implementation - software and hardware

support• Part 6: Results analysis and evaluation


Part 1: Introduction and overview of the

application


WSNs: research motivationStart point:-Smart Dust (1998) – Pister ($35,000) vision of “millions of tiny wireless sensors (motes) which would fit on the head of a pin”

-sharing “intelligent” systems features (self –x) pushed to XLscale – millions of synchronized, networked, collaborative components

Today:-Dust Networks - $30 mil venture (2006);-TinyOS – the choice for 10000 developers-make the news and popular press- fashion accessory & easy lobbying- big spenders have committed already (BP, Honeywell, IBM, HP)-technologies matured (digital, wireless, sensors)-first working prototypes;-getting towards “out of the lab”-social scientists are getting ready!

Attention!Your spatio-temporal activities are recoded and analyzed by the 20000 sensors wide campus net


WSNs –realityMarket forecast:2014- $50bil. , $7bil in 2010 (2004)2014- $5-7 bil. sales (conservative)2011-$1.6 bil. smart metering/ demand response

Industrial Markets- old and new; mostly wired replacements; generally continuous monitoring systems with “data-made-easy” features and internet connected

Prompted by regulations and drive towards process efficiency or else…

the “cement motes” from Xsilogy come with 30 min warranty!

Connecting 466 foil strain gages to a wing box

Invensys asked a Nabisco executive what was the most important thing he wanted to know. The reply came without a moment's delay: "I'd like to know the moisture content at the centre of the cookie when it reaches the middle of the oven."

Research: mainly newly enabled applications; “macroscopes”/ “microscopes” ; adventurous money savings ideas

Infineon tyre sensor


WSNs - pushing the frontiers The motivational square

Practical, application oriented research and deployments

Theoretical research for large scale networks

Visions

Industrial needs

Research space

Research space

Making the most out of a bad situation

Commercial endeavours

Internet able Microclimate, soil moisture, disease monitoring

Research/Adoption roadblocks

Largest part of community

…forget about throwing them from the back of that plane!...


Why is it all so hard?…the WSN design space (Ray Komer, ETH, 2004)

deploymentmobilitycost, size, resources and energyheterogeneitycommunications modalityinfrastructurenetwork topologycoverageconnectivitynetwork sizelifetimeother QoS requirements

Highly theoretical worksVspractical deployments


WSN challenges

• Application specific (deployment, size, weight, etc)

• System specific – the network is the SENSOR– Distributed processing- system infrastructure– Information extraction– Scalability– Robustness

• Node specific – hardware integration/fabrication/packaging


WSN – challenges cont’d

• Physical environment is dynamic and unpredictable (Hw&Sw)• Small wireless nodes have stringent energy, storage, communication

constraints (Hw mainly)In-network processing of data close to sensor source provides (Sw, systems

design)– Scalability for densely deployed sensors– Low-latency for in situ triggering and adaptation

• Embedded nodes collaborate to report interesting spatio-temporal events (Sytems design)

Embeddable Portable Adaptive Low cost Robust Self healingSelf configuring Globally query-able


Application related challenges

• User requirements definition – novel technology hence this is hard

• Capability/expectations mitigation• Lack of comparison measure at end-to-

end systems level!!!Consequence!!!Don’t underestimate the role of cyclic

requirements/development/demonstration methodology


Data acquisition phase

• Sensors availability – MEMS technologies are just maturing - many physical sensors available

• Digital or analogue output - Digitization required• Sensors compatibility with other systems

components• SENSORS CALIBRATION, DRIFT AND

FAULTS- Mostly uncalibrated, but…very cheap• Integration sometimes a problem


Processing and comms challenges

• Nodes size, weight, energy resources and processing capabilities – contrary constrains which need mitigating

• Unreliability of wireless communications• Lack of debugging tools and wireless

technology immaturity• Off-the-shelf comms encapsulation;

unlexible protocols• Processing with little on much data


Processors and Motes Hardware

MoteSensor device

interfaceProcessor Memory Communications Form factor

Renee Mezzanine card

Atmel 8 bit 4 MHz

49 kB 916MHz, software modulation

484 mm2

rectangle

Mica 2 Mezzanine card (4 sensors)

Analog

Atmel 8 bit 8 MHz

644 kB 916/433MHzhardware modulation19.2 kbps

1800 mm2

rectangle

Mica2Dot Single sensorAnalog

Atmel 8 bit 4 MHz

644 kB 916/433MHzhardware modulation19.2kbps

255 mm2

disc

MicaZ Mezzanine card (4 sensors)

Analog

Atmel 8 bit 8 MHz

644 kB 2.4GHzZigBee

1800 mm2

rectangle

Intel mote Digital interface

ARM 32-bit 12 MHz

586kB 2.4GHzBluetooth

900mm2

rectangle


Information extraction challenges

• Timeliness of acquired data• Time synchronization• Data storage• Information extraction at source• Co-opertive behaviour• Global vs local treatment of the challenge• Mitigating energy vs quality/detail vs

timeliness vs system cost, size, etc


Information delivery challenges

• Raw data is too much saying too little• Huge range of user requirements motivated by –

conservativeness of some engineering fields (ref- Energy sector, aerospace, defence)

• Ease of interpretation by human in the loop – hard to accommodate with limited resources

• Range of useful options continuously growing presently


Actuation enablers

• Are still in its infancy• Much to be gained from any

breakthroughs here

Enabling actuation has serious consequences in the overall system design


User satisfaction

• Usually unknown/unpredictable till the development ends

• Trail and error as the favourite methods presently

• Huge range of reported work which failed to satisfy for all possible resons

• Unreliability of the put-together systems is damaging to the filed


The Grand WSN challengeFacilitating the migration of pervasive sensing from

future potential to present success Design space

•Care for the un-expert user – “beyond data collection systems”

•Robustness, fault tolerance

•Long life – across system layers and system components- in network processing &distribution

•Maintenance free systems – scalability, remote programming &generic components/ infrastructure

VLS networks asScientific instruments

Permanent monitoring fixtures

“The network is the sensor”


Software - design features

• designing for information visualization

• designing for robustness and long life - Fault Detection and management

• designing for practical applications

• designing for robust services support

• designing for information extraction- Complex Querying


Designing for practical applications

BSNThe problems:•Robustness of deployment•Technologies Integration•Fitness for purpose•Non-experts will use it!!!

End-to-end system design approach


Matching application requirements with available technology in a

safety critical application


Project history

• Commissioned late 2005• Externally funded• Client: NP Aerospace Plc - protective

clothing manufacturer for Defence - mostly for bomb disposal missions, de-mining, etc

• PhD student project


Project aim: Increased safety of missions

through remote monitoring


The problem: the suit Environment

• Increased heat production and reduced ability to remove heat results in storage

• Thermoregulatory system becomes unable to correctly regulate core temperature

• This may result in physical and psychological impairment

• Increased risk of making an avoidable error and jeopardising the mission


Possible solutionsManufacturer solution: add a cooling system to the suitInadequate:a) Inefficient use due to human factorsb) Distraction

Alternative:a) in-suit instrumentation and continuous monitoringb) automated cooling actuation based on state


Architecture• Sense-model-decide-act

architecture• Two control loops

– Rapid feedback to autonomously adjust cooling

– Support for modifications to mission plans and investigation into the construction of the suit.


Instrument Requirements

• provide detailed physiological measurement - better insight into what is happening

• support on-line and real-time thermal sensation estimates

• report of useful information (rather than data) to a remote station and the operative

• enable rapid assessment of hazardous situations

• allow the provision of thermal remedial measures through control and actuation


Part 2 : The deployment environment - a physiological perspective


UHS and Suit Trials• UHS- the thermoregulatory system is unable to defend against

increases in core body temperature

• UHS - associated with significant physical and psychological impairment

• Trials activity regime -four 16:30 min:sec cycles – treadmill walking– unloading and loading weights from a kit bag– crawling and searching– arm cranking– standing rest– seated physical rest


Experimental data• Measurands- wired instrumentation

– Heart rate– rectal temperature– skin temperatures (arm, chest, thigh and calf )

• Assessment– Subjective thermal sensation – twice per cycle, per segment and

overall– Comfort – as above

• Measurands - wireless– Skin temperature - 12 sites (symetrical + neck +abdomen)– Acceleration - 3D - 9 sites– Pulse oximetry, heart rate, CO2, galvanic

Experimental data


Figure 5. Core temperature responses (n=4; error bars are omitted for clarity) FS-NC=full suit, no cooling; NS= no suit

Experimental data


Figure 3. Typical heart rate response to EOD activity simulation (based on a single subject trial). FS-NC=full suit, no cooling; NO-S=no suit; W=walking; U=unloadin/loading weights; C=crawling and searching; A= arm exercise; R= seated rest. NB. Two of four subjects were not able to complete four activity cycles.

Figure 4. Mean skin temperature responses (averaged over 4 subjects; error bars are omitted for clarity). FS-NC=full suit, no cooling; NS=no suit

Figure 6. Skin and rectal temperature over time for a subject wearing the full suit with no cooling. Note how core temperature rises with thigh temperature after the two merge. This experiment needed to be terminated as the subject could not continue.

Figure 7.Self-assessed thermal sensation compared with chest skin temperature for subject 1.


Part 3: System design


Constraints and design choices- I

Suit related– Mix of wired and wireless– Multiple sensors to each node– Wires in suit– Size, power and weight a concern

Suit modularity accounted for – multi-node BSNThree tiers of comms

Sensors to nodeNode to nodeNode to base station

Two separate systems for:- posture monitoring Physiological ???


Constraints and design choices- II

Application relatedIntermittent comms - jammers, obstaclesMaintaining autonomous operation - key

Two modes of wireless commsIn-suit, on body - short range, near fieldExternal to mission control - long rangeBuffering - avoid overflowPriority transmissionInformation extraction in-suit


Constraints and design choices-III

Safety critical– Cooling actuation– Operative alerts– Mission alerts– Hardware redundancy

Information extraction in-network - major design implications

Fault isolation and management


Constraints and design choices-IV

Instrument scope-dual– In field– In the lab - for physiological research and manufacturer research

User led choice of operationIn field

max infromation output - thermal sensation, cooling status, trends, alerts x2

Data on demand - temperature and other selectedIn the lab

Data output - continuous - all including accelInformation output - continuous


Part 4: In-network modeling


Processing

• Basic filtering performed on sensor node– Allows rejection of invalid data and generation of

alarms• Additional filtering using a Kalman filter on the

processing nodes– Smooths data as well as providing estimates of error

• Modelling of thermal sensation• Operative alerts• Mission control alerts

Include postureCO2 thresholdingHRPrediction models


Temperature and Thermal Comfort


Temperature, Filters and Fusion – Kalman Filtering• Why filter?

– Basic measurements may be too noisy– Can’t estimate gradient meaningfully

otherwise• Why fuse measurements?

– Two measurements are more reliable than one

– Allow for / detect faulty sensors


Thermal sensation Modelling

• Takes skin temperature (and optionally core temperature) readings as input

• Provides an estimation of thermal sensation, both per body segment and globally, as output

• The main part of the model is a logistic function based on two main parameters:– the difference between the local skin temperature and its

“set” point (the point at which the local sensation is neutral) – the difference between the overall skin temperature and

the overall set point• Thermal sensation is given in the range −4 to 4, with −4 being

very cold and 4 being very hot

Zhang’s model



Zhang’s model evaluation

Figure 8. Overall thermal sensation over time during the activity regime with no suit.

Figure 9.Overall thermal sensation over time during the activity regime with the full suit and with no cooling.

Figure 10.Overall thermal sensation over time for a habituated subject with the full protective suit and no cooling.


HR and CO2


Posture

Follow-up• New model needed

• Activity needs monitoring – posture

• Other physiological parameters have to be tried out –HR, galvanic response, heat flux

• Model needs to predict not estimate/assess



Part 5: Prototype implelentation


Platform and sensors

JOHN’ New DIAGRAM HERE

Picture of CO2 and HR


Networking

• Wireless links between actuation / processing nodes• Wireless link between actuation node and remote

monitoring point• Data/information buffered in case of link failure - may

be uploaded at future point

New pic from John and Ramona here

Temperature Component Data Flow


Figure 13. Data and information system flow


Posture Component Data Flow


Remote Monitoring

• Main information display panel includes:– a 3D figure showing the interpolated temperature

distribution across the subject’s skin– the current average skin temperature, and – the current thermal sensation level

• Other panels show the location and status of the sensors and the history of the incoming data

New pic from John paper

New pic from Ramona paper


Actuation• Reinforcement Learning algorithms (such as

SARSA) can be used to develop a “policy” for controlling the cooling fan based on the “state” of the user

• Action is to turn fan on or off and regulate volume

• Utility is based on maintaining good comfort levels over time

• Takes account of battery depletion, likely mission duration, posture, as well as current thermal comfort


Operative alerts• Framework in place• Data and information processing flows readily

available (piggy back on mission control)• Avoid false alarms - link to robustness and fault

management• Sound considered at this stage but tactile

sounds good too• Research into HCI issues badly needed

Elena to change


Evaluation and results

Figure 19. Predicted thermal sensation including dynamic component of model


Figure 10. Skin temperature over time for (a) arm, (b) neck, (c) abdomen, (d) chest, (e) thigh, and (f) calf sites. The two leg sensors (thigh and calf positions) were placed on the right leg only. For several skin sites, temperature values were also obtained using a wired-in data logger (denoted "Logger"). The vertical lines in each graph show the start and end of activities. Each activity is represented by a number.

(a) (b)

(c) (d)

(e) (f)


Enriching the system for larger informational

gain - posture monitoring


Aim and postures• Dual aim

– Direct activity information to mission control for• Supervision of mission - health hazards/colapse/restrains• Technical assessment - problems - controller expertise• Inferrence of abstract info by controllers

– Parameter for thermal state prediction• 8 postures required: stand, walk, crawl, sitting,

lying down (up, down, side x2)


Results and evaluation for posture monitoring


Review of tutorial and summary

• exposition of design techniques and design choices• focus on an example

• BSN- neither large nor widely distributed but there are a number of fundamental requirements– the size of the nodes, wearability of the instrumentation, robustness,

reliability and fault-tolerance, etc• they dictate the majority of the design and implementation choices.

• Pursuing application driven design processes will enable the development of industrially strong systems which will increase confidence in the technology and contribute to its adoption in near future.


WSN – theoretical wonders- Scoping of large scale applications- Complex problems solved for individual

functional components/services- Theoretical proofs and simulation only- Lack of integrative work

1. Dust size- mm cube

2. Unplanned deployment3. Distributed

4. Millions of5. Re-configurable nets6. Self-healing7. Scalable8. Autonomous9. Information systems10.Collaborative decisions

1. Stack of quarters & miniaturization vs mote life trade-off2. Planned, carefully measured; ID based3. Gateway based – centrally controlled; backboned4. Hundreds at most (ExScal)5. Hard coded6. Prone to failure (more than 50% usually)7. Only through complete re-design8. Tightly controlled9. Data acquisition – relay to base10. Central post processing

Visions led SENSE and SEND

gaura, brusey

Documents

wireless sensing

intelligent systems

endtoend systems design

body sensor networks

middleware design

wsnsdr elena gaura

tiny wireless sensors

research centre