Automation & Robotics Research Institute (ARRI)The University of Texas at Arlington
F.L. Lewis, Assoc. Director for ResearchMoncrief-O’Donnell Endowed Chair
Head, Controls, Sensors, MEMS Group
http://ARRI.uta.edu/acs
Autonomous Systems
ORGANIZATION LEVEL
COORDINATION LEVEL
EXECUTION LEVEL
ABSTRACTION
PRECISION
THREE-LAYER PLANNING AND CONTROL ARCHITECTURE
Saridis
NASREM- Jim Albus
Sense Model Execute
Hum
an In
terf
ace
Kno
wle
dge
Bas
e
Sense Model Execute
Sense Model Execute
Sensors and Actuators
SensorFusion
Model Update (Learning)
TaskDecomp.
Swarm Behavior – Cooperative?
Three agents to lift 100 lb heavy object.Each agent can lift only 40 lbs. ?
Solution- Two Simple Rules:1. Attract towards object2. Repel from each other
RESULT- even spacing around the object
Decentralized Emergent Intelligence
A little example -
Can easily handle adding or taking away agents
Potential Fields
Initial
Final
Combining Behaviors Through Potential Fields
S. S. Ge and Y. J. Cui, ``New Potential Functions for Mobile Robot Path Planning'', IEEE Transactions on Robotics Automation, Vol. 16, No. 5, pp.615 -620, October 2000.
Ron Arkin
Sam Ge
Toyota Kanban Assembly Plant
Job 1 Job 2 Job 3
Part Flow
Worker 1 Worker 2 Worker 3
Two Simple Rules:1. Do jobs in sequence on any part you have until it is taken from you2. When you are idle, take the part from the worker upstream
Kanban Pull System
Result- self-balancing flow line
Note- part flows forward but information flows backwards!
Traffic Platooning
Leaderd43
d42
d41
3 step lookahead
Lookahead gives smoother control
Solution- Two Simple Rules1. Attract towards your station2. Repel from each other
Leader
Define- Station – Position WRT Leader
Better Solution:
Before - rules had equal weight for goal.Now – rule 2 is weaker – only for safety.
Leader’s Rule1. Attract towards your target
Formation Flight
Leader
Station Keeping
Simple rules for each agent Overall “Intelligent” Behaviorof squadron
Where is the Intelligence?
Autonomous?
Decentralized Intelligence and ControlEMERGENT BEHAVIOR
Depends on Leader !!
Internal Organization Autonomy (proprio-) VS. External Autonomy (extero-)
Behavior-Based Control
The Example of the Mobile Robot
follow wall
go through door
seekgoal
avoid obstacle
Behaviors-
st.1
st.2
st.3
st.4go throughdoor
follow wallseekgoal
avoid obstacle
vehicleCtrl. 1
output 1input 1ref. traj. 1
vehicleCtrl. 2
output 2input 2ref. traj. 2
vehicleCtrl. 3
output 3input 3ref. traj. 3
vehicleCtrl. 4
output 4input 4ref. traj. 4
gone through door
Behavior-Based Control
use ctrl.1
Fini
te S
tate
Mac
hine
FB c
ontro
ller
HYBRID CONTROL
Rules
TableXY Stage mounted on
rotational stageResolution: 3 m, 0.01 deg
MP-285XYZ degree of freedom
Motorized probeResolution: 40 nm
VerticalFocusing
Stage
Rotational stageResolution: 0.01 deg
CameraResolution: NTSC
Microscope RS-232
MOTIONCONTROLLERS
IMAGE AQUISITIONCARD
Analog / Digital Input- output CARD
LabView&
Windows CVI
MEMS Microassembly StationSemiautonomous Teleoperation
Vision Feedback
Map FromMEMS-Pro3D info
Visioninfo
LocalizationWith 3D info
Stored map
Feature-based
Based or pre-programmed behaviours
OpenGL LabWindows /CVI
RS-232
MOTIONCONTROLLERS
IMAGE AQUISITIONCARD
Analog / Digital Input- output CARD
LabView&
Windows CVI
VR control
Behavior-Based Hybrid Control needs good sensors, good controls
ACTUATORSENSORS/
SIGNAL PROC.
PLANT
performanceoutput
CONTROLLAW
RobustAdaptive
Fuzzy / NeuralPIDtracking
error
Tracking Loop
Inner Feedback Loops
select fromseveral outputs
select fromseveral Control Algorithms
select fromseveral inputs
select fromseveral referenceinputs
ReferenceInput
Needs pre-designed control algorithms forPrescribed Behaviors
Example:Wall following behaviourGoal Seeking behaviour
Inputsteeringsteering and speed
Measured outputdistance to walldistance to goal
Mission ControlMission includes segments:• Autonomous Takeoff• Waypoint Steering• Weapon Delivery• Weapon Evasion• Failure Accommodation• Upset Recovery• Autonomous Land
Aircraft ControlJim Buffington- Lockheed Martin, Fort Worth TX
Aircraft Controlincludes autopilots:• Takeoff Angle Hold• Course Hold• Altitude Hold• Wing Leveler• Turn Coordinator• Glide Slope Coupler
GuidanceSystem
Outer LoopControlSystem
Inner LoopControlSystem
ActuatorCommands
Flight PathFeedback
RateFeedback
AttitudeFeedback
PositionFeedback
Command Supervisory System
On-boardCommands
Off-boardCommands
Intelligent Guidance and Control System
On-Board Model
Health Management
Data
SensorData
Jim Buffington- Lockheed Martin, Fort Worth TX
SUPERVISION/ASSESSMENT
IDENTIFICATION
EVALUATION
ACTUATOR SENSORS/SIGNAL PROC.
PLANT
Trend Analysis
prognosis
performanceoutputCONTROL
LAW
RobustAdaptive
Fuzzy / NeuralPID
BLEND
probing
diagnostic output
Model Matching
diagnosis
Model-based Reasoning
Operator Interface
Task Decomposition
Traj./Path Planning
Exception Handling:
Sensor/Act. select
Control Alg. select/tune
On-Line Control Design
Knowledge Base
KB SYSTEM:
RLS, ELS, etc.
reference trajectory
HUMANOPERATOR
operator control input
exception signalcontrol reinforcement signal
Operator Interface
SharedControl
Rule-BasedFuzzyNeural Net
Neural or Fuzzy Classifier
Neural Net
select/tune
Agent-BasedModel-Based
Hierarchical ControlDecentralized ControlGame TheoryMaster/SlaveTask DecompositionPetri Net / DEDS SequencingAI Techniques
Multiple Agent Coordinator
Encapsulated Agent
Knowledge Base
trackingerror
EXECUTIVE
Tracking Loop
Inner Feedback Loops
A Control Engineer's View of Intelligent Control
Note:
All the architectures first shown capture these ideas extremely well!
Albus’ NASREMSaridis Three-Level
AUTONOMOUS SYSTEM
RECONFIGURATION
OPERATORDEPENDENT
DISCRETE EVENT, RULE-BASED
ADAPTIVE,LEARNED
OPTIMALITY
OPERATORDEPENDENT
PRE-PROGRAMMED
ON-LINE REVISION
INTELLIGENCE
AUTOMATION
DEPENDENT
INVOLUNTARY
INDEPENDENT
MANUAL
AUTOMATED
AUTONOMOUS
OPE
RAT
ION
SSYSTEM
CO
NTR
OL
Autonomy is Enabled by Intelligence and AutomationJim Buffington- Lockheed Martin, Fort Worth TX
WORKLOAD
CONTINUOUSINPUT
RELAXEDOVERSIGHT
ONLY HIGHESTLEVEL OBJECTIVES
Distributed decision-making based on often simple rules
Decentralized Intelligence leading to Emergent Behaviors
Autonomy is based on
Multiple sensors
Multiple good control algorithms
Event states related to prescribed pre-programmed behaviors
Wireless Distributed Sensor Networks
Management Center(Database large storage, analysis)
PDA
BSC(Base Station
Controller, Preprocessing)BTS
Wireless Sensor
Machine Monitoring
Medical Monitoring
Wireless SensorWirelessData Collection Networks
Wireless(Wi-Fi 802.11 2.4GHz
BlueToothCellular Network, -CDMA, GSM)
Data Acquisition Network
Data Distribution Network
Printer
Wireland
(Ethernet WLAN, Optical)
Animal Monitoring
Vehicle Monitoring
Onlinemonitoring
Wireless Sensor Networks
Servertransmitter
Any where to monitor and any where to access
Notebook Cellular Phone PC
Submarine Monitoring
The Problem of Complexity
Communication Protocols in a network must be restricted and organizedto avoid Complexity problems
e.g. in ManufacturingThe general job shop allows part flows between all machinesThe Flow Line allows part flows only along specific Paths
We have shown that the job shop is NP-complete
But the reentrant flow line is of polynomial complexity
Think of the military chain of command
Personnel Secure Area Monitoring
• Smart sensors and wireless MEMS sensor networks• Video surveillance and tracking• Biometrics and sensor fusion for personnel
monitoring• Databases and Data Mining• Pervasive and mobile computing (PICO)• Agent-oriented software engineering• Psychology and Human Performance Prediction • Fail-safe construction and smart materials
Contact Behrooz [email protected]
LOWSECURITY
MEDIUMSECURITY
HIGHSECURITY
VENTILATIONGas Sensors
. .Human Performance
Video Tracking/SurveillanceImage Processing
Data FusionData Mining
WALLS
Blast Layer
SensorLayers
....
Biometrics
Smart MaterialsSmart SensorsSmart Structures
Wireless NetworksPICO
Screening
Multiple-Clearance-Level Secure Access Areas
PHM Architecture and Enabling Technologies
Air Vehicle On-BoardHealth Assessment
Health Management,Reporting & Recording
Autonomic Logistics& Off-Board PHM
PVI
MAINTAINERVEHICLE INTERFACE
Mission Critical
PHMData
Displays & ControlsCrashRecorder
MaintenanceInterface Functions
IETMsConsumables
On-Board Diagnostics
PMD
.
PMA
In-Flight &Maintenance Data Link
Flight Critical
PHM / Service Info
Database
AMD/PMD
PHM Area Managers
MS Subsystems
• Sensor Fusion• Model-Based Reasoning• Tailored Algorithms• Systems Specific
Logic / Rules• Feature Extraction
Provides:
• AV-Level Info Management• Intelligent FI• Prognostics/Trends• Auto. LogisticsEnabling/Interface
Methods Used:
FCS/UtilitySubsystems
NVMICAWSManager
AVPHMHosted in ICP
Structures
MissionSystems
• Decision Support• Troubleshooting and Repair• Condition-Based Maintenance• Efficient Logistics
Vehicle Systems
Propulsion
Results In:
ALIS• Automated Pilot / Maint. Debrief
• Off-Board Prognostics
• Intelligent Help Environment
• Store / Distribute PHM Information
Hosted in ICP
Mike Gandy, Lockheed
Autonomous Machinery Monitoring
IEEE Singapore SMC, R&A, and Control Chapters Short Course
Wireless Sensor Networks for Monitoring Machinery,Human Biofunctions, and Biochemical Agents
Date : Thursday, 13 November 2003Time : 9:00 am. - 5:00 pmVenue: National University of Singapore
Wireless Networks DSP and User Interface
WirelessData Collection Networks
Wireless Sensor
Machine Monitoring
Security Personnel and Vehicle Monitoring
C
O O
HH2O
h+
h+
h+
H2OC
O O
H
C
O O
H
C
O O
HC
O O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
C
O O
H
C
O O
HH2O
h+
h+h+
h+
H2OC
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
HC
O O
H
CO O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
Biochemical Monitoring
EnvironmentalMonitoring
WirelessData Collection Networks
Wireless Sensor
Machine Monitoring
Security Personnel and Vehicle Monitoring
C
O O
HH2O
h+
h+
h+
H2OC
O O
H
C
O O
H
C
O O
HC
O O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
C
O O
H
C
O O
HH2O
h+
h+h+
h+
H2OC
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
HC
O O
H
CO O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
Biochemical Monitoring
EnvironmentalMonitoring
N
S
N
S
Ferromagneticmaterial
Non- -ferromagnetic
material
Permanentmagnet
MEMSChip
Vibrating bodywith the coil
MEMS PowerGeneration