Automation & Robotics Research Institute (ARRI)The University of Texas at Arlington
F.L. Lewis, Assoc. Director for ResearchMoncrief-O’Donnell Endowed Chair
Head, Controls, Sensors, MEMS Group
http://ARRI.uta.edu/acs
Controls, Sensors, & MEMS Group
Mission: To help companies implement advanced controls to increase product quality and flexibility in the production process and to perform high quality research to expand the manufacturing knowledge base
Over $5M of research funding from industry, NSF, DoD, and Texas
• Industrial and Aerospace Controls Design - vibratory systems and actuator deadzone / backlash
• Intelligent control - neural networks and fuzzy logic• Aircraft Control and DoD Weapons Pointing Control• Discrete event manufacturing workcell control• Microelectromechanical MEMS Systems and Sensors• Wireless Sensor Networks for machinery and security
monitoring
Research accomplishments with hundreds of technical papers published and Eight Textbooks
Patents on Intelligent Nonlinear ControllersNumerous SBIR Contracts – ARRI won the SBA Tibbets Award
Contact Frank Lewis817-272-5900 [email protected]
Contact Frank Lewis817-272-5900 [email protected]
Autonomous Tank Gun Barrel Control
ARO SBIR Contract
$725K in 4 grantsFrom NSF and ARO
$725K in 4 grantsFrom NSF and ARO
Nonlinear Intelligent Control
Objective:Develop new control algorithms for faster, more
precise control of DoD and industrial systemsConfront complex systems with backlash,
deadzone, flexible modes, unknown disturbances and dynamics
• Three patents on neural network learning control • Internet- Remote Site Control and Monitoring• Rigorous Proofs and Design Algorithms• Applications to Tank Gun Barrels, Vehicle Active
Suspension, Industrial machine control• SBIR contracts for industry implementation• Numerous Books and Papers• Award-Winning PhD students
$300K in new NSF/ARO funding for Basic Controls Research
Neural Network Learning Control
[Λ I]
Robust ControlTerm vi(t)
Tracking Loop
rKr
Nonlinear FB Linearization Loop
F1(x)^ qr = qrqr.e
ee = .
qd =qdqd.
..qd
RobotSystem1/KB1 i
F2(x)^
Kη
ηid
NN#1
NN#2Backstepping Loop
ue[Λ I]
Robust ControlTerm vi(t)
Tracking Loop
rKrKr
Nonlinear FB Linearization Loop
F1(x)F̂1(x)^ qr = qrqr.qr =qr = qrqr.qrqr.e
ee = .e = .
qd =qdqd.qd =qd =qdqd.qdqd.
..qd..qd
RobotSystem1/KB1 i
F2(x)F̂2(x)^
KηKη
ηid
NN#1
NN#2Backstepping Loop
ue[Λ I]
Robust ControlTerm vi(t)
Tracking Loop
rKrKr
Nonlinear FB Linearization Loop
F1(x)F̂1(x)^ qr = qrqr.qr =qr = qrqr.qrqr.e
ee = .e = .
qd =qdqd.qd =qd =qdqd.qdqd.
..qd..qd
RobotSystem1/KB1 i
F2(x)F̂2(x)^
KηKη
ηid
NN#1
NN#2Backstepping Loop
ue[Λ I]
Robust ControlTerm vi(t)
Tracking Loop
rKrKr
Nonlinear FB Linearization Loop
F1(x)F̂1(x)^ qr =qr = qrqr.qrqr.qr =qr = qrqr.qrqr.e
ee = .e = .
qd =qd =qdqd.qdqd.qd =qd =qdqd.qdqd.
..qd..qd
RobotSystem1/KB1 i
F2(x)F̂2(x)^
KηKη
ηid
NN#1
NN#2Backstepping Loop
ueKηKη
ηid
NN#1
NN#2Backstepping Loop
ue
Input Membership Fns. Output Membership Fns.
Fuzzy Logic Rule Base
NN
Input
NN
Output
Linguistics- Fuzzy Logic
Nervous System- Neural Network (NN)
Input x Output u
Input x Output u
(Includes Adaptive Control)
Input Membership Fns. Output Membership Fns.
Fuzzy Logic Rule Base
NN
Input
NN
Output
Linguistics- Fuzzy Logic
Nervous System- Neural Network (NN)
Input x Output u
Input x Output u
(Includes Adaptive Control)
Intelligent Control Tools
Decision-Making Control
Objective:Develop new control algorithms
for decision-making, supervision, resource assignment
Apply to manufacturing workcell control, battlefield C&C systems, internetworked systems
• Patent on Discrete Event Supervisory Controller • Internet- Remote Site Control and Monitoring• Implemented on Intelligent Robotic Workcell• New Algorithms based on Matrices• USA/Mexico Collaboration• New course material for UTA curriculum• Exploring Applications to Battlefield Information
Systems
$75K in ARO Funding for Networked Robot Workcell Control$80K in NSF Funding for research and USA/Mexico Network
USA/Mexico Internetworked Control
Man/Machine User Interface
TexasTexas
Intelligent Robot Workcell
Dr. Jose Mireles- PI
[email protected]@arri.uta.edu
New Books in 03-04
1. F.L. Lewis, D.A. Tacconi, Ayla Gurel, and O.C. Pastravanu, "Method and Apparatus for Testing and Controlling a Flexible Manufacturing System," U.S. Patent 6,185,469, awarded 6 Feb. 2001.
2. R. Selmic, F.L. Lewis, A.J. Calise, and M.B. McFarland, "Backlash Compensation Using Neural Network," U.S. Patent 6,611,823, Awarded 26 Aug. 2003.
RecentPatents
Wireless Sensor Networks
• Machinery monitoring & Condition-Based Maintenance (CBM / PHM / RUL)• Remote site biochemical warfare (BCW) toxin monitoring• Personnel monitoring and secure area denial• Optical MEMS human biosensors
[email protected]://mems.uta.edu
C&C UserInterface forwireless networks-Wireless
Data Collection Networks
Wireless Sensor
Machine Monitoring
Security Personnel and Vehicle Monitoring
C
O O
HH2O
h+
h+
h+
H2OC
O O
H
C
O O
H
C
O O
HC
O O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
C
O O
H
C
O O
HH2O
h+
h+h+
h+
H2OC
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
HC
O O
H
CO O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
Biochemical Monitoring
EnvironmentalMonitoring
WirelessData Collection Networks
Wireless Sensor
Machine Monitoring
Security Personnel and Vehicle Monitoring
C
O O
HH2O
h+
h+
h+
H2OC
O O
H
C
O O
H
C
O O
HC
O O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
C
O O
H
C
O O
HH2O
h+
h+h+
h+
H2OC
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
HC
O O
H
CO O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
Biochemical Monitoring
EnvironmentalMonitoring
Objectives:Develop MEMS sensors for biochemical &
environmental monitoring, industrial and DoD equipment monitoring, and secure area personnel denial
Develop self-configuring wireless networks for remote site monitoring
LabVIEW Real-time Signaling & Processing
CBM Database and real time Monitoring
PDA access Failure Data from anytime and
anywhere
User Interface, Monitoring, & Decision AssistanceWireless Access over the Internet
Tiered Disaster/Emergency Response
• Full response requires local, state, and federal assets• State response includes National Guard• Military support requires Total Force involvement
Local ResponseIncident Commander
Federal Response
State Response
Dr. Ron Blanck, President, UNT Health Science Center
HomelandSecurity
Wireless Sensor Nets for BCW Monitoring• MEMS sensors for biochemical species including anthrax,
nerve gases, NOx, organophosphorus • Wireless Sensor Networks for remote site biochemical
monitoring
Structured chemically-activenanosphere thin film- Rajeshwar
3x3 IGEFET sensor micro-array- Kolesar
DSP and C&C User Interface for wireless networks- Lewis
Molecular Recognition- RudkevichEnzyme-Based Detection- Bob Gracy
Microcantilever MEMS sensor array- Kolesar
Collaboration!
Personnel Secure Area Monitoring
• Smart sensors and wireless MEMS sensor networks• Video surveillance and tracking• Biometrics and sensor fusion for personnel monitoring• Databases and Data Mining• Pervasive and mobile computing (PICO)• Agent-oriented software engineering• Psychology and Human Performance Prediction • Fail-safe construction and smart materials
Contact Behrooz [email protected]
Contact Behrooz [email protected]
LOWSECURITY
MEDIUMSECURITY
HIGHSECURITY
VENTILATIONGas Sensors
. .Human Performance
Video Tracking/SurveillanceImage Processing
Data FusionData Mining
WALLS
Blast Layer
SensorLayers
....
Biometrics
Smart MaterialsSmart SensorsSmart Structures
Wireless NetworksPICO
Screening
Multiple-Clearance-Level Secure Access Areas
© L. Holder
Larry HolderSajal Das, UTA CSE
PHM Architecture and Enabling Technologies
Air Vehicle On-BoardHealth Assessment
Health Management,Reporting & Recording
Autonomic Logistics& Off-Board PHM
PVI
MAINTAINERVEHICLE INTERFACE
Mission Critical
PHMData
Displays & ControlsCrashRecorder
MaintenanceInterface Functions
IETMsConsumables
On-Board Diagnostics
PMD
.
PMA
In-Flight &Maintenance Data Link
Flight Critical
PHM / Service Info
Database
AMD/PMD
PHM Area Managers
MS Subsystems
• Sensor Fusion• Model-Based Reasoning• Tailored Algorithms• Systems Specific
Logic / Rules• Feature Extraction
Provides:• AV-Level Info Management• Intelligent FI• Prognostics/Trends• Auto. Logistics
Enabling/Interface
Methods Used:
FCS/UtilitySubsystems
NVMICAWSManager
AVPHMHosted in ICP
Structures
MissionSystems
• Decision Support• Troubleshooting and Repair• Condition-Based Maintenance• Efficient Logistics
Vehicle Systems
Propulsion
Results In:
ALIS• Automated Pilot / Maint. Debrief
• Off-Board Prognostics
• Intelligent Help Environment
• Store / Distribute PHM Information
Hosted in ICP
Mike Gandy, Lockheed
New Business Models for Machinery MaintenanceOriginal Equipment Manufacturer Becomes the Service Provider
Integrate Manufacturing, Service, and MaintenanceLifetime Machine Service ContractGuaranteed Up-Time for UserGuaranteed Lifetime Revenue Stream for OEM
• Internet-Based E-Maintenance• Integrate Internet with Machine On-Board Diagnostics• Centralized Service Scheduling and Dispatching• Reduced Service Costs
Subcontracted Maintenance Service ProvidersMSP provides and maintains the wireless sensor networkMSP monitors equipment, schedules & provides maintenanceLike current Security Systems- Brinks, etc.
Dr. Jay Lee Univ. of Wisc. MilwaukeeG. Vachtsevanos, Ga. Tech
http://www.echelon.com/
Building automationUtilities automationIndustrial automationHome automationTransportation automation
LonWorks
Control Networks for
Smart Home Networksreprinted with permission of Artech House, from R. Frank [2000]
Smart Home Networks
Major driver for the smart home is the power distribution industry,Could save enormous sums with demand-side regulation and automated remote meter reading. The Intelligent Building Institute has been a force in developing appropriate standards.
Contact Diane CookUTA CSE [email protected]
Contact Diane CookUTA CSE [email protected]
sensor signalconditioning
DSP
local userinterface
applicationalgorithms
data storage
communicationanalog-to-
digitalconversion
NETWORK
hardwareinterface
Network SpecificNetwork Independent
Virtual Sensor
IEEE 1451 Standard for Smart Sensor Networks
Concept of Smart Sensorcontains functions in addition to those needed foraccurate presentation of the measurand
Objective: Develop-MEMS sensors for biomedical, industrial and DoD
equipment monitoring, environmental, gas monitoring, and wireless security networks.
MEMS control systems, micro robotics, and actuators for micro assembly and manipulation.
$180K in ARO/ UTA/ Texas funding to set up ARRI MEMS lab $250K in MEMS related Grants from NSF and ARO
F.L. Lewis, Head, Controls, Sensors, MEMS GroupUTA Automation & Robotics Research Institute
Stored Map FromMEMS-Pro3D info
Visioninfo
LocalizationWith 3D info
Feature-basedVision
User-Guided Microassembly Station
User Interface & DSP for wireless networks
WirelessData Collection Networks
Wireless Sensor
Machine Monitoring
Security Personnel and Vehicle Monitoring
C
O O
HH2O
h+
h+
h+
H2OC
O O
H
C
O O
H
C
O O
HC
O O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
C
O O
H
C
O O
HH2O
h+
h+h+
h+
H2OC
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
HC
O O
H
CO O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
Biochemical Monitoring
EnvironmentalMonitoring
WirelessData Collection Networks
Wireless Sensor
Machine Monitoring
Security Personnel and Vehicle Monitoring
C
O O
HH2O
h+
h+
h+
H2OC
O O
H
C
O O
H
C
O O
HC
O O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
C
O O
H
C
O O
HH2O
h+
h+h+
h+
H2OC
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
HC
O O
H
CO O
H h+
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
C
O O
H
e-
e-
e-
e-
TiO2TiO2
Ni
Biochemical Monitoring
EnvironmentalMonitoring
Wireless MEMS Sensor Networks
Frank [email protected]://mems.uta.edu
1 2 3 4 5 6 7 8 940
60
80
100
120
140
160
Rtto
t[K/W
]
V[V]
Trm
RtrRtcvCt
T
qRtcd
Thermal ModelMechanical ModelElectrical ModelOptical Model
( )( )
( )( )TsCR
TRsqsT
ttot
ttot
+=
1
MEMS Modeling and Control
FEA
Experiment
ARRI Probe Station
Karl Suss Probe StationMitutoyo 2000x MicroscopeManual ProbesMP-285 Automated Probe (40 nm)Camera & Vision SystemNI control boardsLabVIEW for image proc. & DSP
MEMS Design & FEA Software MEMS-ProIntelliSuiteANSYS
Trm
RtrRtcvCt
T
qRtcd
MEMS Modeling and Control
Thermal Model( )( )
( )( )TsCR
TRsqsT
ttot
ttot
+=
1
1 2 3 4 5 6 7 8 940
60
80
100
120
140
160
Rtto
t[K/W
]
V[V]
( ) ( ) ( )txkdtdxxc
dtxdm mm f=++2
2
Mechanical Model
Electrical Model
RC
L
Z(s)=R+sL+1/sC
FEA Experiment
( ) ( ) ( )sVTRsCLCs
sCsIeee
e
12 ++=
Optical Model2/1
20
22
20
exp2),(⎭⎬⎫
⎩⎨⎧ +−=
wst
wstU
π
Bruno BorovicWith Ai Qun Liu, NTU Singapore
Intellisuite simulation showing dry (RIE) etching of 110 Si to get Micro Channels. Channels are more than 100 microns high (ARRI).
MicroFluidics for VLSI Chip CoolingNumber of transistors on a chip is now limited by heat dissipation.Fabricate micro-channels on the back of the chip for cooling.
With Dereje Agonafer, DirectorUTA Packaging Center
Wendell McCulley, InterMEMS
Syed Haider
N
S
N
S
Ferromagneticmaterial
Non- -ferromagnetic
material
Permanentmagnet
MEMSChip
Vibrating bodywith the coil
C0
VP
++++
____
V0
+ +
++
+ +
++
_ _
__
_ _
__
CP
Vibrations
Cross section
Cross section
+ + ++ + ++ + + - - -- - -- - -
New Sensor Technologies- MEMSPower Generation
E-Mag Vibration
E-Static
RFID
New Sensing MethodsOptical Pulmonary Pressure Sensor
Membrane deflection FEA
TableXY Stage mounted on
rotational stageResolution: 3 µm, 0.01 deg
MP-285XYZ degree of freedom
Motorized probeResolution: 40 nm
VerticalFocusing
Stage
Rotational stageResolution: 0.01 deg
CameraResolution: NTSC
Microscope RS-232
MOTIONCONTROLLERS
IMAGE AQUISITIONCARD
Analog / Digital Input- output CARD
LabView&
Windows CVI
Microassembly StationBruno Borovic
Micro ManipulatorMP-285
Automated- 40 nm accuracy
Automatic Probe MP-285
Manual Control
Probe ControllerTableXY Stage mounted on
rotational stageResolution: 3 µm, 0.01 deg
MP-285XYZ degree of freedom
Motorized probeResolution: 40 nm
VerticalFocusing
Stage
Rotational stageResolution: 0.01 deg
CameraResolution: NTSC
MicroscopeRS-232
MOTIONCONTROLLERS
IMAGE AQUISITIONCARD
Analog / Digital Input- output CARD
LabView&
Windows CVI
Vision Feedback
Map FromMEMS-Pro3D info
Visioninfo
LocalizationWith 3D info
Stored map
Feature-based
Based or pre-programmed behaviours
OpenGL LabWindows /CVI
RS-232
MOTIONCONTROLLERS
IMAGE AQUISITIONCARD
Analog / Digital Input- output CARD
LabView&
Windows CVI
International Workshop Series
Workshop Series:Hong Kong Workshop on Controls, Chinese Univ. of HK, Nov 2001UTA/UNM Joint Workshop on Research and Academic Opportunities with
Latin America, 1 April 2003Texas MEMS Workshop, 6 May 2003MEMS Workshop on Mexico/USA Collaboration, 7 May 2003ARRI / TMAC/ Georgia Tech Workshop on Automated Machinery
Maintenance, 17 July 2003ARRI/UTD/HSC International Symposium on Homeland Security, 28-30
JulyIberoAmerican Workshop on Micro-Systems MEMS, Puerto Vallarta,
Mexico. 19-20 Sep. 2003A-Star Workshop on Autonomous Systems, Nanyang Technological Univ.,
Singapore, 27 Oct. 2003Wireless Sensor Networks, National Univ. Singapore, 13 Nov. 2003
Summer Program for K-12 students and undergraduate students, summer 2003.Robotics Workshop for local High School Teachers, summer 2003.Initiated a scholarship program for UNT Health Science Center students to
do research at UTA. A top female minority student is the first awardee.Started US/Mexico MEMS workshop series with J. Mireles, Univ. Autonoma
de Ciudad Juarez. Two workshops have been organized so far, one at UTA and one in Puerto Vallarta.Started a UTA/Mexico student exchange program for Mexico graduate and
undergraduate students.Obtained NSF REU supplement funding to support minority students, one
of whom wrote a winning paper.Started several Dallas/Ft Worth Metroplex-wide activities in MEMS, CBM,
and Homeland Security with TCU, UTD, UNT Health Science Center, Lockheed-Martin. Ran a series of technology transfer workshops in these areas.
Broader Impact Activities
Partners Respironics Inc., InterMEMS Inc., UT Southwestern Medical Institute, “Design of MEMS Optical Biosensors for Control of Critical Care Pulmonary Ventilators,” funded by NSF.
Partners Texas Instruments, TEXMEMS 03 (see list above).Partners Univ. Autonoma de Ciudad Juarez, Mexico, and Mexico/US
Foundation for Science, Workshop on US/Mexico MEMS Collaboration, (see list above).
Partners Georgia Tech, Texas Manufacturing Assistance Center, National Instruments, Lockheed-Martin, “Workshop on Automated Machinery Maintenance,” (see above list).
Partners Univ. Texas at Dallas, US EPA, Russian Academy of Rocket & Artillery Sciences, UK QinetiQ, International Symposium on Homeland Security,” (see above list).
Tech. Transfer & Outreach Initiative Partners
General Chairman, Mediterranean Control Conference, Rhodes, June 2003.
General Chairman, IEEE Conference on Decision and Control, Hawaii, Dec. 2003.
Organized “Plenary Panel on International Funding Thrusts and Mechanisms” at CDC 2003. Attendees were:
Paul Werbos, US NSF, ChairJean-Michel Dion, French Ministry of ResearchAlkis Konstantellos, European CommissionKishan Baheti, US NSFIan Petersen, Australian Research CouncilXiren Cao from Hong Kong.Chenghong Wang of the Chinese National Natural Science
Foundation could not attend due to visa delays.
International Outreach Activities
F.L. Lewis, Assoc. Director for ResearchMoncrief-O’Donnell Endowed Chair
Head, Controls, Sensors, MEMS Group
Automation & Robotics Research Institute (ARRI)The University of Texas at Arlington
Sponsored byIEEE Singapore SMC, R&A, and Control Chapters
Organized and invited by Professor Sam Ge, NUS
UTA / IEEE Singapore Short CourseWireless Sensor Networks for Monitoring Machinery,
Human Biofunctions, and Biochemical Agents
Mediterranean Control AssociationFounded 1992
Conferences held inCrete, Cyprus, RhodesIsraelItalyDubrovnik, CroatiaKusadasi Turkey 2004
Founding MembersM. ChristodoulouP. IoannouK. ValavanisF.L. LewisP. AntsaklisTed DjaferisP. Groumpos
Bringing the Mediterranean Together for Collaboration and Research