mndrive kumar

1!1

Vijay Kumar!UPS Foundation Professor!

University of Pennsylvania!

Aerial Robot Swarms!

University of Minnesota!April 16, 2014

2!

Unmanned Aerial Vehicles!

Mass!0! 1! 10! 100! 1,000! 10,000! 100,000!

Northrop-Grumman Global Hawk (32,200 lbs)!

Boeing X-45A UCAV (12,000 lbs)!

Northrop-Grumman

Boeing X-45A UCAV (12,000 lbs)lbs)

Bell Eagle Eye (2,250 lbs) !

AscTec!Pelican (3.5 lbs)!

Bell Eagle Eye (2,250 lbs)

Hummingbird (1 lb)!Hummingbird (1 lb)Hummingbird (1 lb)

KMel kNanoQuad (0.12 lb)!

Hummingbird (1 lb)Hummingbird (1 lb)

KMelKMel kNanoQuadkNanoQuad(0.12 lb)(0.12 lb)

Gen. Atomics MQ-9 Reaper (10,000 lbs)!

Gen. Atomics Predator (7,000 lbs)!Gen. Atomics Predator (2,250

lbs)!

Images from www.af.mil

Boeing Scaneagle (40 lbs)!

3!

Unmanned Aerial Vehicles!

!!

!> $10B industry!

!" Military: Surveillance, force protection, warfare (> 75 countries)!

!" Civilian commercial: Transportation, agriculture, mining, security!

!" Civilian private: DIY Drones!

!!

!

!!

!!

Increasingly smaller, lower power and inexpensive inertial sensing, GPS, and computing

1!

10!

100!

1000!

10000!

1980! 1990! 2000! 2010!

Number of UAVs worldwide!

4!

Search and Rescue!

Michael, et al, J. Field Robotics, 2012!

5!

Security and First Response!

6!

Security and First Response!Flight through building!Flight through building

7!

Real Time Agrylitics!

8

Apple Orchards,

Pennsylvania State University

Biglerville, PA

10

Quadrotor

11

Pico

11 cm

20 g,

6.5 Watts

Max speed 6m/s

12

$10M

$1M

$100

$10

$1

Global Hawk

Predator, Reaper

Scan Eagle, Raven

Hobby Kits

Toys

Chips

$100K

$1K-10K? Autonomous, agile micro aerial vehicles

our focus

GPS, airbag sensors, processors

DIY Drones, Kick starter projects

10s

100s

1,000,000s

100,000s

10,000s

13!

Length Scales!

13

!"

!#$"

!#%"

!#&"

!#'"

!#("

!#)"

!#*"

!#+"

!" !#!%" !#!'" !#!)" !#!+" !#$" !#$%" !#$'"

,-./"012345"

64078/"92:"

;1C"

B-0

14!

[Kushleyev, Mellinger and Kumar, RSS 2012]!

The KMel Nano Quadrotor!

16!

Size does Matter!

a 1R, 1

R2a 1

R, 1

R2111

linear acceleration 11

angular acceleration

mass M inertia I

r

R

thrust f

17!17

1 Dynamics, Control and Planning!2 State Estimation (Autonomy)!

3 Swarm technology!

Research Challenges!

18

Quadrotor

19!

x

X

Y

Z

r !1"

!2"

!3"

!4"

y

#, yaw!

$, pitch!%, roll!

x, y, z position!

Underactuated

z Control!

#, yaw

$, pitch%, roll

R

20!

x

X

Y

Z

r !1"

!2"

!3"

!4"

y

#, yaw!

$, pitch!%, roll!

x, y, z position!

Underactuated

z Control!

#, yaw

$, pitch%, roll

R

21!

Differential Flatness!

[x, y, z, #]!

A

B

A B

B

B

Minimum snap trajectory!

[Mellinger and Kumar, 2011]!

26666666666666666664

xyz xyz

37777777777777777775min(t)

Z T0k....r (t)k2 + (t)2dt

SE(3)

22!

Control on SE(3)!Specified trajectory!

(t) : [0, T ] R3 SO(2)

x

y

z

t t

Rdese3 =tt

= desRdes

#des"

M = I + I(kReR ke)

eR(Rdes, R)

k

f = m (r+Kver +Kper + g). k

[Wen and Kreutz-Delgado, 1991; Lee et al, 2011; Mellinger and Kumar, 2011]!

23!

Software Architecture!

[Kraft 2003; Mellinger, Michael, and Kumar 2010; Mellinger and Kumar 2011]!

Rigid body dynamics

Motor controller Attitude

controller

Position controller

Trajectory Planner

Rigid body dynamics controller Attitude controller

Rdes

&des

R, '!r, r

f

M Attitude

controller controller RRRdesdesR, , '

r, r

M M M

Basin of attraction almost all of SO(3) !

~0.001 s!

~0.01 s!

~0. 1 s!

24!

Roll and Pitch!

25!

Robust, nonlinear controller!

tr[I (Rdes)TR] < 2 e(0)2 2min(I)

kR

1 1

2trI (Rdes)TR

26!

Translation!

27!

[Mellinger and Kumar, ICRA 2011]!

Minimum Snap Trajectories!

A

C

B

28!

Minimum Snap Trajectories!

[Mellinger and Kumar, ICRA 2011]!

29![Thomas et al, ICRA 2014]!

Aerial Grasping and Manipulation !

30

motion capture cameras

reflective markers

Sensing

31

Photograph by Joe McNally

32!32

Unreliable GPS!

33!33

No GPS!

34!34

1 Dynamics, Control and Planning!2 State Estimation (Autonomy) !

3 Swarm technology!

35!

Microsoft Kinect!

Hokuyo!Laser Scanner!

Operation in Unstructured Environments!

36!

x

d1d2 d3

d03d02

d01

Pillars!

Robot!

(x1, y1) (x3, y3)(x2, y2)

Concurrently estimate !!Locations of pillars (6)!!Displacement of the robot (2)!

Simultaneous Localization !And !Mapping!

New robot position!

Simultaneous Localization !And !Mapping!

37

1.8 GHz Core i3 processor, 8 GB RAM

u- blox LEA-6T GPS module

Hokuyo UTM-30LX LiDAR

2 mvBlueFOX-MLC200w grayscale

HDR cameras

(fisheye lenses, 752 480, 25 Hz)

IMU 100 Hz

[Shen, Mulgaonkar, Michael, and Kumar 2014]

38

Heterogeneous Sensors

Proprioceptive Sensors

Inertial Measurement Unit

Absolute Sensors

GPS, Magnetometer

Altimeter

Relative Sensors

Laser scanner

Cameras

zt = h(xt) + nt

zt = h(xt, . . . ,xtk) + nt

xt = f(xt1,ut1) + nt

39!

f2!

f1! f0!f0f0f

Features in environment!

x0!

x1!

f3! f4!

x2!

x3!

f6!

f3f3fz00

z10

z20 z21z11

z32z33

z23

z43

z63x1 = f(x0, u01,t01)

x2 = f(x1, u12,t12) x3 = f(x2, u23,t23)

Robot!Pose!

Simultaneous !Localization !And !Mapping!

Simultaneous !Localization !And !Mapping!

40!

Map Refine (20 Hz)

Pose Graph SLAM

position

Multi-Sensor Unscented

Kalman Filter (100 Hz)

velocity

Estimation and Control Architecture!

Trajectory Generator (20 Hz)

Planner (20 Hz)

User Interface

Velocity estimator

Visual odometry

Laser odometry

Altitude estimator

Downward Camera (30Hz)

Stereo Camera (25 Hz)

IMU (100 Hz)

Pressure Altimeter (20 Hz)

Laser Scanner (20 Hz)

GPS (10Hz)

Controller (100 Hz)

41!

Onboard State Estimation!

[Shen, Michael, and Kumar 2011]!

IMU, Laser scanner, and camera!

Auton Robot

Lee, T. (2011). Geometric tracking control of the attitude dynamics ofa rigid body on SO 3 . In Proc. of the Amer. control conf., SanFrancisco, CA.

Lee, T., Leok, M., & McClamroch, N. H. (2010). Geometric trackingcontrol of a quadrotor UAV on SE 3 . In Proc. of the IEEE conf.on decision and control, Atlanta, GA.

Mellinger, D., & Kumar, V. (2011). Minimum snap trajectory genera-tion and control for quadrotors. In Proc. of the IEEE intl. conf. onrobot. and autom., Shanghai, China.

Mellinger, D., Michael, N., & Kumar, V. (2010). Trajectory generationand control for precise aggressive maneuvers with quadrotors. InProc. of the intl. sym. on exp. robot., Delhi, India.

Mesbahi, M. (2005). On state-dependent dynamic graphs and theircontrollability properties. IEEE Transactions on Automatic Con-trol, 50(3), 387392.

Michael, N., Mellinger, D., Lindsey, Q., & Kumar, V. (2010). TheGRASP multiple micro UAV testbed. IEEE Robotics & Automa-tion Magazine, 17(3), 5665.

Nieuwstadt, M. J. V., & Murray, R. M. (1998). Real-time trajectorygeneration for differentially flat systems. International Journal ofRobust and Nonlinear Control, 8(11), 9951020.

Ogren, P., Fiorelli, E., & Leonard, N. (2002). Formations with a mis-sion: stable coordination of vehicle group maneuvers. In Proc.of intl. sym. on mathematical theory networks and syst., NotreDame, IN.

Olfati-Saber, R., & Murray, R. M. (2002). Distributed cooperative con-trol of multiple vehicle formations using structural potential func-tions. In Proc. of the IFAC world congress, Barcelona, Spain.

Olfati-Saber, R., & Murray, R. M. (2004). Consensus problems in net-works of agents with switching topology and time-delays. IEEETransactions on Automatic Control, 49(9), 15201533.

Shim, D., Kim, H., & Sastry, S. (2003). Decentralized nonlinear modelpredictive control of multiple flying robots. In Decision andcontrol, 2003. Proceedings. 42nd IEEE conference on (Vol. 4,pp. 36213626). New York: IEEE.

Tabuada, P., Pappas, G. J., & Lima, P. (2001). Feasible formations ofmulti-agent systems. In Proc. of the Amer. control conf., Arling-ton, VA (pp. 5661).

Tanner, H., Pappas, G. J., & Kumar, V. (2002). Input-to-state stabilityon formation graphs. In Proc. of the IEEE intl. conf. on robot. andautom., Las Vegas, NV (pp. 24392444).

Turpin, M., Michael, N., & Kumar, V. (2011). Trajectory design andcontrol for aggressive formation flight with quadrotors. In Proc.of the intl. sym. of robotics research, Flagstaff, AZ.

Vicsek, T., Czirk, A., Ben-Jacob, E., Cohen, I., & Shochet, O. (1995).Novel type of phase transition in a system of self-driven particles.Physical Review Letters, 75(6), 12261229.

Matthew Turpin is a Ph.D. candi-date in the Department of Mechan-ical Engineering and Applied Me-chanics at the University of Penn-sylvania. He works on formation de-sign and control of micro-aerial ve-hicles.

Nathan Michael is a Research As-sistant Professor in the Departmentof Mechanical Engineering and Ap-plied Mechanics at the University ofPennsylvania. He received his Ph.D.in Mechanical Engineering from theUniversity of Pennsylvania in 2008.He works in the areas of dynamics,estimation, and control for groundand aerial robot systems.

Vijay Kumar is the UPS Founda-tion Professor and the Deputy Deanfor Education in the School of En-gineering and Applied Science atthe University of Pennsylvania. Hereceived his Ph.D. in MechanicalEngineering from The Ohio StateUniversity in 1987. He has beenon the Faculty in the Departmentof Mechanical Engineering and Ap-plied Mechanics with a secondaryappointment in the Department ofComputer and Information Scienceat the University of Pennsylvaniasince 1987. He is a Fellow of the

American Society of Mechanical Engineers (ASME) and the Institu-tion of Electrical and Electronic Engineers (IEEE).

42!

Autonomous Exploration!IMU, Laser scanner, and RGBD camera!

44!

Lithium Polymer Batteries!

0!

200!

400!

600!

800!

1000!

1200!

1400!

0! 50! 100! 150! 200! 250! 300! 350! 400! 450! 500!

Spec

ific

Pow

er (W

/kg)

!

Specific Energy (Whr/kg)!

Quadrotors with 10-20 min endurance!

[B. Morgan, ARL and Y. Mulgaonkar, Penn]!

Bolt (10 secs)!

Armstrong (20 mins)!

350

Specific Energy (Whr/kg)[B. Morgan, ARL and Y.

350


1000

Spec

ific

Pow

er (W

/kg)

1000

Spec

ific

Pow

er (W

/kg) Loose

weight!!

45!

Power!

0!

200!

400!

600!

800!

1000!

1200!

1400!

0! 50! 100! 150! 200! 250! 300! 350! 400! 450! 500!

Spec

ific

Pow

er (W

/kg)

!

Specific Energy (Whr/kg)!

10,000 Whr/kg

Lithium Polymer Batteries!

Quadrotors with 10-20 min endurance!

[B. Morgan, ARL and Y. Mulgaonkar, Penn]!

Bolt (10 secs)!

Armstrong (20 mins)!

350


350


1000

Spec

ific

Pow

er (W

/kg)

1000

Spec

ific

Pow

er (W

/kg) Loose

weight!!

46

Reducing the Payload

CPU: Intel Atom Processor, 1.6 GHz, 1 GB Ram

Sensing: 2 grayscale Matrix Vision cameras,

376x240 + IMU

Weight: 740gram

Power: ~120 W

2012 [Shen, Mulgaonkar, Michael, and Kumar 2013]

47

Vision + IMU State Estimation

Estimate rotations by tracking features

Assuming features are known, estimate position

Estimate scale with second camera

Assuming pose is known, map features

48!

Vision + IMU State Estimation!

49![Shen, Mulgaonkar, Michael, and Kumar, 2013]!

Fast (4 m/s), Indoor Outdoor, foliage

Indoor, 3-D Indoor/outdoor, visual SLAM

51!51

1 Dynamics, Control and Planning!2 State Estimation !

3 Swarm technology!

52!

Collaboration in Small Teams!

[Lindsey, Mellinger, and Kumar, 2012]

53!

1 Act independently

2 Require only local information

3 Anonymous behavior

54!

robot i

robot j

g(t) 2 SO(2)R3

si,j(t) = xj(t) xi(t)

Leader-Follower Networks!

[Desai, Ostrowski, and Kumar, 1999; Turpin, Michael and Kumar, 2011]

57!

Control of Formation Shape and Group Motion!

(Turpin, Michael, and Kumar, 2013)!

58!

[Kushleyev, Mellinger and Kumar 2012]

Control of Formation Shape and Group Motion!

59!

Search and Rescue!

[Michael et al, 2012]!

N. Michael, S. Shen, K. Mohta, Y. Mulgaonkar, V. Kumar, K. Nagatani, Y. Okada, S. Kiribayashi, K. Otake, K. Yoshida, K. Ohno, E. Takeuchi, and S. Tadokoro, Collaborative mapping of an earthquake-damaged building via ground and aerial robots, J. Field Robotics, vol. 29, no. 5, pp. 832841, 2012.!

60!7th, 8th, and 9th floors!

Thank you! The contributions and collaborations of the fo l low ing co l leagues are g race fu l l y acknowledged: A. Wolisz, K. Pister, R. Brodersen, E. Alon, G. Kelson, A. Niknejad, B. Nikolic, J. Wawrzynek, P. Wright, D. Tse, M. Maharbiz, J. Carmena, R. Knight, L. Van de Perre, and B. Gyselinckx, R. Muller, M. Mark, D. Chen, A. Parsha, S. Gambini, and all my current and past graduate students. Contributions by the BWRC member companies and the MuSyC and GSRC consortia are truly appreciated. Many thanks to V. Vinge, N. Stephenson, I. Asimov, and C. Stross for providing the true visions!

mndrive kumar

Documents