Using Machine Vision and Hand-Motion Control to Improve

Crane Operator Performance

Journal: Transactions on Systems, Man, and Cybernetics--Part A: Systems

and Humans

Manuscript ID: SMCA-10-09-0357

Manuscript Type: Regular Paper

Date Submitted by the

Author: 26-Sep-2010

Complete List of Authors: Peng, Kelvin Chen Chih; Georgia Institute of Technology Singhose, William; Georgia Institute of Technology, Mechanical Engineering Bhaumik, Purnajyoti; Georgia Institute of Technology

Key Words: Control system human factors, Machine vision, Vibration control

Note: The following files were submitted by the author for peer review, but cannot be converted to PDF. You must view these files (e.g. movies) online.

Glove crane.mov Wand crane.mov

SUBMITTED TO TSMC 1

Using Machine Vision and Hand-Motion Control toImprove Crane Operator Performance

Kelvin Chen Chih Peng, William Singhose, Purnajyoti Bhaumik.

Abstract—The payload oscillation inherent to all cranes makesit challenging for human operators to manipulate payloadsquickly, accurately, and safely. Manipulation difficulty is alsoincreased by non-intuitive crane-control interfaces. This paperdescribes two new crane-control interfaces that allow an operatorto drive a crane by moving his or her hand freely in space. Animage-processing system is used to track the movement of thehand-held device, which is then used to drive the crane. Computersimulations and experimental data show that a combination ofaggressive feedback control to position the trolley, and an inputshaper to suppress payload swing, generates a fast crane responseand low residual oscillation. Studies involving novice operatorsfurther demonstrate the benefits of hand-motion crane control.

Index Terms—Control interface; machine vision; cranes; os-cillation; input shaping.

I. INTRODUCTION

Cranes play a key role in maintaining the economic vi-tality of modern-day industry. Their importance can be seenat shipyards, construction sites, warehouses, and in a widevariety of material-handling applications. The effectiveness ofcrane manipulation is an important contributor to industrialproductivity, low production costs, and worker safety.

One inherent property of cranes that is detrimental toefficient operation is the natural tendency for the payloadto oscillate like a pendulum, a double pendulum [1], orwith even more complex oscillatory dynamics [2]. Significanteffort has been made to develop control schemes to reducethe oscillatory response from both issued commands andexternal disturbances [3]–[9]. There has also been researchin controlling cranes that contain rotational joints, which addsan extra level of complexity due to their nonlinear dynamics[10]–[13]. Abdel-Rahman provides a review of crane controlstrategies developed during the second half of the twentiethcentury [14]. Operators who manipulate a crane utilizingappropriate oscillation-suppression technology generate saferand more efficient crane motions than operators without suchcompensation [10], [15]–[17].

K.C.C. Peng, W. Singhose, and P. Bhaumik are with the Woodruff Schoolof Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA,30332 USA e-mail: Singhose@gatech.edu

(a) Pendent

Trolley

Payload

Operator w/Pendent

(b) Schematic

Fig. 1. Crane Control Using a Push-Button Pendent.

In addition to facing the challenges of controlling large-amplitude, lightly-damped payload swing, operators must alsomaster non-intuitive machine interfaces. Examples of typicalcrane-control interfaces include push-button pendents, joy-sticks, and control levers. Figure 1 illustrates crane controlusing the pendent. The operator must be adept in the cognitiveprocess of transferring the desired manipulation path intoa sequence of button presses or lever deflections that willproduce the desired crane motion. For example, if the operatorwants to drive the crane through a cluttered workspace usinga push-button pendent, then the desired path must be mappedinto a sequence of events where the “Forward”, “Backward”,“Left”, and “Right” buttons are pushed for the correct timeduration and in the correct sequence.

Furthermore, as operators move through the workspace todrive the crane and monitor its progress, they may rotatetheir bodies and change the direction they are facing. In suchcases, the “Forward” button causes motion to the left, right,or even backward. As an additional challenge, the operatorcan only directly drive the overhead trolley, not the payload.Therefore, the operator must account for the time lag betweenthe commanded motion of the trolley, which can be manymeters overhead, and the delayed oscillatory response of thepayload.

While significant strides have been made to improve theoperational efficiency of cranes by controlling the dynamic

Page 1 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SUBMITTED TO TSMC 2

response to issued commands, relatively little considerationhas been given to the way in which operators issue thosecommands [18]. Interfaces that are tailored to the cognitiveprocesses associated with specific control system have beenshown to be beneficial [19]–[21].

This paper presents a novel control interface that allows anoperator to drive a crane simply by moving his or her handin space. Machine vision is used to track the location of ahand-held device (a wand or a glove) that is moved by theoperator. The position of the hand-held device is then used asthe reference command signal to drive the crane. The hand-motion control interface is well tailored to the task of driving acrane through a cluttered workspace because it eliminates thecognitive mapping process that is necessary with traditionalcontrol interfaces. Because of this, operators no longer needto account for the direction in which they are facing. Themanual dexterity required for safe and efficient operation isalso reduced.

Additionally, the control algorithm minimizes payloadswing without significantly slowing the system response.Therefore, the burden of manually reducing payload oscillationis removed, allowing the operator to concentrate solely on thepath planning and final positioning of the payload. Further-more, because a non-oscillating payload will always come torest directly beneath the overhead trolley, final positioning ofthe trolley is equivalent to the final positioning of the payload.

Hand-motion control also offers other cognitive advantagesover traditional interfaces. There are two primary divisionsof cognitive control: analytic problem solving and perceptualprocessing [22]. Perceptual processing tends to be faster andcan be performed in parallel, while analytic processing takeslonger and typically progresses serially. Analytic problemsolving also tends to be more prone to error [22], [23]. Theresults of many studies also suggest that people prefer, andadopt, perceptual processing when possible [17], [22], [24],[25].

From this perspective, hand-motion control helps operatorsby lowering the cognition level required to drive the crane.Operators no longer need to think analytically about thesequence of buttons to push, or to account for the swingingpayload; they only need to move the hand-held device to thedesired position, or along a desired path. This allows them toperform simpler perceptual processing.

II. 10-TON INDUSTRIAL BRIDGE CRANE

The work detailed in this paper uses the 10-ton industrialbridge crane shown in Figure 2. A bridge crane consistsof a fixed overhead runway, a bridge that travels along the

}}Payload

Hook

SuspensionCables

Runway

Bridge

Trolley

Fig. 2. 10-Ton Industrial Bridge Crane

runway, and a trolley that suspends the hook and payloadby cables. The trolley travels along the bridge. Laser rangesensors measure the trolley position along the runway and thebridge. A Siemens programmable logic controller (PLC) isused to control the motor drives and acts as the central controlunit. Commands to the crane can be issued with a push-buttoncontrol pendent, a laptop, or other devices [18].

A downward-pointing Siemens Simatic VS723-2 cameramounted on the trolley measures the relative position of thehook [26]. Reflective material, arranged in a hexagonal pattern,is attached to the top of the hook. The reflectors aid imagesegmentation and blob detection. Using the known geometryof the reflective markers, the position of the hook can bedetermined, and the length of the suspension cables can beestimated. The advantage of using multiple reflectors is two-fold: 1) robustness against changes in ambient light and noise,and 2) robustness against occlusion by the suspension cables.

III. HAND-MOTION CONTROL

In order to provide a more intuitive control interface, asystem was developed to drive the crane by simply movinga wand or a glove. The wand, shown in Figure 3, is a retro-reflective ball mounted to the end of a hand-held pole. Theglove, shown in Figure 4, is monotonically black, with acircular reflective marker attached to the top. After imagesegmentation and processing on the camera, the wand/gloveappear as a single contiguous blob of high pixel intensity. Todistinguish the wand/glove blob from the blobs formed bythe hook reflectors, a K-means clustering algorithm is used[27]. The camera calculates the positions of the hook andwand/glove approximately every 140 ms. Figure 5 shows thedifference in position between the wand/glove and the trolley(e) that is used as the error feedback signal.

Three different control architectures are investigated in thispaper. The standard pendent controller is used as the baseline

Page 2 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SUBMITTED TO TSMC 3

Hook

Wand

Fig. 3. Wand Control

Glove

Hook

Fig. 4. Glove Control

for comparisons. Its control block diagram is shown in Figure6. The pendent produces velocity commands for crane motors.Trolley movement then induces hook dynamics.

A Proportional-Derivative (PD) feedback controller is usedfor hand-motion control. Its block diagram is shown in Figure7. The difference between the horizontal positions of thetrolley and the hand-held device is used as the error signal(e from Figure 5) in negative feedback with PD control. Theoutput of the PD controller is passed through a saturator beforeit is sent to the trolley motors.

To mitigate the oscillatory hook response that results fromusing the PD hand-motion controller, an architecture thatcombines PD feedback with an input shaper (which will beexplained in Section IV) is introduced. Figure 8 shows the PD

Trolley

Hook

Camera

e

Wand/Glove

Fig. 5. Schematic Diagram of Hand-Motion Crane Control.

Standard Pendent Control

Motor,Trolley

HookDynamics

+

-

Trolley Position

e1

PD Wand/Glove Control

PD with Input Shaper Wand/Glove Control

Pendent Motor,Trolley

HookDynamics

Hand-HeldDevice Position

SaturatorPD

1/S

Linearized PD with Input Shaper Wand/Glove Con-trol for Root Locus Analysis

Motor,Trolley

HookDynamics

+

-

Trolley Position

e1KPD(PD)

VR

1/S

VPD VT XHInputShaper

VZVLinearizedConverter

Feedback loop relevant to the stability analysis using root locus

Hand-HeldDevice Position

InputShaper

Motor,Trolley

HookDynamics

+

-

Trolley Position

e1

Hand-HeldDevice Position

SaturatorPD

1/S

Fig. 6. Standard Pendent Control Block Diagram

Motor,Trolley

HookDynamics

+

-Trolley Position

e

Hand-HeldDevice Position

PD

Fig. 7. PD Hand-Motion Control Block Diagram

Motor,Trolley

HookDynamics

+

-Trolley Position

e

Hand-HeldDevice Position

PD InputShaper

Fig. 8. PD with Input Shaper Hand-Motion Control Block Diagram

hand-motion controller with the addition of an input shaperafter the saturator. The linear input shaper is placed afterthe nonlinear saturator (rather than before) to preserve theoscillation-reducing properties of the shaped signal.

Note that from the perspective of control theory, the wandand the glove are identical. However, in terms of ergonomicsduring operation, the wand has a greater reach and can drivethe crane towards tight spaces, such as corners. On the otherhand, the glove sacrifices range of reach for a smaller size andease of use.

Page 3 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SUBMITTED TO TSMC 4

INITIAL EXPERIMENTS WITH A SMALL-SCALE MOBILE BOOM CRANEEhsan Maleki and William Singhose

George W. Woodruff School of Mechanical EngineeringGeorgia Institute of Technology

Atlanta, Georgia 30327Email: Singhose@gatech.edu

Joshua VaughanDepartment of Mechanical and Aerospace Engineering

Tokyo Institute of TechnologyOokayama, Meguro-ku, Tokyo, Japan

Email: vaughanje@gatech.edu

ABSTRACTCranes are used extensively throughout the world in a widevariety of construction and material-handling applications.The throughput and safety of cranes are limited by payloadoscillation. Most cranes are attached to fixed bases thatlimit their workspace size. The operational capabilities ofcranes can be greatly expanded by allowing base motion.However, any such base motion complicates the dynamicresponse of the crane and significantly increases the dif-ficulty of controlling the crane. This paper presents ini-tial experimental results from a small-scale mobile boomcrane. A theoretical model of the crane is presented andexperimentally verified. The oscillatory dynamics of thecrane are highlighted and controllers to combat these un-wanted dynamics are presented.

1 IntroductionCranes are one of the primary machines used for materialhandling throughout the world. They work in a large va-riety of locations, including factories, construction sites,and shipyards. Cranes, by their very nature, are suscep-tible to pendulum-like oscillations induced by the motionof the crane itself or by outside disturbances. These oscil-lations make positioning payloads and navigating clutteredworkspaces very challenging. Therefore, payload oscilla-tion is an important limiting factor of crane throughput andsafety.

Numerous researchers have proposed using feedbackcontrol to limit crane payload oscillation [1], but successhas been limited. Unfortunately, a fundamental conflictexists between computerized feedback control and humanoperators. Crane operators are feedback controllers; theycontinually adjust the input command to achieve a desiredresponse. Any additional computer-based feedback controlcan conflict with the actions of the human operator.

Input-shaping [2, 3, 4], on the other hand, is one con-trol method that is highly compatible with human operatorsand can drastically reduce the motion-induced oscillations.Input shaping is implemented by convolving a sequence ofimpulses, called the input shaper, with the desired refer-ence command. This process is shown in Figure 1. Inputshaping has been successfully applied to bridge [5, 6, 7, 8],tower [9, 10, 11, 12], and container cranes [13, 14] and

0 !

*

!

Figure 1. The Input-Shaping Process

has been shown to improve crane operator performance[15, 16, 17, 18].

A second significant limitation of using feedback con-trol on cranes is the difficulty of measuring the motion ofthe payload. Therefore, some feedback control methodsare constructed to avoid the need to measure the location ofthe actual payload. For example, if the hook swing angleis measured, then a feedback controller could be designedto eliminate the swing by assuming that driving the hookswing angle to zero will stop the payload swing. Figure2 illustrates the problem with this approach when the pay-load creates a double-pendulum effect. Figure 2(a) showsthe location of the payload when the swing angle of thehook is 10◦. The payload is to the left of the hook. Figure2(b) shows the location of the payload for the same hookangle of 10◦, but in this case, the payload is to the right ofthe payload. These two photographs demonstrate that mea-suring the suspension cable angle does not provide reliableinformation about the payload location. Double-pendulumpayloads commonly arise in many types of crane handlingprocesses. A feedback controller designed simply based onthe swing angle would not only function improperly, butcould lead to unstable results.

In addition to payload oscillation, lack of mobility isanother limiting factor of crane performance. Most cranes,like the one shown in Figure 3, have little or no base mo-

Fig. 9. The Input-Shaping Process Applied to a Reference Step Input.

IV. INPUT SHAPING

Input shaping is a technique used for negating the flexiblemodes of a system and does not require the feedback mech-anisms of closed-loop controllers [28]–[32]. The referencecommand is modified in such a way that the resonant modesare not excited [31], [33], [34].

Figure 9 illustrates the input-shaping process. When a ref-erence step command, represented by the dotted line, is issuedto a flexible system, the response is oscillatory. If however, thereference command is convolved with a sequence of impulses,known as an input shaper, then a shaped command is produced.When the shaped command is issued to the flexible system,the amplitude of residual vibration is reduced, as shown at thebottom of Figure 9.

The amplitudes and time locations of the impulses in theshaper can be obtained from closed-form solutions or bysolving a set of constraint equations. The primary designconstraint is a limit on the amplitude of vibration caused by theshaper. The vibration amplitude of an under-damped, second-order system from a sequence of n-impulses is given by [29]:

A∑ =ω√

1− ζ2e−ζωtn

√[C(ω, ζ)]2 + [S(ω, ζ)]2, (1)

where,

C(ω, ζ) =n∑i=1

Aieζωti cos(ωti

√1− ζ2) (2)

S(ω, ζ) =n∑i=1

Aieζωti sin(ωti

√1− ζ2), (3)

and ω is the natural frequency of the system, ζ is the dampingratio, and Ai and ti are the amplitude and time of the ith-impulse, respectively.

0

5

10

15

20

25

30

0.7 0.8 0.9 1 1.1 1.2 1.3

ZVZVDEI

Perc

ent R

esid

ual V

ibra

tion

Normalized Frequency (ω/ωm)

Vtol

0.06

0.29

Fig. 10. Sensitivity Curve for the ZV, ZVD, and EI Shapers

To form a non-dimensional vibration amplitude, (1) isdivided by the amplitude of residual vibration from a singleimpulse of unity magnitude. The resulting expression givesthe ratio of vibration with input shaping to that without inputshaping. This percentage residual vibration (PRV) is given by[35]:

PRV = e−ζωtn√[C(ω, ζ)]2 + [S(ω, ζ)]2. (4)

Equation (4) represents the level of vibration induced by animpulse sequence given at any frequency and any damping ra-tio less than one. A constraint on residual vibration amplitudecan be formed by setting (4) less than or equal to a tolerablelevel of residual vibration at the modeled natural frequencyand damping ratio.

For the simplest, Zero Vibration (ZV) shaper, the tolerableamount of vibration is set to zero. This results in a shaper ofthe form [28], [29]:

ZV =

[Ai

ti

]=

11+K

K1+K

0 π

ω√

1−ζ2

, (5)

where,

K = e−ζπ√1−ζ2 . (6)

To gain more insight, one can analyze the performance ofthe ZV shaper with the use of a sensitivity curve, shown inFigure 10. The vertical axis is the Percent Residual Vibration(PRV ) and the horizontal axis is the actual natural frequency,ω, normalized by the modeled frequency, ωm (which was usedto design the input shaper). The sensitivity curve for a ZVshaper is shown by the solid line. The curve indicates howresidual vibration amplitude changes as a function of modelingerrors in frequency. The sensitivity curves for the Zero-Vibration-Derivative (ZVD) [29] and the Extra-Insensitive (EI)[36] shapers are also shown. These more robust shapers

Page 4 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SUBMITTED TO TSMC 5

0

0.5

1

1.5

2

2.5

3

3.5

0 5 10 15 20 25 30 35 40

Trolley1Hook1Trolley2Hook2

Pos

itio

n (

m)

Time (sec)

Fig. 11. Simulated Point-To-Point Responses Using Pendent Control

contain three impulses and are longer in duration than therelatively non-robust ZV shaper.

V. SIMULATION RESULTS

Computer models were constructed to simulate the controlarchitectures in Figures 6, 7, and 8. Hand-motion trajectorieswere specified as ramps in position with gradients equivalentto the maximum velocity of the industrial crane (0.3577 m/s),which is approximately the speed of a slow walk. This alsomimics the typical hand-motion trajectories from a humanoperator.

A. Pendent Control

Some typical responses using the pendent controller areshown in Figure 11, using a cable length of 5 m. Using thependent, if a move button is depressed for a certain duration,then the trolley will move at constant velocity for a certaindistance. The position of the trolley plotted against time isa ramp that plateaus after a certain distance. Due to thependulum-like nature of the hook, this type of movementwill, in general, induce oscillations. Figure 11 shows the hookresponses to move distances of approximately 2 and 3 m.

B. PD Hand-Motion Control

Figure 12 shows the simulation results for the PD hand-motion controller with low feedback gains. The response isslow, with a 10% to 90% rise time of nearly 9.3 s and a 2%settling time of 55 s. The maximum percentage overshoot isjust over 2%, indicating the residual oscillation is also low.

To increase the responsiveness of the hand-motion con-troller, the gains must be increased so that the trolley is able torapidly accelerate to its maximum velocity. Figure 13 showsthe response of the hand-motion control with increased gains.

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30 35 40

Hand-Held DeviceTrolleyHook

Pos

itio

n (

m)

Time (s)

Fig. 12. Simulation Result of the PD Hand-Motion Controller with LowGains

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30 35 40

Hand-Held DeviceTrolleyHook

Pos

itio

n (

m)

Time (s)

Fig. 13. Simulation Result of the PD Hand-Motion Controller with HighGains

Under this set of gains, the crane is more responsive; the10% to 90% rise time is reduced by 53% to 4.4 s. However,hook oscillations are now more significant. The maximumpercentage overshoot has increased by eight-fold to 16.6%.Due to the large overshoot and lightly damped nature of thehook dynamics, the 2% settling time has increased by morethan 100% to 119 s.

C. PD with ZV Shaper Hand-Motion Control

There is an inherent trade-off using the PD hand-motioncontroller. Low gains do not induce large hook oscillationamplitudes, but the trolley response is slow. High gains pro-duce a faster trolley response at the cost of high amplitudehook oscillations. The goal of combining high gains PDfeedback with a ZV input shaper is to create a controller withthe desirable performance features of fast response and lowresidual oscillations. The ZV shaper was designed using the

Page 5 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SUBMITTED TO TSMC 6

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30 35 40

Hand-Held DeviceTrolleyHook

Pos

itio

n (

m)

Time (s)

Fig. 14. Simulation Result of the PD with ZV Shaper Hand-MotionController

natural frequency (determined by the 5m cable length) anddamping ratio corresponding to the industrial crane discussedin Section II.

Figure 14 clearly shows the improved response of the com-bined controller. The 10% to 90% rise time is only 4.6 s. The2% settling time is reduced to 18 s. The maximum percentageovershoot is only 7.8%. Note that the small oscillation in theresponse results from the PD control of the trolley, not froma deficiency in the input shaper.

VI. EXPERIMENTAL RESULTS

The hand-motion control system was implemented on the10-ton bridge crane. The wand/glove trajectories producedby the human operators were similar to those used in thesimulations. The ramp gradient was approximately equivalentto the maximum velocity of the crane, and the move distancewas approximately 2 m for the tests reported here.

Figure 15 illustrates an operator using hand-motion controlto start and stop the crane. To start moving the crane, theoperator can expose the wand/glove to the camera at anydistance away from the trolley. The result is that the value of ejumps instantaneously when the wand/glove is detected. Whenthe crane approaches the desired stopping location, the craneoperator lowers the wand/glove, which becomes undetectableby the camera. As a result, the control software reacts bysetting e to zero. Due to these operational effects, experimentalresults contain spurious and discontinuous artifacts for thewand/glove position during the starting and stopping stagesof the motion.

A. PD Hand-Motion Control

Figure 16 shows the experimental results for the PD hand-motion controller with a low proportional gain. The 10%

Starting Moving Decelerating Stoppede e e=0 e=0

Fig. 15. Hand-Motion Control: Starting and Stopping

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30 35 40

Hand-Held DeviceTrolleyHook

Pos

itio

n (

m)

Time (s)

Fig. 16. Experimental Result of the PD Hand-Motion Controller With LowGains

to 90% rise time is around 10 s, the maximum percentageovershoot is about 5%, and the 2% settling time is 50 s. Notethat the wand/glove was not moved in a perfect ramp, as inthe simulations.

Figure 17 shows the experimental response of PD hand-motion control with increased gains. The 10% to 90% risetime was reduced by over 40% to 5.8 s; however, the maximumpercentage overshoot doubled to 10%. The hook took longerthan 53 s to settle within 2% of the stop distance. Note thatwith the higher gains, the hook tracks the wand/glove positionmuch more closely than with the low gains.

B. PD with ZV Shaper Hand-Motion Control

The experimental results using the PD hand-motion con-trollers clearly demonstrate the trade-off between low and highgains. Using high gains will reduce rise time at the expenseof increased overshoot and settling time.

Figure 18 shows the experimental response of the combinedPD and ZV input-shaping controller. The 10% to 90% risetime is around 4.8 s, and there is virtually no overshoot orresidual oscillation. For this reason, the 2% settling time isapproximately 8 s (an 84% improvement over PD with lowgains).

Page 6 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SUBMITTED TO TSMC 7

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30 35 40

Hand-Held DeviceTrolleyHook

Pos

itio

n (

m)

Time (s)

Fig. 17. Experimental Result of the PD Hand-Motion Controller With HighGains

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30 35 40

Hand-Held DeviceTrolleyHook

Pos

itio

n (

m)

Time (s)

Fig. 18. Experimental Result of the PD With ZV Shaper Hand-MotionController

The experimental and simulation results demonstrate thatthe hand-motion controller with high PD gains and ZV inputshaping is able to produce a fast hook response withoutsignificant overshoot and residual vibration.

To investigate the robustness of the controller, the samemovement was repeated for suspension cable lengths of 4 mand 6 m, without redesigning the PD controller or the inputshaper. These correspond to a 1 m decrease and increase fromthe original 5 m cable length. The experimental results forthese cases are shown in Figures 19 and 20, respectively.The controller still shows effective suppression of residualoscillations, even when the cable lengths were changed.

To gain more insight, one can analyze the ZV shaper’sperformance with the use of a sensitivity curve, which wasshown in Figure 10. For this specific application, the ZVshaper designed for the 5 m cable length has a modeledfrequency of ωm = 1.4rad/s. When the cable length is

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30 35 40

Hand-Held DeviceTrolleyHook

Pos

itio

n (

m)

Time (s)

Fig. 19. PD With ZV Shaper Hand-Motion Controller With 4 m SuspensionCable Length

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30 35 40

Hand-Held DeviceTrolleyHook

Pos

itio

n (

m)

Time (s)

Fig. 20. PD With ZV Shaper Hand-Motion Controller With 6 m SuspensionCable Length

changed to 4 m and 6 m, the real natural frequency, ω, ismodified to 1.57rad/s and 1.28rad/s, respectively. Thesecorrespond to a normalized frequency of 1.12rad/s and0.91rad/s, respectively. Referring to the ZV sensitivity curvein Figure 10, it can be seen that this is still reasonably goodperformance, as the PRV for both cases is under 20%. Thecase of the 4 m cable length has a slightly higher PRV ,which explains the presence of the visible, but small amountof residual oscillations in Figure 19. If the crane was expectedto undergo large changes in cable length, then the morerobust shapers shown in Figure 10 could be used instead [29],[36]. Alternatively, adaptive input shapers could also providerobustness to parameter variations [37]–[39].

VII. OPERATOR STUDIES

This section presents the results from two studies that wereconducted to compare the operating efficiency of pendent

Page 7 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SUBMITTED TO TSMC 8

Start

Finish

TrolleyAxis

BridgeAxis

1.2 m

0.7 m

Fig. 21. Obstacle Course 1

control versus hand-motion control. In each study, the goalwas to move the crane hook from start to finish as quicklyas possible without collisions with the obstacles. Operatorswere each given five minutes to familiarize themselves withthe control interfaces before commencing the study.

A. PD Hand-Motion Control

The obstacle course used in this study is shown in Figure21. The obstacles were arranged such that the fastest route tothe finish required diagonal crane movements (simultaneousmovement in both the trolley and bridge axes). Twelve noviceoperators completed the obstacle course using the followingcontrol interfaces:

1) Pendent control with the block diagram shown in Figure6.

2) Wand control using PD hand-motion control with lowgains and no input-shaping. The block diagram is shownin Figure 7 and the gains were the same as the controllerthat produced Figure 16. Low gains were used so thatthe crane would not excite large amplitude oscillations.

Figure 22 shows the course completion times for eachoperator. The average completion time using the pendent was97 s. The average completion time using the wand was only46 s, a 53% improvement. A paired t-test indicated that theimprovement in completion time was statistically significant,t(11) = 4.20, p < 0.002.

Figure 23 plots the number of collisions that occurred duringeach trial. Using pendent control, all operators suffered at leastone collision, and the average number of collisions was 5.1.Using wand control, most operators had few or no obstaclecollisions, and the average number of collisions was 0.92, an81% improvement. A paired t-test indicated that the reductionin collisions was statistically significant, t(11) = 4.45, p <

0.001.

0

50

100

150

200

1 2 3 4 5 6 7 8 9 10 11 12

PendentWand

Com

ple

tion

Tim

e (s

ec)

Operator

Fig. 22. Obstacle Course 1 Completion Times.

0

2

4

6

8

10

12

14

1 2 3 4 5 6 7 8 9 10 11 12

Pendent

Wand

Nu

mb

er o

f co

llis

ion

s

Operator

Fig. 23. Obstacle Course 1 Collisions.

-2.5

-2

-1.5

-1

-0.5

0

-0.5 0 0.5 1 1.5 2 2.5 3 3.5

PendentWandT

roll

ey A

xis

(m)

Bridge Axis (m)

Fig. 24. Obstacle Course 1 - Overhead View of Hook Response.

Figure 24 shows a typical two-dimensional hook responsefor a single operator using both pendent and wand control. Itis evident that wand control allows the hook to be controlledmore precisely. The reduction in the amplitude of hook swingand overall smoothness of hook travel are major contributors tofewer obstacle collisions and more efficient navigation throughthe obstacle course.

Page 8 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SUBMITTED TO TSMC 9

1.3 m

0.7 m

0.5 m

1.2 mStart Finish

TrolleyAxis

BridgeAxis

Fig. 25. Obstacle Course 2

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10

PendentGlove

Com

ple

tion

Tim

e (s

ec)

Operator

Fig. 26. Obstacle Course 2 Completion Times.

B. PD with ZV Shaper Hand-Motion Control

The obstacle course used in this study is shown in Figure25. Ten novice operators completed the obstacle course usingthe following control interfaces:

1) Pendent control with the block diagram shown in Figure6.

2) Glove control using PD feedback with high gains andZV shaper hand-motion control. The block diagram isshown in Figure 8 and the gains selected were the sameas the controller that produced Figure 18.

Figure 26 shows the course completion times for eachoperator. The average completion time using the pendent was77 s. The average completion time using the glove was only24 s, a 69% improvement. A paired t-test indicated that theimprovements in completion time were statistically significant,t(9) = 9.71, p < 0.0001.

Figure 27 plots the number of collisions that occurred duringeach trial. Using pendent control, many operators collided thecrane with the obstacle. The average number of collisions was0.9. However, all operators were able to avoid the obstacleusing the glove. A paired t-test indicated that the reductionin number of collisions was statistically significant, t(9) =

3.25, p < 0.01.

0

1

2

3

1 2 3 4 5 6 7 8 9 10

Pendent Glove

Nu

mb

er o

f C

olli

sion

s

Operator

Fig. 27. Obstacle Course 2 Collisions.

-2.5

-2

-1.5

-1

-0.5

0

7.5 8 8.5 9 9.5 10 10.5 11

PendentGloveT

roll

ey A

xis

(m)

Bridge Axis (m)

Fig. 28. Obstacle Course 2 - Overhead View of Hook Response.

Figure 28 shows a typical two-dimensional hook responsefor a single operator using both the pendent and glove. It isevident that glove control reduces hook swing and allows theoperator to control the hook more precisely. For this reason,operators were able to navigate the obstacle course with moreefficiency and without colliding with the obstacles.

VIII. CONCLUSIONS

Crane controllers based on operator hand-motion have beensuccessfully installed on an industrial bridge crane. An over-head camera tracks the position of a hand-held device (awand or a glove), which is moved by the operator throughthe desired trajectory. The crane then follows the wand/glove.Three types of hand-motion controllers were simulated. Itwas found that aggressive PD feedback gains to position theoverhead trolley, combined with ZV input shaping to limithook swing, produced the desired performance characteristicsof fast response, short settling time, low amplitude overshoot,and low residual oscillations. Experiments conducted on anindustrial crane verified the usefulness of the proposed controlarchitectures. Furthermore, the results from operator studiesprovide evidence that hand-motion control is more effective

Page 9 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SUBMITTED TO TSMC 10

than traditional control interfaces such as a push-button pen-dent.

IX. ACKNOWLEDGMENTS

The authors would like to thank Siemens Industrial Automa-tion, the Manufacturing Research Center at Georgia Tech, andBoeing Research and Technology (BR&T) for their support ofthis work.

REFERENCES

[1] W. Singhose, D. Kim, and M. Kenison, “Input shaping control of double-pendulum bridge crane oscillations,” ASME J. of Dynamic Systems,Measurement, and Control, vol. 130, no. 034504, May 2008.

[2] J. Yoon, W. Singhose, J. Vaughan, G. Ramirez, M. Kim, and S. Tawde,“Dynamics and control of crane payloads that bounce and pitch duringhoisting,” in IDETC/CIE 2009, San Diego, CA, 2009.

[3] Y. Sakawa and Y. Shindo, “Optimal control of container cranes,”Automatica, vol. 18, no. 3A, pp. 257–266, 1982.

[4] G. P. Starr, “Swing-free transport of suspended objects with a path-controlled robot manipulator,” J. of Dynamic Systems, Measurement andControl, vol. 107, pp. 97–100, 1985.

[5] D. Strip, “Swing-free transport of suspended objects: a general treat-ment,” IEEE Trans. Robotics and Automation, vol. 5, no. 2, pp. 234 –6, 1989.

[6] N. Singer, W. Singhose, and E. Kriikku, “An input shaping controllerenabling cranes to move without sway,” in ANS 7th Topical Meeting onRobotics and Remote Systems, vol. 1, Augusta, GA, 1997, pp. 225–31.

[7] K.-S. Hong, J. H. Kim, and K.-I. Lee, “Control of a container crane: Fasttraversing, and residual sway control from the perspective of controllingan underactuated system,” in Proceedings of the 1998 American ControlConference, vol. 2, 1998, pp. 1294–1298.

[8] Y.-S. Kim, H.-S. Seo, and S.-K. Sul, “A new anti-sway control schemefor trolley crane system,” in Thiry-Sixth IAS Annual Meeting. ConferenceRecord of Industry Applications Conference, vol. 1, 2001, pp. 548–552.

[9] Z. Masoud and M. Daqaq, “A graphical approach to input-shapingcontrol design for container cranes with hoist,” IEEE Transactions onControl Systems Technology, vol. 14, no. 6, pp. 1070–1077, 2006.

[10] G. Parker, K. Groom, J. Hurtado, J. Feddema, R. Robinett, and F. Leban,“Experimental verification of a command shaping boom crane controlsystem,” in American Control Conference, San Diego, CA, 1999, pp.86–90.

[11] O. Sawodny, A. Hildebrandt, and K. Schneider, “Control design forthe rotation of crane loads for boom cranes,” in IEEE InternationalConference on Robotics and Automation, 2003. Proceedings. ICRA’03,vol. 2, 2003.

[12] J. Neupert, T. Mahl, O. Sawodny, and K. Schneider, “A NonlinearControl Strategy for Boom Cranes in Radial Direction,” in AmericanControl Conference, 2007. ACC’07, 2007, pp. 25–30.

[13] C. Ko, J. Chiou, C. Y. Gau, S. Lee, and W. Chang, “Tower cranevibration supression using generalized input shaping,” International J.of Fuzzy Systems, vol. 6, no. 1, pp. 38–43, 2004.

[14] E. Abdel-Rahman, A. Nayfeh, and Z. Masoud, “Dynamics and Controlof Cranes: A Review,” Journal of Vibration and Control, vol. 9, no. 7,p. 863, 2003.

[15] A. Khalid, J. Huey, W. Singhose, and J. Lawrence, “Human operatorperformance testing using an input-shaped bridge crane,” ASME J. ofDynamic Systems, Measurement, and Control, vol. 128, no. 4, pp. 835–841, 2006.

[16] D. Kim and W. Singhose, “Performance studies of human operatorsdriving double-pendulum bridge cranes,” Control Engineering Practice,vol. 18, no. 6, pp. 567 – 576, 2010.

[17] J. Vaughan, A. Smith, S. Kang, and W. Singhose, “Predictive graph-ical user interface elements to improve crane operator performance,”Accepted to IEEE Trans. on Systems, Man, and Cybernetics–Part A:Systems and Humans, 2010.

[18] K. L. Sorensen, J. B. Spiers, and W. E. Singhose, “Operational effects ofcrane interface devices,” in IEEE Conference on Industrial Electronicsand Applications, Harbin, China, 2007.

[19] S. Guerlain, G. Jamieson, P. Bullemer, and R. Blair, “The mpc elucida-tor: a case study in the design for human-automation interaction,” IEEETransactions on Systems, Man and Cybernetics, Part A: Systems andHumans, vol. 32, no. 1, pp. 25–40, January 2002.

[20] G. Jamieson, “Ecological interface design for petrochemical processcontrol: An empirical assessment,” IEEE Transactions on Systems, Manand Cybernetics, Part A: Systems and Humans, vol. 37, no. 6, pp. 906–920, 2007.

[21] S. V. Dam, M. Mulder, and M. van Paassen, “Ecological interface designof a tactical airborne separation assistance tool,” IEEE Transactions onSystems, Man and Cybernetics, Part A: Systems and Humans, vol. 38,no. 6, pp. 1–1233, 2008.

[22] J. Reason, Human Errors. Cambridge, UK: Cambridge UniversityPress, 1990.

[23] E. Brunswick, Perception and the Representative Desing of Experiments.Berkeley, CA: University of California Press, 1956.

[24] A. Kirlik, “The organization of perception and action in complex controlskills,” Ph.D. dissertation, The Ohio State University, Columbus, OH,1989.

[25] K. J. Vicente and J. Rasmussen, “Ecological interface design: Theoreti-cal foundations,” IEEE Transactions on Systems, Man and Cybernetics,vol. 22, no. 4, pp. 589–606, 1992.

[26] K. Hekman, K. Sorensen, C. Staheli, and W. Singhose, “Measuringcrane payload swing through computer vision,” in Int. Symp. on FlexibleAutomation, Osaka, Japan, 2006.

[27] S. Lloyd, “Least squares quantization in pcm,” IEEE Transactions onInformation Theory, vol. 28, no. 2, pp. 129–137, 1982.

[28] O. J. M. Smith, “Posicast control of damped oscillatory systems,”Proceedings of the IRE, vol. 45, no. September, pp. 1249–1255, 1957.

[29] N. C. Singer and W. P. Seering, “Preshaping command inputs to reducesystem vibration,” J. of Dynamic Sys., Measurement, and Control, vol.112, pp. 76–82, 1990.

[30] W. Singhose and W. Seering, Command Generation for Dynamic Sys-tems. ISBN 978-0-9842210-0-4, www.lulu.com/content/621219, 2007.

[31] W. Singhose, “Command shaping for flexible systems: A review ofthe first 50 years,” International Journal of Precision Engineering andManufacturing, vol. 10, no. 4, pp. 153–168, 2009.

[32] T. Singh and G. R. Heppler, “Shaped input control of a systemwith multiple modes,” Journal of Dynamic Systems, Measurement, andControl, vol. 115, pp. 341–347, September 1993.

[33] P. Meckl and W. Seering, “Experimental evaluation of shaped inputsto reduce vibration for a cartesian robot,” J. of Dynamic Systems,Measurement, and Controls, vol. 112, pp. 159–165, June 1990.

[34] P. H. Meckl and W. P. Seering, “Minimizing residual vibration for point-to-point motion,” Journal of Vibration, Acoustics, Stress and Reliabilityin Design, vol. 107, pp. 378–382, October 1985.

[35] K. Kozak, W. Singhose, and I. Ebert-Uphoff, “Performance measures forinput shaping and command generation,” Journal of Dynamic Systems,Measurement, and Control, vol. 128, no. 3, pp. 731–736, 2006.

[36] W. Singhose, W. Seering, and N. Singer, “Residual vibration reductionusing vector diagrams to generate shaped inputs,” ASME J. of Mechan-ical Design, vol. 116, no. June, pp. 654–659, 1994.

[37] S. Rhim and W. Book, “Noise effect on adaptive command shapingmethods for flexible manipulator control,” IEEE Transactions on ControlSystems Technology, vol. 9, no. 1, pp. 84 –92, jan 2001.

[38] A. Tzes and S. Yurkovich, “An adaptive input shaping control scheme for

Page 10 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

SUBMITTED TO TSMC 11

vibration suppression in slewing flexible structures,” IEEE Transactionson Control Systems Technology, vol. 1, no. 2, pp. 114 –121, jun 1993.

[39] E. Pereira, J. Trapero, I. Daz, and V. Feliu, “Adaptive input shapingfor manoeuvring flexible structures using an algebraic identificationtechnique,” Automatica, vol. 45, no. 4, pp. 1046 – 1051, 2009.

Page 11 of 11

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960