robotic apple harvesting in washington state - fieldrobot · robotic apple harvesting in washington...

Supported By: National Institute of

Food and Agriculture

Robotic Apple Harvesting in

Washington State

Joe Davidsonb & Abhisesh Silwala

IEEE Agricultural Robotics & Automation Webinar

December 15th, 2015

a Center for Precision and Automated Agricultural Systems (CPAAS) & b School of Mechanical and Materials Engineering,

Washington State University (WSU)

1



Acknowledgements

This work was funded by the United States Department of

Agriculture – National Institute of Food and Agriculture (USDA-

NIFA) through the National Robotics Initiative (NRI).

2



Presentation Overview

• Motivation

• Working environment

• Design objectives

• Hand picking analysis

• System design

• Preliminary field testing results

• Future work

• Questions

3



Research Motivation• Washington State fresh market apple industry in 2014

– 2.7 million metric tons of apples valued at $1.84 billion USD1

– Accounted for 70% of U.S. apple production

• The WA fresh market apple harvest requires

– Employment of 30,000 additional workers

– An estimated cost of $1,100 to $2,100 USD per acre per year2,3

• Labor costs are rising and there is increasing uncertainty about the availability of

farm labor

• Lack of mechanical harvesting for fresh market apples is a significant problem

1 USDA National Agricultural Statistics Service. (2014 Washington Agriculture Overview). Retrieved November 2, 2015, from

http://www.nass.usda.gov/Quick_Stats/Ag_Overview/stateOverview.php?state=WASHINGTON

2 Galinato, S., & Gallardo, R. K. (2011). 2010 Estimated Cost of Producing Pears in North Central Washington (FS031E). Retrieved January 13, 2013, from

http://extecon.wsu.edu/pages/Enterprise_Budgets.

3 Gallardo, R. K., Taylor, M., & Hinman, H. (2010). 2009 Cost Estimates of Establishing and Producing Gala Apples in Washington (FS005E). Retrieved January 7, 2013, from

http://extecon.wsu.edu/pages/Enterprise_Budgets.

4



Long-term Goal: Reduce dependence on the labor force for

fresh market tree fruit harvesting

5



Working Environment

• Commercial apple orchard located in Prosser, WA

• Highly unstructured environment

• Modern cultivation systems with formal tree architectures

• “Fruit Wall” concept simplifies the task6



Design Objectives

• Cycle time < 6 sec

• Detachment success > 90%

• Fruit damage < 10%

• Pick multiple apple varieties

• Modular design that is cost-effective

• Our Approach: An ‘undersensed’ design that executes

look-and-move fruit picking, is mechanically robust

to position error, and replicates the human picking

process

7



Initial Design Development: Manual Apple Picking

• Fruit is grasped with a spherical power grasp4 with the index

finger applying pressure against the stem

• No dexterous manipulation of the fruit with the fingers

• To separate the apple from the branch, the hand moves the fruit

in a pendulum motion

4 Cutkosky, M. R. (1989) On Grasp Choice, Grasp Models, and the Design of Hands for Manufacturing Tasks. IEEE Transactions on Robotics and

Automation 5 (3): 269-279.

8



‘Undersensed’ Hand Picking5

• Are there effective methods to pick fruit that do not require fruit

orientation and stem location?

9

5 Davidson, J., Silwal, A., Karkee, M., Mo, C., & Zhang, Q. (2015). Hand Picking Dynamic Analysis for

Undersensed Robotic Apple Harvesting. Transactions of the ASABE. (Under review)

Representative Hand Picking Data


Food and Agriculture10

Mechanical Design



• Custom design

• 7 degrees of freedom

• Modular configuration

(Dynamixel Pro

actuators)

11

End-Effector Design



• Underactuation provides shape-adaptive

grasping

• Passively compliant joints enhance

robustness to position error & unplanned

collisions6

• Grasping is executed in an open-loop

manner

• Fabricated with additive manufacturing

12

6 Davidson, J., Silwal, A., Karkee, M., & Mo, C. (2015). Proof-of-Concept

of a Robotic Apple Harvester. Robotics and Autonomous Systems. (Under

review)

Vision System7



Color CCD Camera

PMD Camcube 3.0 (ToF, 3D camera)

IdentificationFusion

1

2

3 Localization

Camera Rig

13

7 Silwal, A., Gongal, A., & Karkee, M. (2014). Identification of Red Apples in Field Environment with

Over-the-Row Machine Vision System. Agricultural Engineering International: Agric Eng Intl (CIGR

Journal), 16(4), 66-75.

Experimental Setup8



8 Silwal, A., Davidson, J., Karkee, M., Mo, C., Zhang, Q., & Lewis, K. (2015). Design, Integration and Testing of a Robotic

Apple Harvester. Journal of Field Robotics. (To be submitted)

Hardware Architecture



Video



Vision Performance



Actual vs. Recovered Image17

Task Timing & Vision Accuracy



Vision Accuracy:

Total # of Images = 54

Total Fruit Manual Count: 193

Total Fruit Identified: 193

Identification Accuracy = 100%

Total Fruit in Workspace = 150

Average Fruit per Image = 4

Average Vison Time per Image = 6.3 s

Average Vision time per Apple = 1.7 s

18

Picking Results



• 127 of 150 fruits attempted were picked (approximately 85%)– 8/127 – No stems

– 33/127 – Spur attached to fruit

– 86/127 – Stems attached to fruit

• Misses fall into the following five general categories1. Poorly thinned branch (aka “fruit pendulum”) – 7 instances

2. Finger grabbed adjacent obstruction – 3

3. Position and/or calibration error – 8

4. Fruit slipped from grasp – 2

5. Previous fruit stuck in hand - 3

• No obvious evidence of bruising

• Ideal fruit location is 3 – 6 in away from the trellis wire

19

Picking Time



• Mean picking time – 6.01 sec/per fruit– 1st fruit in a cycle: 6.22 sec

– Remaining fruits in a cycle: 5.84 sec

• Each task in the picking sequence was segregated into an

individual function and timed

– Motion planning computation: 0.15 sec

– Approach: 2.14 sec

– Grasp: 1.5 sec

– Removal: 1.23 sec

– Fruit release: 1 sec

0

0.5

1

1.5

2

2.5

Motion

Planning

Approach Grasp Removal Fruit Release

Picking time

Tim

e

20

Future Work



• Higher level decision making based on detection of

trunks and trellis wires

• Grasp planning based on visual input

• Tactile sensor integration for detection of stem break,

missed fruits, etc.

21

Questions???



22

robotic apple harvesting in washington state - fieldrobot · robotic apple harvesting in washington...

Documents