crops: high tech agricultural robots - geyseco · crops: high tech agricultural robots ... a crops...

Proceedings International Conference of Agricultural Engineering, Zurich, 06-10.07.2014 – www.eurageng.eu 1/8

Ref: C0141

CROPS: high tech agricultural robots

Jan Bontsema, Jochen Hemming and Erik Pekkeriet, Wageningen University and Research Centre, P.O. Box 644, 6700 AP Wageningen, The Netherlands, ([email protected]) Wouter Saeys, Katholieke Universiteit Leuven, Dept. Biosystems, Kasteelpark Arenberg 30, BE - 3001 Heverlee, Belgium Yael Edan and Amir Shapiro, Ben-Gurion University of the Negev, POB: 653, Bee'r-Sheva, 84105, Israel Marko Hočevar, University of Ljubljana, Faculty of Mechanical Engineering, Aškerčeva 6, 1000 Ljubljana, Slovenia Thomas Hellström, Umeå University, Dept. of Computing Science, SE-901 87 Umeå, Swe-den. Roberto Oberti, Università degli Studi di Milano, DiSAA– Dept. Agricultural and Environmen-tal Sciences, via Celoria 2, Milano, 20133 Italy Manuel Armada, Centre for Automation and Robotics, CSIC-UPM, Ctra. Campo Real, Km. 0.200, La Poveda, Arganda del Rey, Madrid 28500, Spain Heinz Ulbrich and Jörg Baur, Technische Universität München, Institute of Applied Mechan-ics, Boltzmannstr. 15, Garching, 85748 Germany Benoit Debilde, CNH Industrial Belgium NV, Zedelgem, Belgium Stanley Best, INIA, Quilamapu, Chili Sébastien Evain, FORCE-A, Sensors & acquisition systems Department, Centre Universi-taire Paris Sud, Bâtiment 503, 91893 ORSAY Cedex, France Wolfgang Gauchel, Festo AG & Co. KG, Ruiter Str. 82, 73734 Esslingen, Germany Ola Ringdahl, Swedish University of Agricultural Sciences, Dept. of Forest Biomaterials and Technology, SE-901 83, Sweden

Abstract

In the EU-funded CROPS (Clever Robots for Crops) project high tech robots are developed for site-specific spraying and selective harvesting of fruit and fruit vegetables. The harvesting robots are being designed to harvest high-value crops such as greenhouse vegetables, fruits in orchards and grapes for premium wines. The CROPS robots are also developed for cano-py spraying in orchards and for precision target spraying in grape vines to reduce the use of pesticides. A CROPS robot will be able to detect the fruit, sense its ripeness, then move to grasp and gently detach only the ripe fruit. For crop protection the canopy sprayer can detect contours of trees in an orchard and consequently only spraying on the trees and the precision target sprayer can detect diseases on leaves of vine grapes and only spray pesticides on the af-fected spots of the leaves. In the CROPS project also attention is paid to reliable detection and classification of objects and obstacles for autonomous navigation in a safe way in plantations and forests. For the several applications within the CROPS project platforms were developed. Sensing systems and appropriate vision algorithms for the platforms have been developed. For the software platform the Robot Operating System (ROS) is used. A 9 degrees of freedom (DOF) manipulator was designed and built and tested for sweet-pepper harvesting, apple harvesting and in close range spraying. The 9-DOF manipulator is modular, since the joint configuration can be adapted to the applications, e.g. 6 DOF for the close range spraying. For the different applications different end-effectors were designed and tested. The main results of the CROPS project will be the applications, the so-called demonstrators.


For sweet pepper a platform that can move in between the crop rows on the common green-house rail system which also serves as heating pipes was built and equipped with a sensing and lightning system, the manipulator and end-effectors. The complete system was tested and showed to growers in a lab situation. The apple harvesting platform is based on a current mechanical grape harvester. In discus-sion with growers so-called 'walls of fruit trees' have been designed which bring robots closer to the practice. This system, equipped with a sensing system the CROPS manipulator and a special end-effector, has been successfully tested in an orchard. A canopy-optimised sprayer has been designed as a trailed sprayer with a centrifugal blower. The system has been successfully tested in an orchard with a significant reduction of pesti-cide use. For close range target spraying the spraying robot in a greenhouse experiment with grape vines reduced the pesticide consumption with 84%.

Keywords: robotics, harvesting, sweet pepper, apples precision spraying

1 Introduction

The main objective of CROPS (www.crops-robots.eu) is to develop a highly configurable, modular and clever carrier platform (Fig. 1) comprising a carrier plus modular parallel manip-ulators and “intelligent tools” (sensors, algorithms, sprayers, grippers) that can easily be in-stalled onto the carrier and that are capable of adapting to new tasks and conditions. Both the scientific know-how and a number of technological demonstrators will be developed for the agro management of high value crops like greenhouse vegetables, orchard fruits, and grapes for premium wines. The CROPS robotic platform will be capable of site-specific spraying (targeted spraying only on foliage and selected targets) and selective harvesting of fruit (i.e., it will detect the fruit, determine its ripeness, move towards the fruit and grasp it and softly detach it). Another objective of CROPS is to develop techniques for reliable detection and classification of obstacles and other objects to enable successful autonomous navigation and operation of the platform in plantations and forests. The rationale for this aspect of the project is that agricultural and forestry applications share many common research areas, primarily regarding sensing and learning capabilities.

Figure 1. Artist impression of the CROPS modular platform, including intelligent sensing, manipulation and end-effectors.

2 The hard- and software architecture

For the robotic middleware the Robotic Operating System (ROS) was chosen as software framework (Barth et al, 2014). The supervisory control system as well as the high-level soft-


ware architecture have been developed and tested. The scheme of the computer hardware platform is shown in Fig. 2.

Figure 2. Hard- and Software Architecture of the developed harvesting/spraying robot system.

3 Sensing

Requirements for the sensing systems for CROPS were derived. The design and implemen-tation of the sensory systems for CROPS have been completed, including the sensory sys-tem for detection and localisation of fruits in orchards and greenhouses, the sensory system for detection and classification of objects, the sensory system for fruit ripeness evaluation and the sensory system for diseases detection in crops (Fernández et al, 2013). In Fig. 3 examples of the detection of apples and grapes are given.

(a) (b)

Figure 3. Multispectral classification results for an apples crop scene, Red=apples, Green=leaves, White=background (a) and multispectral classification results for a vineyards scene, Pink=grapes, Green=leaves, Yellow=branches, White-background (b).

4 Manipulators and end-effectors

A nine degree of freedom manipulator was designed and manufactured (Bauer et al, 2012). End-effector prototypes were designed and tested. A first prototype of a canopy sprayer was manufactured. The first manipulator prototype, the grippers, and the precision spray end-effector (see Fig. 4a) were tested in laboratory and field experiments. Based on these tests a final manipulator was designed (Fig. 4b).


(a) (b)

Figure 4. First manipulator prototype with precision sprayer and protection cover (a) and a CAD ren-dering of the final design of the CROPS manipulator (b).

Several grippers were designed (Gauchel & Saller, 2012), some examples are shown in Fig. 5a and Fig. 5b

(a) (b)

Figure 5. Sweet pepper gripper (a) and grapes gripper (b).

5 Intelligent sensor fusion and learning algorithms

System architectures for sensing, grasping and sensor fusion and algorithms for sensor fu-sion and for learning in sensing and grasping were developed. The adaptive sensor fusion algorithm was implemented and tested for apples and sweet pepper (Vitzrabin & Edan, 2013). A learning framework was demonstrated to learn features for classification of forestry objects. A method for construction of a discrete fuzzy grasp affordance manifold based on learning from human demonstration was developed. In Fig. 6 an example of the outcome of the adaptive sensor fusion is given, together with some challenges.


Figure 6. Example of apple detection with adaptive sensor fusion.

6 Sweet pepper harvesting

For sweet pepper harvesting the requirements for a harvesting robot were obtained. A num-

ber of modules for hardware (sensors, grippers) and software (e.g. algorithms for sweet-

pepper fruit localization) were built and tested (Hemming et al, 2014). The manipulator, a

platform to transport the manipulator through the greenhouse, a gripper and a sensing sys-

tem were integrated into a complete system. This system was successfully tested and

demonstrated to growers in a laboratory setting (Fig. 7a) and in 2014 the first successful trial

was made in a commercial greenhouse, with the final prototype of the manipulator and with

the Festo gripper (Fig. 7b).

(a) (b)

Figure 7. Lab demonstration of the system to the grower’s advisory board on July 15th, 2013 (a) and the robot in the greenhouse, April 29th, 2014 (b).

7 Harvesting of apples and grapes

For harvesting of grapes and apples the requirements have been defined based on discus-sions with the growers. To maximize the visibility and reachability of the fruits, the so-called ‘walls of fruit trees’ growing system has been chosen, see Fig. 8


(a) (b)

Figure 8 Illustration of the improved visibility in flat planar canopies of apples (a) and grapes (b).

The Crops manipulator, grippers, sensors and software architecture have been tested both in laboratory as well as in apple orchards (Tien Thanh, 2013). All the modules have been inte-grated into one system, which was successfully tested in the laboratory and in the orchard, see Fig. 9.

(a) (b)

Figure 9. The robot platform for apple harvesting (a) and the CROPS manipulator inside the platform (b).

8 Precision spraying

For canopy optimised spraying and close range precision spraying requirements were se-lected in discussion with spraying specialists and growers. A canopy optimised sprayer was designed as a trailed sprayer with centrifugal blower. An eight DOF hydraulic driven manipu-lator with three arms was used. Orchard experiments were performed with the canopy opti-mised sprayer during 2013, see Fig. 10a. A good spraying quality was achieved with signifi-cant reduction of pesticide use (Osterman et al, 2013).


(a) (b)

Figure 10. Field experiments using the CROPS canopy optimized sprayer (a) and field test of close range precision spraying in a greenhouse (b). The close range precision spraying was focused on testing disease detection with various sensing principles. The Crops manipulator with waterproof protecting case, sensors precision spraying end-effector were integrated in a precision spraying robot for viniculture. The robot was successfully tested in a greenhouse environment and the attained pesticide reduction was 84% (Oberti et al, 2014), see Fig. 10b.

9 Forestry

For the forestry application the requirements for the detection of bushes, rocks, and trees and for the estimation of ground bearing capacity (for propulsion of forest machines) have been specified. A sensory system for detection and classification of trees has been evaluated in a field test (Ringdahl et al, 2013), see Fig. 11.

Figure 11. A sensory system for detecting and classifying trees was mounted on the cabin of a Valmet 931 harvester from Sveaskog. Photo: Ola Lindroos.

10 Conclusions

For sweet pepper a platform that can move in between the crop rows on the common green-house rail system which also serves as heating pipes was built and equipped with a sensing and lightning system, the manipulator and end-effectors. The complete system was tested and showed to growers in a lab situation and a first successful test was performed in a com-mercial greenhouse. The apple harvesting platform is based on a current mechanical grape harvester. In discus-sion with growers so-called 'walls of fruit trees' have been designed which bring robots closer


to the practice. This system, equipped with a sensing system the CROPS manipulator and a special end-effector, has been successfully tested in an orchard. A canopy-optimized sprayer has been designed as a trailed sprayer with a centrifugal blower. The system has been successfully tested in orchards with a significant reduction of pesticide use. For close range target spraying the spraying robot in a greenhouse experiment with grape vines reduced the pesticide consumption with 84%.

11 Acknowledgements

This research was (partly) funded by the European Commission in the 7th Framework Pro-gramme (CROPS GA no 246252).

References

Barth, R., Baur, J., Buschmann, T., Edan, Y., Hellström, T., Ringdahl, O., Salinas, C., & Vitz-rabin, E. (2014). Using ROS for agricultural robotics - design considerations and experienc-es. Proceedings RHEA-2014, Madrid, Spain, May 21-23, 2014. Fernández, R., Montes, H., Salinas, C., Sarria, J., & Armada, M. (2013). Combination of RGB and multispectral imagery for discrimination of Cabernet Sauvignon grapevine ele-ments. Sensors 2013 (13), 7838-7859. Baur, J., Pfaff, J., Ulbrich, H. & Villgrattner, T (2012). Design and development of a redun-dant modular multipurpose agricultural manipulator. Proceedings 2012 IEEE/ASME Interna-tional Conference on Advanced Intelligent Mechatronics (AIM), Kaohsiung, Taiwan, July 2012, 823-830. Gauchel, W., & Saller, S. (2012). Adaptive gripper jaws for high-value crops harvesting (2012).Proceedings 8th International Fluid Power Conference, Dresden, Germany, March 26-28, 2012. Vitzrabin, E., & Edan, Y. (2013), Apple detection using multi-dimensional adaptive threshold-ing with multi-resolution windows. Proceedings 4th Israeli Conference on Robotics, Tel Aviv, Israel, 19-20 November 2013. Hemming, J., Ruizendaal, J., Hofstee J.W., & Van Henten, E.J. (2014). Fruit detectability analysis for different camera positions in sweet-pepper. Sensors 2014 (14), 6032-6044. Tien Thanh, N., Kayacan, E., De Baerdemaeker, J., & Saeys, W. (2013). Task and motion planning for apple harvesting robot. Proceedings Agricontrol 2013, Helsinki, Finland August 28-30, 2013. Osterman, A., Godeša, T. Hočevar, M., Širok, B., & Stopar, M. (2013). Real-time positioning algorithm for variable-geometry air-assisted orchard sprayer. Computers and Electronics in Agriculture, 98 (2013), 175–182. Oberti R., Marchi, M. Tirelli, P., Calcante, A., Iriti, M., Hočevar, M., Baur, J., Pfaff, J., Schütz, C., & Ulbrich, H. (2014). Crops agricultural robot: application to selective spraying of grape-vine’s diseases. Proceedings RHEA-2014, Madrid, Spain, May 21-23, 2014. Ringdahl, O., Hohnloser, P., Hellström, T., Holmgren, J., & Lindroos, O. (2013). Enhanced algorithms for estimating tree trunk diameter using 2D laser scanner. Remote Sensing, 5 (10), 4839-4856.

crops: high tech agricultural robots - geyseco · crops: high tech agricultural robots ... a crops...

Documents