agra webinar - agrofood robotics - from farm to tableieeeagra.com/ieeeagra/downloads/20140926-van de...

28-9-2014

1

Agrofood Robotics and Automation:

From Farm to Table

Rick van de Zedde

Wageningen UR – 26th of September 2014

IEEE RAS TC on Agricultural Robotics and Automation Webinar #021 (AgRa)

Introduction

Rick van de Zedde, business developer/ project leader for 10 years at Wageningen University & Research centre

in The Netherlands

Background:

Artificial Intelligence, University of Groningen.

Focus: computer vision/ robotics

Contact: [email protected]

28-9-2014

2

Wageningen University & Research Centre

A university plus R&D organisations, mission statement;

“To explore nature and to improve the quality of life”.

Wageningen UR:

● 6.500 employees

● 8.000 students

● 1.900 PhD’s

● 106 countries

Wageningen UR - campus

Wageningen UR - GreenVision

● GreenVision - the Wageningen UR centre of expertise on computer vision.

● Introduce new technology / scientific novelties into the agrifood industry together with industrial partners.

● 25 computer vision researchers within Wageningen UR.

Coordinated by: Rick van de Zedde, Erik Pekkeriet, Gert Kootstra and Gerrit Polder

● One of the largest computer vision research groups in the (Dutch) agrifood industry.

http://greenvision.wur.nl


http://greenvision.wur.nl/



28-9-2014

3

Outline

Food inspection: from farm to table

Farm/ breeding/ phenotyping

Post-harvest quality inspection

Packaging and robotics

Retail

Future perspective of R&D


Outline





Retail



28-9-2014

4

Mechanical intra-row weeders

Erik Pekkeriet (PL), Pieter Klop, Jochen Hemming, ea.

MARVIN – 3D based seedling sorting

Rick van de Zedde (PL), Gerwoud Otten, Franck Golbach, ea.

28-9-2014

5

Specs, Scale and speed

Current capacity: 19.500 seedlings/ hour is 185 ms/ seedling

10 GigE industrial machine vision camera’s, hardware triggered.

3D reconstruction technique: shape-from-silhouette/ volumetric

intersection which requires a very accurate 3D camera calibration.

Software runs on a fast Windows 7 desktop computer;

National Instruments Labview/ core engine using CINs (C/C++).

Database: SQL server/ .Net web-interface

Raw data collection issue - ±5 seedlings per second =

10 camera’s x 5 per second x 1.2MB per image = 60 MB / second

216 Gb / hour .... saving 3D models only = 0.5 Gb / hour

28-9-2014

6

Plant phenotyping and food production

Challenge: produce food for 9 billion in 2050

Focus on improvement of crops (maize, rice, potatoes, tomatoes, etc).

Novel genotyping technologies ‘deliver’ new varieties much quicker;

● Faster and less expensive DNA sequencers

● More efficient breeding cycles (GM and ‘classical’)

Plants still need to be grown to determine yield, resistance against heat/drought stress, diseases , etc. (= phenotyping).

So an increase of capacity/ objectivity is required:

Opportunity for automated inspection, robotics, big data.


European Plant Phenotyping Network (EPPN) – 5.5 M€

Goals:

1. Create a European integrated network/ community

2. Offer trans national access to EPPN facilities

3. Research – a. Novel sensors, b. Good practice phenotyping c. IT for high throughput

Website: www.plant-phenotyping-network.eu Wageningen UR is WP leader of WP3 Novel sensors

Grant Agreement No. 284443.

http://www.plant-phenotyping-network.eu/





28-9-2014

7

28-9-2014

8

28-9-2014

9

Vision-guided robotics

Goal: propagating roses

from cuttings

3D reconstruction

and plant architecture.

Rick van de Zedde (PL), Sanja Damjanovic, Gerwoud Otten, ea.

28-9-2014

10

Rose robot – automated planting

Wageningen UR – Phenobot

Gerrit Polder (PL), Fred van Eeuwijk, Marco Bink, Gerie van der Heijden, ea.

28-9-2014

11

Outline





Retail




Automation in fruit/ vegetable production is widely used.

● Apple, oranges, mango’s, etc. using optical sorters.

Individual products are analysed and graded.

Common inspection method: several images of one product while continuously rotating on ‘wheels’.

Alternative approach: 3D reconstruction and shape analysis developed for bell peppers (irregular shapes)


28-9-2014

12

Laser triangulation: shape measurements


3D shape analyses – multiple views

Result of 6 sides Result based on one view Feature calculation like: Shape - identify middle of lobes Shape analysis: Block, point or ... shape Contact: [email protected]

28-9-2014

13

3D shape analyses of bell peppers

Quality criteria:

● Length/width

● Diameter

● Number of lobes

● Shape regularity

● Curvature

NB: additional sensors

required for colour/ defects.


Lettuce handling with robots

Kolen orienteren video hier toevoegen


28-9-2014

14

Outline



Post-harvest quality inspection (2)


Retail



Optical bulk sorting


28-9-2014

15

Video - impression

Franck Golbach (PL), Gerwoud Otten, Roeland v. Batenburg, ea.

Recording quality in high-speed mode


28-9-2014

16

Specs

Detection and reject of products based on

● Colour defects (brown/ yellow/ green spots)

● Shape (length, width, curvature, complex shapes).

Capacity:

● Conveyor belt speed up to 5 m/s

● 10.000 objects/ second, monitored with 4 camera’s.

● 20 - 30 tons per hour (with French fries) = 3 truck loads

4 linescan camera’s (2k pixels)/ Matrox Solios eCL framegrabbers/ PC-cluster with windows 7 plus Linux

Patented ‘intelligent puffing’ product removal– shape based reject


Hyperspectral / NIRS

Measurement of quality aspects such as:

● Diseases/ product quality

● Moisture/ starch content

● Foreign materials in bulk streams

Non-destructive and very fast (1–25 ms)

Hardware:

● Spectrometers (260 – 2500 nm)

● Hyperspectral line-scan NIR camera

for bulk sorting applications.


28-9-2014

17

Hyperspectral NIR camera

Near InfraRed linescan camera (940 – 1790 nm)

● Xenics XEVA-343 xc104 - Specim N17E30 μm slit.

● Spectral resolution: 256 pixels - 3.3 nm/ pixel.

● Spatial resolution: 320 pixels.

● 100 frames (lines)/ second.

NB: multispectral RGBi camera’s have colour channels from 400-700 nm and a near-infrared (NIR) channel at 750-900nm.


Hyperspectral imaging

Consider hyperspectral imaging when product and defects have:

● No density difference (no x-ray)

● No clear colour differences (no RGB)

● Quantitative measurements (ie. moisture content, fat content, inner decay)

Warnings:

● Expensive hardware - InGaAs sensor instead of CCD/CMOS.

● Sensitive hardware - calibration/ humidity/ damage.

● Training set should cover all ‘real-life’ occurrences. NB:

● Seasonal differences will change the product

● Robust for several varieties of the product.

● Sensitive for the relevant range within product?


28-9-2014

18

Outline





Retail



Food processing factories have to be flexible:

● Large number of products and packaging variations

● Small batches

● Retailers place theirs orders late

Food processing industry


28-9-2014

19

To meet increasing demands:

● Enormous amount of manual labour

● People are flexible

Robots/machines are not flexible (yet)

Food processing industry


EU-project - PicknPack

Large-scale EU funded research project

Coordinator:

Wageningen UR

(Erik Pekkeriet)

Budget: 14 M€

Consortium:

14 universities, research institutes, companies incl. retail.

www.picknpack.eu

Reducing manual labour in quality assessment and packaging of food products.


http://www.picknpack.eu/

28-9-2014

20

Flexible robotic systems for automated adaptive

packaging of fresh and processed food products

Sensing module Robot module Packaging module

PicknPack – demonstrator

PicknPack demonstrator

Pick-and-place demo of vine tomatoes

Dedicated gripper

28-9-2014

21

Outline





Retail


EasyFlow – automated check-out

Concept: Supermarket with no barcodes. Fully automated check-out to identify 30,000 different products.

Solution

Integration of machine learning based on multiple sensors using: computer vision, NIR technology, weight and statistics. Continuous self learning – shared with other Easyflow check-outs

Result Hitrate 99%, better than a human cashier. Benefits: less personnel costs, less fraud, better and faster service for supermarket customers.

28-9-2014

22

ITAB – Match X project

Nicole Koenderink (PL), Don Willems, Roeland vBatenburg, ea.

Outline





Retail


28-9-2014

23


Focus on automation of plant phenotyping activities

● Ultra high-throughput phenotyping machines

● Complex plant features, ie. 3D shape, # leaves, etc.

● Robotics to replace manual labour

● Big data hardware/ techniques to analyse results.

Increase of machine learning in agrifood for

● Quality grading of food products based on complex

combinations of features (sensor fusion).

● To optimize device parameters – not all operators get the

‘max’ out of their machine.


An increase in vision guided robotics developments in agrifood

industry, driven by :

● Only alternative for Europe/ North-America to compete

as a ‘production location’.

● Price of hardware is decreasing - quicker ROI .

● Need for objective and stable product quality

● Need for more production capacity and flexibility

● Lack of skilled and motivated manual labour

28-9-2014

24

Acknowledgement

Colleagues at Wageningen UR:

● Erik Pekkeriet

● Eldert van Henten

● Gerrit Polder

● Gerwoud Otten

● Franck Golbach

● Nicole Koenderink

● Sanja Damjanovic

● Mari Wigham

● Gert Kootstra


Plus industrial partners

Lacquey (R. vd Linde):

Optiserve (A. v Kasteren):

Enza (M. Klooster):

IsoGroup (P. Oomen):

WPK (E. vd Arend):

ITAB (J. Möller):

EU projects:

Thank you for

your attention!

Questions?

More info:




agra webinar - agrofood robotics - from farm to tableieeeagra.com/ieeeagra/downloads/20140926-van de...

Documents