seminar i on agricultural process engineering 農産 …...seminar i on agricultural process...
TRANSCRIPT
Seminar I on Agricultural Process Engineering農産加工学演習 I
No.6
農学研究科 地域環境科学専攻農学研究科 地域環境科学専攻
近藤 直・清水 浩近藤 直・清水 浩
Naoshi Kondo, Hiroshi ShimizuDivision of Environmental Science & Technology,
Graduate School of Agriculture, Kyoto University
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Applications as robotic eyes
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Strawberry harvesting robot
Color conversionPreprocessingPreprocessing
BinarizationBinarization
Processing ofProcessing ofbinary, gray levelbinary, gray levelimagesimages
Feature extractionFeature extraction
RecognitionRecognition
3 3 D understandingD understanding
Software of machine vision as robotic eyes
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Stereo imaging for Depth measurement for 3 D reconstruction
Active stereo visionPassive stereo vision
Stereo geometryScene pointP
Z
Xf f
pL pR
b
xL xR
CL CR
bf
: Baseline: Focal length: disparity: P’s x coordinate on left camera: P’s x coordinate on right camera: Lateral offset: Distance from camera
XZ
d
Identical parallel cameras
xL
xR
d = ‐ =Z
f
Zf=
XxL
Zf=
xR +
,
d is proportional to f and binversely proportional to Z
Image plane
xLxR
X b
b
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Area-based stereo matching(use of larger image regions (or areas) that containenough information to yield unambiguous matches)Feature-based stereo matching(Feature extraction by color or edge detection and dealwith only points that can be matched unambiguoisly)
Correspondence problem
Occlusion problem, Ambiguous matching
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
1) Kise M., et al. : A Stereovision-based Crop Row Detection Method for Tractor-automatedGuidance, Biosystems Engineering, 90(5) 357-367 (2005).
Stereo cameraStereo camera mounted tractor 1)
Area-based stereo vision
Disparity image(darker pixel is farther)
Image size: 320 X 240Mask size m: 25 X 25d’= 0~32 (setting value)
E(d’) = Σ Σ |IL(x+i, y+j) - IR(x+i+d’, y+j)|
L R
i=-m/2
i=m/2 j=m/2
j=-m/2
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Feature-based stereo vision
Dx
Dy
b
P1
P2
Px d
Y
X
Dy=d b/(P2-P1) Dx=Px Dy/d Dz=Py Dy/d
Q1: Describe how we can getQ1: Describe how we can get Dy Dy==d bd b/(/(PP22-P-P11))
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Left Right
Strawberry fruit images from stereo vision
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Left Right
Matching criteria: Horizontal level and strawberry sizeshould be almost equal
Matching on binary images
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Cluster of Strawberry fruits
1
2
3
4
56
56
4
3
2
Left Right
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
1
2
3
4
56
5
6
4
3
2
Left Right
No 1: out of view in right imageNo.3: occluded on right imageNo.4: different immature parts from different angle
Un-Matched fruits (No.1, 3, 4)
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Depth measurement by use of differential object size
Dy=P1 L/(P2-P1)Dx=P2 Dy/d
Dy=L√Na1/(√Na2‐√Na1)
Dy= Nl1L/(Nl2‐Nl1)
P1
DyL
Dx
P2
d
DyL
Na1
Nl1
Na2
Nl2
Nai: area of object on image Nli : length of object on image
Q2:Describe how we can getQ2:Describe how we can get Dy= Nl1L/(Nl2‐Nl1)
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Na1: pixel numberrecognizing fruit
at position P1 at position P2
Actual images from camera attached to manipulator end
Dy=L√Na1/(√Na2‐√Na1)
Na2: pixel numberrecognizing fruit
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
3 D image from active range finder
Operating principleOperating principle(Time of flight)(Time of flight)
TransmitterReceiver
Rotating mirror
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
3 dimensional shape recognition
Phyllotaxis
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Machine vision forfruit grading system
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Extracted features from images
SizesColorsShapeDefectsMaturity Dullness (Gloss)……
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
An apple fruit with discoloration
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Color component images
R G B
G‐BR‐BR‐G
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
FFT
Internal defects detection by X-ray CT
Peach with
Split-pits
Moth suckeddefect
Rotten core
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Experimental result(sprit-pit of peach)
(a) Appearance (b) Cut sample (Sprit-pit)
(d) Top view(c) Side view
Transparent ImageOutput Voltage: 50keV
Y=250
Intersection at Y=250
Gra
y Le
vel
CT Image
Split-pit
Peach with Split-pits
X ray image
X ray GeneratorMonochromeTV camera
Scintillator
Rotten core onion
Hollow potatoes
X-ray images of orange fruits
Empty portion
Top view Side view
Empty portion
Rind-puffingfruit
Normalfruit
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Ultra violet image
Visible camera
UV camera
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Review
Describe what was the phenomenon Describe what was the phenomenon ““fluorescencefluorescence””..
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Rotten part
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Fluorescence image
Fluorescence reaction
Rotten part
Color images fluorescence images
(365nm exciting light)
Rotten part
UV LEDs
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Infrared
Light
Rotten portion
Transmitted image
Texture on bioproducts
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Cooccurrence matrix and textural features
Haralick, R.M., K. Shanmugam and Its'hak Dinstein. Textural features for image classification,IEEE Transactions on systems, man, and cybernetics, Vol.SMC-3, No.6, 610-621.1973.
1
2 3
0 00 00 2
22 2
11 1
3Gray level image
0 1 2 30123
4 2 1 02 4 0 01 0 6 10 0 1 2
d=1, θ =0
Distance and direction
dθ
i j
jj
ASM:ΣΣ{p(i,j)} 2i=0 j=0
n n
ΣΣ p(i,j)/{1+(i-j)2}IDM:i=0 j=0
n n
ΣΣ (i-j)2p(i,j)CON:i=0 j=0
n n
Measurement Box Power source, PC,
Spectroscopy,PLC, amp board
Antenna of DGPS
Chisel(sensorprobe
housing)
※ Date: 2004. 11. 21 Depth: 150mm, Speed: About 30cm/ secCooperation with SHIBUYA MACHINERY CO.,LTD., TUAT
SAS1000 – Outline (Real time soil sensor)
Ready to sell!
Measurement
Nitrogen MC SOM EC pH Compaction + Image data
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Illumination fiber
EC Electrode
Laser Displacementsensor Color camera
CondensedfiberGround surfaceGround surface
Soil flattener
Arrangement of equipments in chisel
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Under-ground-Images by the soil sensor
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
a b c
d e f
g h i
(max{a,b,c,d,e,f,g,h,i}-min{a,b,c,d,e,f,g,h,i})×k
(a+b+c+d+e+f+g+h+I)/9
Textural analysis
USAD-ARS Imaging Research TeamUSAD-ARS Imaging Research Team
RDA 2004 Symposium October 13, 2004
Common ApertureCamera
Carcass
IntelligentBird Washer
Method & System for Intelligent Washing ofFecal & Ingesta Contaminants
RDA 2004 Symposium October 13, 2004
Multispectral imaging
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Machine vision for seedling production
Cutting sticking robot Transplanting robot
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
A fruit grading robot with machine vision
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
Assignment
Describe how you may apply the imagingDescribe how you may apply the imagingtechnologies to your own research project.technologies to your own research project.
KYOTO UNIVERSITY京都大学
AgriculturalAgriculturalProcessProcessEngineeringEngineeringLaboratoryLaboratory
For improving my lecture
Tell me anything (teaching techniques) I canTell me anything (teaching techniques) I canimprove for your understanding in myimprove for your understanding in mylecture. (For example, small voice, too fastlecture. (For example, small voice, too fastPPT page changing, hard to see letters onPPT page changing, hard to see letters onblackboard, more explanation, use moreblackboard, more explanation, use moreblackboardblackboard……..)..)