university teknikal malaysia...
TRANSCRIPT
UNIVERSITY TEKNIKAL MALAYSIA MELAKA
DEVELOPMENT OF AUTOMATIC QUALITY CONTROL
SYSTEM USING VISION SYSTEM
This report submitted in accordance with requirement of the Universiti Teknikal
Malaysia Melaka (UTeM) for the Bachelor Degree of Manufacturing Engineering
(Robotics and Automation) with honours
by
CHIN GEE SING
B050710006
FACULTY OF MANUFACTURING ENGINEERING
2011
DECLARATION
I hereby, declared this report entitled “Development of Automatic Quality Control
System Using Vision System” is the results of my own research except as cited in
references.
Signature :
Author’s Name : CHIN GEE SING
Date : 18 MAY 2011
APPROVAL
This report is submitted to the Faculty of Manufacturing Engineering of UTeM as a
partial fulfillment of the requirements for the degree of Bachelor of Manufacturing
Engineering (Robotics and Automation) with honours. The member of the supervisor
committee is as follow:
……………………………………….
Supervisor
i
ABSTRAK
Mesin sistem visi banyak digunakan dalam industri untuk tujuan pemeriksaan.
Sistem menyusun automatik dan sistem pemeriksaan visi yang dilaksanakan untuk
integrasi telah membina sistem automasi. Kecekapan ia memeriksa dan menyusun
secara teratur telah member banyak faedah untuk mengatasi kelemahan sistem
konvensional yang dijalankan oleh pekerja. Integrasi konveyor ke sistem yang
dicadangkan adalah untuk menghantarkan objek untuk diperiksa. Kemudian, sistem
pemeriksaan dimulakan dari pengesanan sensor fotoelektrik untuk mengaktifkan
“webcam” menangkap gambar melalui satu PLC. Gambar yang ditangkap akan
didigitalkan oleh “webcam” dan menghantar ke hos komputer untuk pemprosesan
gambar dengan menggunakan perisian Vision Builder Automated Inspection (VBAI).
Konfigurasi untuk perisian VBAI akan dipelajarikan sebelum pemeriksaan visi
dijalankan. Sementara itu, grafik antara muka pengguna direkakan dengan
menggunakan perisian Labview tujuan untuk pengguna mengawasi and
mengawalkan sistem tersebut. Objek yang diperiksa akan disusun secara teratur oleh
robot. Ia adalah satu robot enam paksi yang mempunyai ruang kerja yang berbentuk
separuh-bola. Namun, integrasi elektronik seperti papan “relay” adalah diperlukan
untuk menyelesaikan masalah ketidaksamaan voltan antara PLC dan prosesor Rhino.
Sebagai kesimpulan, projek ini telah dijalankan untuk mengklasifikasikan objek
dengan menggunakan pemeriksaan secara visual dan eksperimental validasi ke atas
prestasi sebenarnya.
ii
ABSTRACT
Machine vision systems are widely used in manufacturing industries for inspection
purpose. The implementation of automated sorting system to machine vision system
has develops an integration of automation system. Its efficiency to inspect parts and
sort according to order offers advantages which against the limitation of conventional
human-operated system. The integration of a conveyor to the proposed system feeds
the inspected parts for inspection purposes. Then, the inspection system is started
from a detection of photoelectric sensor to trigger USB webcam by image acquisition
via a Programmable Logic Controller (PLC). The webcam digitized the image data in
order to send to host computer for image processing by using Vision Builder
Automated Inspection (VBAI) software. The configuration of VBAI is studied on a
personal computer before vision inspection is carried out. On the other hand, a
graphical user interface (GUI) is designed by using Labview for person-in-charge to
monitor and control the entire system. After classification from VBAI software, the
inspected parts are arranged accordingly by Rhino robot. It is a six axis robot arm
which consists of a hemisphere of work space. Though, integration of electronic
hardware such as relay boards are necessary to solve the different voltage signal
between PLC and Rhino controller. As a result, this project is performed to classify
parts by using developed visual-based inspection and experimental validated its
actual performance.
iii
ACKNOWLEDGEMENT
I would like to thank for peoples who guided and contributed in various ways to this
PSM I report. This report would not have been possible without the expert advice
and suggestion of my PSM supervisor, Dr. Zamberi bin Jamaludin. Thank you Dr.
Zamberi for the invaluable information and suggestions that you provided and your
willingness for contribute your precious time to this effort. Besides that, I also thanks
to laboratory technician, En. Faizul and Pn. Norhafisah for giving me technical
support.
Thanks also to the Universiti Teknikal Malaysia Melaka (UTeM) for offering PSM
as a compulsory subject for students to gain skills on organizing a project.
Beyond the professional involved in this report, my coursemate and family were also
quite influential in this work. Thanks to my coursemate for sharing their precious
opinion, experience, comments and resources. Thank you for being there to listen to
my gripe about my work. Thanks for my beloved family unending support and
motivation.
Finally, thanks for the peoples who were there to lend a hand along the way. Thank
you.
iv
DEDICATION
I dedicate this report to my beloved parents, without their patience, support
understanding and most of all loves, the completion of work would have not been
possible.
v
TABLE OF CONTENT
Abstract
Abstrak
Dedication
Acknowledgement
Table of Content
List of Tables
List of Figures
List Abbreviations
1. INTRODUCTION
1.1 Background
1.2 Problem Statement
1.3 Objectives
1.4 Scope
1.5 Expected Result
1.6 Content
2. LITERATURE REVIEW
2.1 Introduction
2.2 Vision System
2.3 Machine Vision Component
2.3.1 Vision Sensor Types
2.3.1.1 CCD
2.3.1.2 CMOS
2.3.1.3 Comparison between CCD and CMOS
2.3.2 Image Processing Technique
2.3.2.1 Smoothing Technique
2.3.2.1.1 Lowpass Filter
2.3.2.1.2 Local Average
2.3.2.1.3 Gaussian Filter
i
ii
iii
iv
v
ix
x
xiii
1
1
2
2
3
3
3
5
5
5
7
9
9
14
17
18
20
20
21
21
vi
2.3.2.1.4 Median Filter
2.3.2.2 Edge Detection Technique
2.3.2.2.1 Prewitt Operator
2.3.2.2.2 Sobel Operator
2.3.2.2.3 Roberts Operator
2.4 Sorting System
2.4.1 Robot
2.4.2 Types of Robot Manipulator
2.5 Conveyor
2.5.1 Traditional Conveyor
2.5.2 Evolution of Conveyor
2.6 Measurement Devices
2.6.1 Analogue-to-Digital Converter
2.6.2 Photoelectric Sensor
2.7 Programmable Logic Controller (PLC)
2.8 Summary
3. METHODOLOGY
3.1 Introduction
3.2 Overall Research Strategies
3.3 Project Structure
3.3.1 Image Processing Procedures
3.4 Characterization of System Components
3.4.1 Conveyor
3.4.2 Rhino Robot Arm
3.4.3 Webcam
3.4.4 Personal Computer
3.4.5 Photoelectric Sensor
3.4.6 Programmable Logic Controller (PLC)
3.5 Summary
4. EXPERIMENTAL SETUP AND PROCEDURES
4.1 Introduction
21
22
23
24
24
25
26
27
29
30
30
31
32
33
36
38
39
39
39
43
45
46
47
48
49
49
50
52
52
55
55
vii
4.2 Experimental Setup
4.2.1 Hardware Component
4.2.2 Software Component
4.2.2.1 Logic of Electrical Control System
4.2.2.2 PC Access
4.2.2.3 Graphical User Interface (GUI)
4.2.2.4 Image Processing Configuration
4.2.2.5 Pick and Place Configuration
4.3 Experimental Setup Procedures
4.4 Summary
5. RESULT AND DISCUSSION
5.1 Introduction
5.2 Results
5.2.1 Ladder Diagram
5.2.2 Development of I/O Server
5.2.3 Graphical User Interface Design and Data Binding
5.2.4 Vision Inspection
5.2.5 Sorting
5.3 Discussion
5.3.1 Circuit Board for Relay
5.3.2 Matched Timing and Sequence between Robot Arm
and Image Acquisition
5.3.3 Numeric Indicators
5.4 Suggestion
5.4.1 Robot Feedback
5.4.2 Robot Efficiency
5.5 Summary
6. CONCLUSION
REFERENCES
55
55
58
58
60
62
63
66
67
68
69
69
69
70
71
72
74
75
75
76
77
77
78
78
79
79
80
82
viii
APPENDICES
A Ladder Diagram
B Rhino Robot Programming
C Block Diagram of GUI
85
86
90
92
ix
LIST OF TABLES
2.1 Feature and Performance Comparison of CCD and
CMOS
2.2 Correction Factor for Direct Reflective Photoelectric
Sensor
3.1 Gantt Chart for FYP I
3.2 Gantt Chart for FYP II
4.1 Step for PLC Configuration
4.2 Configuration of PC Access
4.3 GUI Configuration of Labview
4.4 Image Processing Configuration on VBAI
4.5 Rhino Robot Configuration using RoboTalkWindow
5.1 Condition for Combination of Colour and Shape
18
35
53
54
59
61
62
64
66
70
x
LIST OF FIGURES
2.1 Basic Functions of Machine Vision System
2.2 The Coordinate Convention of Pixels That Represent Digital
Images
2.3 The Basic Components of a Vision System
2.4 The Example of CCD Image Sensor
2.5(a) CCD Pixel
2.5(b) Architecture
2.6 Basic Principle of The Charge-Coupled Device
2.7 CMOS Sensor is Implemented in Low-End Web-Cam
2.8 CMOS Vision Sensor Floorplan
2.9 Schematic of The RGB Colour Cube
2.10 The Level of Histogram of Gray Scales
2.11 Calculating The Median Value of a Pixel Neighbourhood
2.12 A 3 × 3 Region Mask
2.13 Prewitt Horizontal and Vertical Operators
2.14 Sobel Convolution Kernels
2.15 The Computation Method of a Pixel
2.16 Roberts Cross Convolution Masks
2.17 The Closed-Loop Control System
2.18 Cartesian Robot
2.19 Cylindrical Robot
2.20 Spherical Robot
2.21 Articulated Robot
2.22 Traditional Conveyor
2.23 Steps in Analog-to-Digital Conversion of Continuous Analog
Signals from Process
2.24 Thru-Beam Photoelectric Sensor Detection
2.25 A Reflection with Reflector Photoelectric Sensor Detection
Method
2.26 A Direct Reflection Photoelectric Sensor Detection Method
6
7
8
10
12
12
13
14
15
19
19
22
23
23
24
24
25
27
27
28
28
29
30
32
34
34
35
xi
2.27 Siemens Programmable Logic Controller, S7-200
2.28 Components of a PLC
3.1 Overall Work Flow Chart
3.2 The Flow Chart Illustrates The Data Collection and Data Study
3.3 The Flow Chart of Proposed System
3.4 Image Processing Flow Chart
3.5 The Schematic Diagram of Integrated System
3.6 A Belt Conveyor
3.7 Rhino Robot Arm
3.8 A Home-Used Webcam
3.9 A Personal Computer
3.10 Photoelectric Sensor
3.11 Siemens S7-200 PLC
4.1 CAD Drawing for Webcam Attachment on Conveyer
4.2 Actual Frame Mounting on Conveyor
4.3 Designed Logic Circuit Board
4.4 Overall Experimental Setup
5.1 Actual Sorting Result That is Performed by Rhino Robot
5.2 Designed Ladder Diagram
5.3 Develops and I/O Server Module using PC Access
5.4 GUI of The Integrated System
5.5 Select The Source Item in Data Binding
5.6 State Program for Vision Inspection
5.7 Inspection Step for State: Circle Red: x1
5.8 Relay Circuit Board Used to Triggered PLC and Motor of
Conveyor
5.9 Relay Circuit Board Used to Trigger Robot from PLC Input
Signal
5.10 Numeric Indicators for Accept and Reject Counting
36
37
41
42
44
46
47
47
48
49
50
51
52
56
56
57
68
70
71
72
73
73
74
75
76
77
78
xii
A1 Ladder Diagram
A2 Ladder Diagram
A3 Ladder Diagram
A4 Ladder Diagram
A5 Ladder Diagram
C1 Block Diagram for GUI
85
86
87
88
89
92
xiii
LIST OF ABBREVIATIONS
ADC - Analogue-to-Digital Converter
APS - Active Pixels
CAPS - Complementary Active Pixels Sensors
CCD - Charged-Couple Device
CCTV - Close Circuit Television
CID - Charge-Injection Devices
CMOS - Complementary Metal-Oxide Semiconductor
DOF - Degree of Freedom
GUI - Graphical User Interface
I/O - Input/Output
LED - Light Emitter Diode
OLE - Object Linking and Embedding
OPC - OLE for Process Control
PC - Personal Computer
PCB - Printed Circuit Board
PIC - Person-in-charge
PLC - Programmable Logic Controller
PMOS - P-type metal-oxide-semiconductor
PPP - Prismatic, Prismatic, Prismatic
PPS - Passive Pixels
PVC - Polyvinyl Chloride
RGB - Red, Green, Blue
RPP - Rotary, Prismatic, Prismatic
ROI - Region of Interest
RRP - Rotary, Rotary, Prismatic
RRR - Rotary, Rotary, Rotary
USB - Universal Serial Bus
VBAI - Vision Builder Automated Inspection
VDC - Voltage Direct Current
VLSI - Very-large-scale integration
xiv
2-D - Two dimensions
3-D - Three dimensions
1
CHAPTER 1
INTRODUCTION
1.1 Background
The essential goals in the implementation of automation system in the industries are
to provide improvement in human operator variances and operational efficiency. It is
known in that many cases, product quality consistency can vary depending on the
characteristic of human operators who are on duty. Hence, automation system should
be provided to improve this variance. The problem associated with human-assisted
operation is characterized by cycle time, repeatability and material handling.
Although operational efficiency has been given a top priority, human operators are
still exposed to high probability of injuries, long-term similar working scope in fast
motion, applications naked eyes inspection-based system and insufficient break time.
In order to improve operational efficiency and to improve safety of human operators,
there is a need for efficient, high repeatability and cost effective human-assisted
operation equipment. Mechanized and automized solution that are properly sized
with suitable modularized structure can greatly improve the safety of personnel as
well as efficiency and quality of production. In addition, such efficient automation
system with high repeatability can greatly improve the operational efficiency,
product quality and product consistency when applied in combination with intelligent
instrumentation and advanced control.
The mechanized and automized operation system include robot arm, programmed
conveyor system, inspection system and host computer are necessitated to implement
in industries purposely to increase operational efficiency and product quality. A 6
degree of freedom (D.O.F) of robot arm which possess wide working area has
2
enlarge the approaching region and various orientation. Therefore, it is a wise
solution to be used to substitute the folklore operation system like human operator in
industries by robot arm. Besides, the evolution of conveyor from manual operate to
automated conveyor system has enhanced the product handling and increase the
cycle time as well. The implementation of vision system for inspection in production
line becomes a booster for inspection compare to old-fashion naked eye inspection.
The mechanized and automized systems are integrated with intelligent vision system
enable the whole production processes execute in fully automatic. Thus, the
operational efficiency is increase effectively and greatly decreases the variances of
product quality.
1.2 Problem Statement
Nowadays, human-operator based industries require longer time to inspect using
naked eye and sort the products into the correct slot or container. This action of
inspection and sorting consume more time and results in the operators unable to meet
the cycle time. Low repeatability is one the effects of long working time on similar,
repeating working scope at fast motion. Therefore, the operators may get tired and
bored resulting in bad inspection and sorting quality of the products. Due to the
limitations, an automated inspection system based on a vision system integrated into
a pick-and-place robot is required.
1.3 Objectives
Based on the problem statements stated above, several objectives are to be obtained
at the end of this project. The objectives of this project are:
i. To design visual-based inspection system
ii. To classify and sort products using vision inspection system
3
1.4 Scope
This project will focus on vision-based inspection and products sorting. The scopes
of this project are:
i. Image acquisition using a webcam.
ii. Configure vision inspection algorithm based on Vision Builder Automated
Inspection (VBAI).
iii. Validate vision inspection system by interfacing a pick-and-place robot, a
webcam, a personal computer, a programmable logic controller (PLC), and a
conveyor.
iv. Classify product in term of different colours and shapes.
The fabrication of conveyor, robot arm and development of software will not be
covered in this project.
1.5 Expected Result
This project is expected to improve the inspection process in terms of colour sorting
and shape orientation for packaging purpose. The product will be conveyed and
inspected for its colour and differentiated in term of shape using image processing
technique. A robot then removes the product and places it in a correct slot at right
orientation.
1.6 Content
This report is presented in several chapters that describe the development of
automatic quality control system using vision system. Chapter 2 discusses the
Literature Review that describes about the related information from journals and
books. Chapter 3 describes the Methodology of the proposed work and shows a flow
chart of process that explains the operational process supported with some statement.
4
Chapter 4 explains the Experimental Setup and Procedures of the installation and
integration of vision-based inspection system and sorting system. Chapter 5
illustrates on Results and Discussion about the results obtained from experiment. The
important findings will be presented in a comprehensive manner. A comparison
between the desired result and the actual result are performed in order to identify any
limitations to the system that is being proposed. In addition, the projected objectives
will be reviewed to ensure that goals met. Furthermore, suitable recommendation for
further improvement to the system will be included. Chapter 6 will conclude the
entire project with summarize the main findings and review the scopes covered.
CHAPTER 2
LITERATURE REVIEW
2.1 Introduction
This chapter discusses the literature review related to the proposed vision-based
inspection system. Review of previously developed systems in literature review are
discussed and illustrated. This chapter discusses mainly the vision system, sorting
system, conveyor and sensors that are relevant to the development of the proposed
system.
2.2 Vision System
Vision system can be described as image acquisition, followed by image processing
and interpretation. The processed image data is transferred by computer for some
other useful application. Vision system is classified into two dimensional (2-D) and
three dimensional (3-D). 2-D is most suitable for industrial applications that include
dimension measurement, gauging, check for the presence of components and etc.
(Groover, 2000). Vision systems are widely used in inspection section in industry.
The implementation of vision system increase the efficiency and accuracy of
inspection process because the high repeatability and accuracy of machine has
overcome the disadvantages of human-based operation system. Thus, sales of 2-D
vision system outnumber those of 3-D systems by more than ten to one. Figure 2.1
shows the basic functions of machine vision system.
5
Figure 2.1: Basic functions of machine vision system (Groover, 2000).
Image data acquisition is accomplished using a video camera where the image data is
transferred to digitizing system to store the data for further analysis. The camera is
focused on the region of interest (ROI), and an image data is obtained by dividing the
viewing area into a matrix pixel, arranged in pixel arrays. The intensity value of a
pixel is converted by analog-to-digital converter (ADC) to equivalent digital value
which is proportional to the light intensity. An image is represented in continuous
with respect to the x- and y- coordinates and in amplitude as well. In order to convert
it to digital form, an image is needed to digitizing the coordinate value (sampling)
and digitizing the amplitude value (quantization). In practice, the arrangement of
sensor used will determines the method of sampling to generate the image (Gonzalez
and Woods, 2002). The result of sampling and quantization of an image function, f(x,
y) now become discrete quantities resulting in a digital image with M rows and N
columns based on the matrix shown in equation 2.1. The coordinate convention of
pixels that represent digital images is shown in figure 2.2.
6