computer vision for high-speed, high-volume manufacturing

7/29/2019 Computer Vision for High-Speed, High-Volume Manufacturing

http://slidepdf.com/reader/full/computer-vision-for-high-speed-high-volume-manufacturing 1/6

Computer Vision For High-speed, High-Volume M anufacturing

Mark S.BurrellComputing Department

University of Northumbria at NewcastleNewcastle upon Tyne, NE1 8ST, England

.4h.s~mc/ This paper i s a discussion of the use of computer

v is ion techniques for qual i ty contro l in high-volume, high speed

manufac tur ing - with part ic ular reference to elect r ical fuses.

An overview of typical real-t ime manufacturing vision systems

i s given, together with a description of an example problem -that o f inspecting electric al fuses wit hin a high-speed

manufacturing environment . A vis ion inspect ion method i s

introduc ed that uses 3-D imaging.

ITypical Inspection Problems

Much work has been Joiie oil th e use 01' computer visionsystems to solve specitic industrial problems These specificproblems are largely tackled using off-the-shelf systems thatprovide data driven descriptions to select parameterisedprocedures The many reasons tor the use of computervision inspcction systems have been highly documented; a

succinct explanationcan be found in [ I ] .

( ' o i i i pu ter v i s i o n has been used to detect inany componentproblciiis in real-time systems Amongst the problems thathave been solved by re;il-lime vision inspection is that ot '

tinding surticc dct'ects in iiiatei-ials121, analvsing objectshapcs fo r pass/rc,icct 1 I. inspecting the shape of glassware

141and the tinding of manutacttiringfaults in bricks 15 I

assumption for components such as inspecting PCBs, butcertainly not for many other components that vary greatly inall three dimensions. It can be shown that the useof addeddepth information, in conjunction with a thoroughknowledgeof the component being inspected, can improvereliability of inspection without a vast decrease in inspectionspeeds

There is a large family of translucent symmetric componentsfor which the traditional two dimensional approach does not

give the required level of inspection reliability required forhigh volume manufacturing. This family consists of

coiiiponents such as pharmaceutical glassware,thermometers, bottles, light bulbs and electrical glass fuses.Within this familv of objects. faults generally lie withindefinable regions of the object. To determine whether afault feature lies within a specitic region three dimensionalinformation of the feature is necessary. The exampleproblem discussed further is the visual inspection of glassfuses Within this particular vision problem reliability is thekey ob.iective. the aim being to reduce fault detection errorsto .;.-Ier niillion fuses inspected The secondary aim is tonieet speeds of' inspection in the order of three componentsper second

'The problem of visual inspection for this family of objectscan be broken down into two very distinct areas and becauseof this w e treat the problem solution in two separate ways.Tlie t w o areas are those of detection and fault identification.Dctcction can be reduced to a multiple 2-D image problemwhereas fault identification is a 3- Dmage problem. This isbecause the identificationof object faults can be simplifiedby determining the region in which theobject fault lies.

'The approach discussed here treats the inspection as a threediiiiensional problem 'l'his i s not to say that it is necessaryto build up a complete -3-D model of the component underinspection but rather to gain an understanding of depth forcci-tain tkatures of the component to aid in deciding the

rciisonas to why a component i s classified as a reject. I f theproblem i s treated as three dimensional. then it i s possibleto ii1;d-x iiseof 2-11 inforniationfor thedetection of certaincriteria, bu t if thc problem is treated from a solely 2-D

approach i t is much niore ditlicult to make use of 3-D

349BEST COPY AVAILABLE

-___ _ _ -



information. Although it is possible to gain someinformation about a feature within a 2-D image, it is stilllogicallya 3-D inspection problem.

inclusion of the image within this paper. The original imageis of nearphotographic quality.

Three dimensiona imaging has been used within the realmof medical imaging for a number of years. A typicalexample is the field of computerised tomography which islargely based on the use of 3-D imaging. In computeri sedtomography scanned 'slices', using a beam of radiation,through an object (for instance, a human body) can be

digitised and then a 3-D model can be constructed usingconsecutive2-D images, with the scanningbeamwidth as

the third dimension. Theseimages,madeup ofmany voxelsorvolumee l m s , CUI thenbeused to give a morepreciserepresentation of the3-Dobject. Detaledaccounts of thiscomplex subject can befound in [6]and [7].

Fig. 1

Alternatively, 3-D imaging has been used in an industrialsetting to match mechani cal partswith geometric models, inorder to dedde if the part meets precise geometric

tolerances. The work in [SI and 191 describes the use ofdepth data acquired by the use o f laser range findingtechniques to sotvethisproblemo f tolerance inspection.

11. Fuse Inspection Requirements

A vision inspection system to reliably solve this exampleprobl emis preaently under development.

Theexample system is to find faults in glass f uses and to

pasor fail each fuseon the results of the inspection. Thesefisesare 20 millimetres by S mill imetres in sue, consisting

of 2 metal end caps joined to a glass tube containing anelement. Theelement is solderedto the inside bases of each

cap. Many millions of these f uses are produced weekiy.There are a number of reasons for which a particular fuse

may be classified as a reject and, understandably, aproduction linevision system i s required not only to reject afailed fuse but also to report the reason for that rejection.These statisticscan then be used in a timely fashion to detectpr obl ems within the production process as and when theyoccur. The reliability of the system is the primary systemaim, being defined as a failure rate of not more than six

sigma. which is equivalent to 3.4 failures per million fuse

inspections.

A fuse image is shown in Fig. 1 . The &se i s hed in a testrig (the cones holding thehse in place). it isback li t and isan example of a &se that containsylass cracks. The image

is512pixels by S12 pixels in size. The original greyscaleof256 intensities has been reduced to 16 intensities for

initially the system is being designed to discover a subset offuse fault features, but in timewill be improved to detect all

faults. The initial faults required to be detected are asfollows :-

0

0

e

Body damage (cracks in the glass etc.)Any internal debris (l oose solder beads, internal solderand internal debris)Element distortionand multipldincorrect elements

The main feature that we wish to expand the system todetect at a later date is element geometry, but all hsefeatures to bedetected by the longer term complete system

are listed in the table below. together with theircharacteristics and particular identification requirements.

Featwe

Requiromntsand

Charadarbtics

wanted feabre

3'0

. _ _ .. . .



111.Techniques For Feature Categorisation

Within the example components it was very quickly realisedthat the detection of faults is relatively simple, but the key toidentifjllng the category of the fault was primarily the depthposition of the fault feature.

As we have in this particular example an object withrotational symmetry (basically a cylindrical object), we candefine position with reference to the Z axis and distancefrom the Z axis. This provides more useful positionalinformation than Cartesian co-ordinates would, a cylindricalcoordinate system matches the object geometry so well andhas a significant effect in reducing the computationalrequirements. However, in order to know the distance fromthe Z-axis we will have to use X and Y axes (within each 2-

D image) in order to calculate the distance from the Z a x i s .

Measurement of distance from the Z axis can be shown inthe following diagram with z d representing a featuredistance from the Z a x i s . This is shown in Fig. 2 below.

The feature in this diagram represents intemal solder.

I axis

X ax is

Fig2.

In can be seen in Fig. 2 that the fuse feature could either liewithin the region that is the glass tube (i.e. it isa crack) orwithin the hollow central area of the tube (i .e. that it is someform of internal debris). It is only the use of depthinformation that can clarifi, to which region the featurebelongs.

In accordance with cylindrical polar co-ordinates an angle8can be used in conjunction with an arbitrary X axis withinthe symmetric object (this need not necessarily match theaxes from the 2-D images) If we wished to takemeasurements between a number of features we would have

to record the angle of the feature from the specified arbitraryorigin or relative angles between features This could beused to take measurements of feature length, Nidth etc

35

The vision system to find the above faults uses a doubleinspection format, to match the problems of detection andidentification. These two different methods are closelycoupled in the inspection system. A design structurediagram for the inspection system is shown in Fig. 3. Note

for checking a good fuse and the route for checking asuspect fuse.

the difference in computational intensity between the route I

h p 3

The fi rst inspection, completed for every fuse, is a fastdetector to find anomalies within the fuse. If the fuse isfound to have a possible defect then a second slower andmore detailed inspection is conducted. It is possible for afuse to fail the first inspection only to be re-classified as apass after a second closer inspection. This is in order for allreject fuses to fail the first inspection, even if a small numberof good fuses also fails; this prevents catastrophic failure ofthesystem.

The examination of a fuse in order to categorise it as a passor fail is a fairly simple problem and can be done quiterapidly by a high level feature detector. However, once afuse has been categorised as a reject it isthen necessary toanalyse the fuse to deduct for what reason the f use hasfailed. Understandably, these statistics can be used in a



timely fashion to detect production line problems as andwhen they occur.

Both types of inspection (detection and identification) makeuse of the sametwo2-D images, taken at an angle to eachother. It is only in the second identification inspection that

high level i nformati onfrom these images is melded to form3-D information of specific features.

It mustbestated that thetwo inspection format listed belowwll not find external solder on the example components, but(although presently optimised to the above mentioned subsetof faults) it has in theory the ability to find all internal faults.

A. F irst Inspection -Detectioii

The first inspection on each fuse takes place on both 2-Dimages. Basically the images are x-ed at certain pointsto build up a higher level abstract image of the basic lower

level captured image. This higher level abstract image is athresholded imageand ischecked for feature abnormalities.This abstract image is merely used to vastly reduce theamount of data that is required to be examined. An exampleabstract imageisshownin Fig. 4 . This example is not a trueabstract image but is simulated in order to give the reader anidea of what an abstract image consists of An abstractimage is presently stored as an array one hundred pixels bytwenty five pixds.

The abstract image is constructed by the scanning of everyfifth vertical line of pixels and every fifth horizontal line ofthe fuse position within the original image. and so everypoint within the abstract image represents a scan of fivehorizontal pixels and five vertical pixels in the originalimage. If any of these pixels is above a specific thresholdvalue then the corresponding abstract point is also set,otherwise it is cleared. This grid will detect all possibleanomalies greater than five vertical or five horizontal pixelsin size, this is enough to find objects as small as loose solderbeads. This form of abstract image contains only 2500pixels compared to over a quarter of a million in the originalimage.

The method of fast checking can be listed below :-

Regions are counted. If there is more than a singleregion then the fuse requires closer inspection,because we are only expecting a single region which is

the two caps joined by a single element strand.Region finding is completed by checking each setabstract pixel to determine whether it belongs to an

already known region, and once this iscompleted it isa simple matter to count the number of regions.

If there is only a single region then it is scanned toensure that it contains no clear regions within it.

otherwise this could indicate that two elements

entwine each other. This is done by searching eachknown region for any clear abstract pixels within itsboundary.

No peaks (apart from a single element connectingboth caps) stretch from either cap as this couldindicate glass cracks. This is done by checking theinner boundaries of each cap area for peaks of setabstract pixels that reach into the inner area of thefuse (or indeed joins the opposite cap). We areexpecting at least one peak to reach the opposite cap(caused by the element), but more than one peak is anindication of a possible crack feature.

1 1 1 l l l l l l I 1 l l l 1 1 l 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

11111111111111111111 I I

I l l

1 1

I II I

1

I II II I

IIII

I I

I I

1 1I I

1 I 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Note that this particular fast inspection method (specificallydesignedtomeet finda suhset offaults) is only designed t o

find cracks, internal debris and multiple elements. If any

anomalies are discovered then the fuse i s sub.jected to a

second analysis inspection. I t is expected that all failedcomponents wil l fail the first inspection. whereas over OS ? h

of good components wi l l pass the tirst inspection Those

good components that fail the first inspection will be i e -

categorised by the second analysis inspection



The use of an artificial neural network has not been ruledout to replace this algorithmic method of checking abstractimages. It is possible to train artificial neural networks tofail safely, rather than give catastrophic failures, and as suchcouldbe reliably used in the context of our first inspection.Papers discussing the fail-safe attributes of artificial neural

netscan befoundin[lO]and[II].

The stereopic view can be obtained by the high levelmelding of information gained from two separate 2-Dimages taken at an angletoeach other. This angle in humanstereopsis s about four degrees, but for this problem may beas much as ninety degrees. This angle is presently thesubject of testing to discover the optimum. Factors toconsider are the accuracy of depth, that no features arehidden by the element, and the ease of matching featurepairs. At high angles, such as ninety degrees we have anaccuracy in depth which matches that ofthe X and Y axes ofeach individual 2-D image. bur the problem of matchingfeature pairs is made more dif icuft. A lternatively, at a lowangle matching pairs isa simpler operation but a low anglealso surers from a lossof depth accuracy and thepossibilityof a failure feature (for instance, a crack} being hidden bytheelement in both images.

.

The3-D analysis consistsof five basic steps. These are :-

0 Obtain feature inl'orination (width, length, position) foreach feature within both 2-D images (this is done usinythe original 2- 0 images, using information from theabstract images to indicate where to look in the originalimages)Intelligently map rhe tkature pairs. This is the wellknown correspondence problem, the accurate mappingof a feature in one image to the correct l'eature in the

second image, soas to complete the image pair. This isnot as ditlicult for us as it is in other problems due tothe small number of'leatuies tomatch and the fact thatw e are marching features usiny specific featurecharactcristics, namely length and widrh. and we can

also make intelligent assumptions as to where thematching feature in the second image will be situated.Make assumptions to complete feature mapping (forinstance, if a feature is hidden in one imageby thekseelement)Make a deduction of the fault, using the added depth

infonation

0

0

0 Update the fault log

Two symbolic images of a fiseare shown in Figs. 5 and6.I n Fig. 7 we can see a fuse from above, and the relationshipof the images in Fig. 5 and Fig. 6 to each other, bei ng

rotated around the Z axis from each other by an angle 8.The obliquecrosses in both Figs. 5 and6 indicate a feature(for instance a piece of loose solder, a representation ofwhich can be seen in Fig. 7) that hasbeen found.

Element1 \liewffomF~..

F i p. 5

The vertical cross points indicate the top and bottom of thefeature while the ends of the horizontal bar indicate the

width at half the feature height. Once we have calculatedthe height, width and position of the feature in both images,and the angle between the images is known then we can

make assumptions about the depth position of the fatureusing stereo triangulation (calculated on the difference inpositions between the two images). If the angle 8 is less

than ninety dqrees then the depth accuracy wiil be lessthanthat of theX and Y axes, but a suitable angle will still giveus acceptable accuracy - enough to determine whether thefeatureiswithin thevolume ofthe glass tube or not.

For instance, ifthe depth information shows that the featurelieswell within the inner wall of the glass tube (within thecentral body of the fuse) then we can deduct that the feature

is some form of intemal debris (it is certainly not a crack),and then we can make further deductions using the failurefeatures' size and intensity.



This use of depth information is possible because of theknowledge of the object being inspected. There are anumber of reasons as to why this method is possible to useJ ! high-speed, not only for our particular problem discussedahwe but also for the family of translucent objectsiumtioned earlier. These can be listed thus :-

0 We know that we only need depth idormation atcertain areas within the original 2-D images and wecan concentrate on theseareas.

0 As to the mapping of features, it is unlikely for us tohave to match a large number of features per imagepair, for instancethemajorityof failedk s e s contain atmost three failure features. Also, we are matchingfault features using characteristics such as length,width and position - we are not trying to match anedge at all points along it's length (a computationallyintensive operation).

0 The image pair are only rotated around one axis, thereis no translational difference to deal with. The bodyof the symmetrical object remains in a stationaryposition, whereas failure features (and, in ourparticular problem, to some extent the element) willappear to be shifted between pairs of 2-D images.The components can be placed precisely forinspection at the same point mechanically.

I V . Reliability Considerations

The reliability of the system is strongly aided by the use of atwo pass inspection method. Potentially, the initial reliabil ity

isgiven by the ability of the first inspection (detection) tofail safely and not to fail catastrophically. Due to the firstinspection being optimised in this manner it is possible atthis point to also fail a proportion of good components butthis is rectified by the second inspection (identification),which has the ability to re-categorise a failed component asa good component.

V . Conclusions

A family of translucent objects, easily described by acylindrical coordinate framework, can be inspected for

defects using two different. and yet closely coupled.methods. The first method can detect failure features using

multiple2-D images. whereas the second method makes useof 3-D image information to enable the determination of theregion in which the failure feature occurs.

The typically low reject rates of components for high-volume manufacturing allow the use of these two different

inspection methods. The initial detection is fast while thesecond identification method is substantially slower. Thecomplete inspection system is potentially reliable to a highdegree.

Conventional vision systems are unlikely to be suitable forthis task.

Bibliography

J . Mackrory, "Industrial vision systems in today's manufacturing

envi" ent", InrageProcessing 89Proceedings, I SBN 0-86353-189-X

M.A. Coulthard, "Image processing for automatic surface defect

detection", 3rd. In!. ConJ on Imqge Processing ami it s

application.̂ IFE (,ConE.Pub. No. 07)

R.D. l'illett, C.M. (hyango, P.F. Dak i s , J . A . Marchant, " Imageanalysis for biologcal objects", 3rd. Int. COP$ on ImageProces.singand i n Applications. EE (Conf Pub. N0.307)

M. Nixon, "Shape inspection of crystal glassware",3rd. Int. Con/:on huge Processing and its Applicutions. Et<(Conf. Pub.No.3071

S.K . Ala. D. Chamberlain. T J . FCllis, "Real time insptxtion ofmasonr y units" , 4th. Int. Con/: on Image Processing md it s

..lpplicutions,EE (Conf.Pub.No.354)

Gahor T I lennan, Image reconstmction ,from projects : he

J irnilamental.v computeri.red tomography,Academic Press I nc.

M . Freedman. Clinical maging. Churchill L ivingstme.

A. Marshall et al. "Automatic nspection of mechanical parts using

geometric models and laser range finder data". huge und vision

tanipririny. Vd 9.No 6

A Marshall. "Autoniatiwll~nspecti ng gross features id machined

o l j ~~tsuinp three-dimmsicoal depth data". SPIF' I id.13x6.I uchrirc~ icion .Srtrt.nrs1ntt.grariotr 111 I t r l r i s t n~Y90)

I) M chlichncl. "HAKI' IN minimising I h!es risk and

i i ic~rp~ra~nigo r x using supen scd learning inet\\ ork.d". IFF.'

/'rocct~~t'ii~,pr.d 139. No 6.1)ccemht.r 1902

I) McMicli;tcl. "II/\Kl.IN applid ti, \ i i~u i l in>pectlim (It'

a\:ia\nimetric siipinecriiig part.;". PuhliaheJ under Hritish cron 11

oq>>ripIii \ the I>R:\.10%

354

computer vision for high-speed, high-volume manufacturing

Documents