new machine vision applications in germany - cvl - ikeuchi laboratory

MVA '92 tAPA Workshop on Machine Vision Applications Dec. 7-9.1992. Tokyo

NEW MACHINE VISION APPLICATIONS IN GERMANY

Claus-E. Liedtke Institut fiir Theoretische Nachrichtentechnik und Infomationsvwarbeitung

Universitst Hannover

Appelstrasse 9.4, D-300a Hannover I , F. R. Gennany

ABSTRACT

The availability of robust sensing devices. powerful image pmessing hardware and advanced system de- sign has increased rhe use of computer vision methods considerably. Image analysis methods based on s ~ t i c 2D binary-, graylevel-, and color images have become standard sensors for various applications in the biom- dical field. the printing industry, industrial, automation, navigation, remote sensing. office automation, etc.

T h i s paper reports about new machine vision applications in Gennany resulting from the recent progress which has k e n made in this field. Since the complete spectrum of new machine vision applications cannot be covered we will limit ourselves ro systems which me based on TV-like cameras and nre primarily used for image analysis. The paper is structured by metl~odologies of image pmessing, like 3D Scene Analysis for Quality Conaol, Image Sequence and Mo- tion Analysis, 3D Modelling, Knowledge Representa- tion. and Human Interface techniques. The progress is illusmted by selected new applications which have recently been observed in this field in connection with products. prototypes, or concepts developed at research institutes and the indusq in Germany.

1. INTRODUCTION

nancing models which do research and development in machine vision for various applications. They are partially supported by the federal government or by the states. Finally, research is conducted in the universities.

The universities and the public research institutes concentrate more on fundamental research. whereas the industry concentrates more on applied research. There are several initiatives to support know-how transfer and cooperations between the industry and the public research institutes.

The research is financed on one hand from private resources, mainly from the industry and on the other hand from public resources. A major role in public fund- ing is played by European research programs like ES- PRIT, RACE, PROMETHEUS, and others.

Since it is not possible to assess and review all activities which are related to machine vision, we have con- cennated on new machine vision applications which are prirnariIy based on an analysis of TV-like camem images. The analysis of other types of sensors, like NMR- and X-ray systems, ultrasound, IR, microwave. etc. will not be considered here. Further we have ex- cluded those areas of image analysis which are not p- nerally k e n accepted to belong to the fieM of machine vision like for instance telecommunications. archiving, document processing and photogmmmeay.

The paper is structured by methodologies of image processing. The selection of the topics and the points of emphasis have been guided by our own research activi-

Research and development in machine vision is carried ti"s'

oirt in Germany in numerous institutions in the public and private sector. Especially the largecompaniesof the 2, 3D SCENE ANALYSTS FOR Q U A L W electrical and optical industry who are suppliers of vi- sian systems and the large users of vision components

CONTROL

like the car industry and the machine indusv support to In industrial automation an increased request for com- a Feat extent research work in machine vision. ~ u t e r vision svstems can be observed. Most vision svs-

I n the public sector there exis$ a number of research terns are used for quality monitoring which mean1 sight instilutions with different designations and different fi- inspection and completeness check, For the fully auto-

mated production the early recognition and removal of determined. The assessment of different ermr types re- defective parts is indispensiblt on all production levels. quires a complex arrangement of illumination sources For the detection of faulty materials. defective assern- and camemq ap is shown in Ag.1. mical insptcbon blics. manufacturing dtftcrs. and shipping damage, the timcs of such systems are 20 s for a single board of 360 inspection by vision systems is the only solution to x 400 mmcontaining approximately 4Ocomponents. achieve an economic automation. It guarantees an improved and documented qudity, even i f the number of different p d u c t s and the work-pieces involved are in- c~as ing.

A second important companent of fully a u t o m t d praduction lines ia the use of mechanical manipulators. They are mainly uscd for machine loading. welding, painting, and sealing. As oftoday mechanical manipulators have not k e n utilized cxltnsively for assembly. With he increasing importance of medium and smaIl batch pducuon, manipulators have to bccom mare and most flexible to be adaptable to chan~ing trtsks. While indusmial robotq with the ques ted flexibility are on the market, some of the complementary equipment, Iikt thepartfetding cquipmentwhich hasto be designed in a task and pan specific way, still requires a lot of engineering cffon and often costs more than the robot. Here machine vision plays an important role in pro- viding the requirecl flexibility.

Most of the vision tasks in indusbial automation use 2D vision systems. The inspection of complex swFaces. the action of mechanical manipulators in 3D space, requires a 3D scene analysis. Because of the requirements of industrial automation concerning environmental conditions. reliability, prmessing s p e d , cost effective- ness, ctc. we observe that inmasingly 3D sensors with conwl ld illumination and passive sensors using model bascd approaches arc usad for 3D scene analysis.

The principle of shape from shading can be uscd in connection with an appropriate illumination source to identify slopes of different rtscent by the intensity of reflectance. The method permits a fast and reliable analysis of local 3D surface properties.

One m a of application is the soldered joint and the assembly inspectian of ptintFd boards, Soldering errors like open soldered joints, solder bridges, balled sold& joints and solder meniscus faults can be recognized by this method. In rrddition mounting emrs Iike missing components, wrongly placed components and twisted components can bt detected. Using sequences of di- rected illumination from d i i m n t directions, the exis- tence of componenfi like chips, Melfs, SOICs, Hat- packs, PLCCS. etc. and their orientation can be

Fig. 1 : Arrangement of camens and illumination In a soldered joint inspaction system (Viscom)

Stnrchrrcd illumination constitures another common approach to assess 3D geometry by conml!olled illumination. The avriIa'bitity of robust projection devices. ti- able optical components and the availability of spccial purpose p m s s o r s for h e data interpretation pennits real time optrations in an industrial environmnt and enables new applications of computer vision in industrial aurornation.

The line-scan+nethod can $e used for edge fotlow- ing undercxmmt conditions like high tempcratu~s, for instance to direct a precision welding robot with high accuracy.

The projection of gratings and the analysis b a d on line ftinge techniques or moirt'-rtchniques enables thc recording of 3D crwrdinates with very high resolurion. The masuring range for TV-camera bascd systems goes from a few microns up to a few metres at a lateral resolution of up to 1000 x 1000 pixels. The analysis by mans of phase shift tttehniquts or fringe analysis allows the acquisition of 313 measuring data in an online mode. Application fields are form and form fidelity tests. surface inspection, 3D contouring, deformation, and vibration analysis. The advantage compared to clas- sic intcrferomeaic sytems are the simple and rugged type of construction and easy handling. Examples for application are postopcrativc control rneasuscrnents in surgery or dent analysis on car surfaces [ 1 I.

Another methid for the fast acquisition of depth maps i s the coded light approach. A stquence of stripe p a l m s is projectad, where the bit panem over time for

neighbowing stripes cornsponds to Gray code encod- ing. Since each stripe is uniquely identified by i t s code the depth map can bc calculated very fast and efficiently by using triangulation techniques. A map of 512 x 512 pixels at 8 Bit depth resolution can Ix recorded in two seconds. One area of application is the use as 3D sensing device in robot cells [2].

2.2 UNCONTROLLED ILLUMINATION

The recognition of 3D objects and the deterrninatlon of their position and orientation in 3D space have a key function in industrial automation processes. The solution of this problem by using for minimization of the processing effort just one monocular image is only possible. when the objects under investigation are: known beforehand and when a relevant description of the ob ject. i.e. a model, is made available to the image analysis system. The usual approach is that characteristic primitives of the object are extracted from the image and arc compared in their properties and spatial relations to the equivalent primitives of the model description. Onc problem is to select characteristic primitives which can bc recognized reliably and efficiently in an image, independently of the natural fluctuations of the illumination source, small changes of the camera position. and changes in the reflective properties of theobject surface. Usualty e d ~ c type primitives are used, like straight edges, circular edges or the cdgs of vertices, The second problem i s to find a fast and efficient method for the comparison between the structural description of the model and the structural description exmcted from the image which is robust enough to tolerate misclassifica- tions of primitives and missing primitives due to partial occlusion. Several systems of this type have k e n re- ported [3],[41. Heuser 141 descriks a system which works in 3D, where the simctutal comparison ktween image data and model uses the relaxation principle. Fig.2 shows an example of a typical scene. where one of the objects found is overlaid by the 3D model which has been used for the scene analysis. The computation time for recognizing the object and cf culating the 6 spatial translation and rotation parameters forthis example was less then 20 s on a 68030 processor.

'Ihe application of this principle of mode1 based image analysis is still in an txptrimental stage, It is usually used with resbictions concerning the d e p of fmdom in spatial position and orientation in order to reduce the pmcessing time further. Applications are found in the recognition of partially ordered workpieces for the loading and unloading of boxes or palettes, sup- pon for the flexible control in automated assembly pro-

Fig. 2: 3D object recognition using a model based approach

cesses, check for comcmess and completeness of parts. size measurements of planks in wood processing, etc.

3. IMAGE SEQUENCE AND MOTION ANALYSTS

For the assessment of d y namic processes it is necessary to process image sequences. The computational eiToon to process for instance 25 h a p s per second of a standard video camera in real- time i s significant. Therefore the application of computer vision methods to image sequence analysis is very often connected with the use of specid purpose hardware or with the use of parallel computer architectures. Applications frequently deal with the observation and description of time varying scenesor the motion compensated assessment of particular objects.

An example is the on-line evaluation of experiments in pharmaceutical laboratories. The devetopment of m- dication forthe treatment of thrombosis q u i r e s in vivo observations of the formation of thrombi in animal tx - periments. The analysis of sequences is perform4 in two steps. In a first initializing step the scene has to be segmented into tissue, blood vessel and thrombus, where the thrombus is found growing from the inside of the vessel wall into the blood sWeam. In a second step the thrombus is tracked over the time while the motion of the vessel wall is compensated. Fig.3 shows one frame of the sequence, where the relevant parts of the scene. like the vessel and the thrombus are marked.

Fig. 3: Ohsecerva~ion of thrombus formation in bio-

medical experiments.

The ohqervcd objects and the observing camera respective are not still but are moving in a real world scene. Fwqucntty a motion analysis is used in connection with the sequence analysis to compensate for the motion between the object or scent under investigation and the observation platform before further analyses are applied. There exists a wealth of literature on the problem of 2D- and 3D-motion estimation from image sc- guences. One example is a survey from Aggmal [5 ] . There are various applications for the use of motion analysis in connection with image sequence analysis. Frequently there is rhe q u e s t lo read text on moving ohjects. where the text becomes deteriorated by adverse i!lumination effcca, motion blur. noise, perspective dis- tortions. etc.. In these cases the signal to noise ratio of the text imge to be interpreted can be considerably improved by tracking the object and averaging the reIe- vant part containing the text. In Fig.4 we show the im-

Fig. 4: a) individual image of a eacked object exhibiting a low dn ratio and b) averagd image over 6 samples exhibiting a higher l;ln ratio.

provenlent of the quality of imprinted text on a metal

surface, where 6 images have k n taken over time. Applications are the reading of license plates on cars for the determination of admission pcrrnits to parking lots, msbicted areas, etc., or the identification of cam on highways for automated traffic measurements and highway conml, and the automated reading of traffic signs from the driving car 161.

In industrid automation motion detection and motion cornpensarion. i.e. image regismtion, is widely used in all instances, where aquality assessment is mndt on the basis of a point to point comparison between the tested object and a promtype. like multicolor+rint check on cigarette packet$, s m n prints, etc.

A number of applications for imp sequence analysis in hmt varying 3D sccnts is investigated in connee- tion with the European research program PRO- METHEUS for the improvement OF the road mffic of the future. One partial goal of the overall project is to obtain increased driving safety and cornfan via envimn- ment sensing, interpretation, and intervention, An autonomous test vehicle VITA (VIsion Technology Ap- plication) has been quipped with sensors. elecaonically controlled actors for braking, accclera- tion and steering, processing systems and acopilotcom- puter in order todemonsmre the feasibility and to assess the potentials of computer vision tc~hnology[71. The visual sensors employ two B/W CCD video cameras with different focal lea~th, one for the observation in the near fieEd with small focal length and one for the o b r - vaton in the mid- and far field with large focal length. The envisioned system functions arc (a) lane keeping, @) obstacle detection, (c) stop& go driving and (d) lane change support, and (e) inttmction conmol.

The prerequisite for any autonomous driving on a,

street is the lane keeping capability. 7he 4D problem of observing a time variant 3B scene i s treated very efficiently as a c o n d problem, i.e. no efforts have been made nocxwct txpticitly the underlying 3D structure of !he scent. The expectations abaut the dynarnical btha- viour ofthe vehicle and relevant objects in the surmund- ing is mdetled by a state space system. On the baqis of the relevant featurns of the surrounding like lane markings or obstacle edges the position of the vehicle position is obtained by recursive estimation ttchniqucs.This information is used to guide the vehicle in l a m ! d im- tion. Figure 5 shows a frame of the sequence and indi- cates that tight areasob interest we used to track the lane markings. The superiority of this approach. at Itast for well1 s r m c t u d scenes like mads. has bten dtmon- strated repeatedly during the last years by drjving a 5 ton

Fig. 5 : Window positions for street edge tracking during automated navigation on public streets

testvehicleat aspeed close to 100kmh an afreeway and up to hll kmh on an unmarked tw+lane couniq road.

A survey of the previous work i s given i n [R]. The rnclhod has been cxtended to k used for obstacle detection 191. If a candidate for an obstacle satifies [he shape and location criteria the object is mckd and the relative position and speed are determined. The information is used to stop the car or to perform a lane change. A stop & go function is used to rack the vehicle ahead and to maintain a safe distance. The vehicle ahead is automatically recngnirpd by rome syrnrneay measure 1101. The distance to the vehicle and its speed are measured. Tn the intersection c o n ~ o l function the type and the distance of intersections are recognized. The system is based on the fitting of edge information against varirieus versions of internal intersection models.

4. 3D MODELLING

There i s an increasing request for obtaining computer graphic models of natural scenes in order to create: realistic visual sensations by producing computer anima- tions and three dimensional montage pictures, kpplica- tions are

- in the movie and TV industry. for film production, syntheaic studio environments,

- in future communication systems and virtual reality applications 1 1 1 1,

- in advertising, instruction and education, - for the visunt ization part of various baininp sirnula-

tors, like flight- and driving simulators, - for pl~nningpurposcs in landscapennd city planning,

architecture, erc.

Though considerable progress has been made in the past years in computer graphics and 3D visuatimfion techniques it is still not possible ta prduce with mason-

ableeffon natunl looking landscapes, persons, animals. etc.. A selutioo to the problem is expected from the

integration OF computer graphics with the analysis of natural scenes by machine vision techniques.

The machine vision approach ta TD modellinp i s to choose an appropriate parametric description of the 4D scene and toestimate the parameters from a .wt of spatial and temporal image sequences. Various problems are connected with this task, First, the scene has to be under- stood, i.e, to be explained by a set of independently moving 3Dobjec'ts. Fbreach object an appropriatemd- el description bas to be chosen. On one hand appropri- ateness is detemined by the type of object. since a hu- mnn head requires a different pmmemc description than acar. a me, a sea surface, ora cunain waving in the wind. On the other hand the use of the model kcoms important for the appropriate choice. The adequaze model for a house is different, when it i s u s 4 for view- ing from a far distance, for producing the sights from a walk around the house or for a simulated walk into the house. The ueatrnent of many of these problems requires till today human interaction. Automated mdel- ling becomes only feasible. when restrictions about the type of scene and various properties in the scene can be made. Problems in modelling which are still subject to ongoingresearch are image segmentation, object recognition, motion detection and motion compensation. surface interpolation, matment of flexible objects, enter- ingof prior knowledge about rhe objects in the scene and the relations between them. etc.

Some principles which have evolved lately are texture mappingand modelling through analysis by synthesis. Texture mapping [12] exploits the fact that humans can perceive shape only coarsly. Therefore, naturally looking objects can be obtained by using a coarse 3D object shape and modelling the fine structure like the hairs on a human head, the grass on a ground, etc. by projection of an appropriate texture onto the surface.

Analysis by synthesis is a strategy to estimate motion. shape and the surface properties of objects like color and texture simultaneously [ I 33. The basic idea is that a 3D model of the obsemed scene is used to predict the views which are obtained from the real carnew which observes the real 3D scene. For this purpose a model image sequence is calculated by perspective projection of the dynamic 3D model onto a vinual mdel camera target. The measurable differences between the

real image scquence and the m d t l image sequence are then minimized by recursive adjustment of h e model parameters. The set of parameters which minimizes the e m r criteria together with the parametric model consti- tutes then the best 3D model for the description of the scene. Since the p m t e r s ; rcpmscnt an explicit^ description of the object shapes, the motion p m c a r s and the surface propetties, like textures. an explicite modelling of the scene can be obtained. The approach requires an initif model scene as a first estimate of the scenecontent which is then modificdduring the analysis of the image scqucnces. The constntction of the initial mdel is based on some few selected images of the sequence and q u i r e s scenc specific knowledge.

These principles have been applied to he modtljing of terrain from stereoscopic aerial photos for flight sirn- ulators, the assessment of objects under laboratory conditions from multiple views, and to future model based communication systems.

Presently nhe modelling of complex outdoor street sccncs a$ they ate needed for driving simulators, city planning purposes etc.. arc invtstigtcd. The approach is based on the analysis of sequences of stereoscopic image pairs. Starting with convtntionaI stereoscopic correspondence analysis a 3D model scent with 3Q surface gmmttry is generated. Not only the scene geometry but also surface texture is stored within the model. The 3D model permits to detect and correct geomehc errors by comparison of synthesized imagcs with real input images through analysis by synthesis techniques. 3D camera motion can be estimatd directly from the image sequence to track camera motion and to fuse measurements from different viewpoints thoughout the sequence into a common 3D model scene. First results have been published in [IS].

5. KNOWLEDGE REPRESENTATION

An image cannot be anal@ without knowledge about the context in which the image has to be intcrpretd. The direct comparison of computer vision methods with the performance of the human visual system has for a long time given the impression that the human eye is the ktter sensor. In reality it is the 'background knowledge of the human interpreter about the scene and the case of the use of his knowledge which makes the human visual system superior in its abilities to intcrprefe a scene. For this reason considetable effort has recently btcn observed to store in a systematic fashion knowledge content about the scene and a b u t the image processing

mehods in computer vision systems and to make this knowledge available for the image analysis task.

There are variaus paradigms for image rtptescnta- tion. Of particular significance are the paradigms for mplicitc knowledge representation, k a u s e these knowlodge content can directly be undmtod by a human. The use of these paradigms kcomcs advanta- geous, when complex systems are developed or when systems have to be updated, or modified, for insmce in connection with the extension of a computer vision system to new imageanalysistasks. Systems which employ cxplicitt knowledge representation are generally re- f e d to as 'knowledge based systems". In the area of knowledge based vision systems a nurnbtr of activities can today beobservedinGermany /16], 1171. [lR]. [19], F201. [21]. [22l. One bas to differentiate between systems where the knowledge content are rtlatul to the scene and systems where the knowledge content are related to the image prmessing methods. Since the paradigms for explicite knowledge representations like semantic nets and rule bas4 systemscannot be tficicntly impltmentd on the presently available general purpose computers, thc applications of these paradigms tend to be found in areas. where the processing time plays a minor role, namely in system configuration [ I 61,[131,[201,[211. One example is the configuration system CONNY 1211 which will be mated in more de- tail in the following.

The system C O W ha5 been built to investigate the basic concepts of a self-configuring imag analysissys- tem. It permits a complete and fully automated selection of the processing path and the adaptation of the parameters for the low-level part of image analysis, i.e. prep* cessing and segmentation.

For the solution of h e problem knowltdgc of a human e x p m has to bc m n s f c d into the knowIcdgt b a d system ahbout

- known complex and elementary image ananalysis op- CratOTS.

- the properties of the operators and the influence of modifications of the parameter values,

- the characterization, respective the interpretation of image processing resulk.

- the evaluation of the image analysisresults in respect to the user specified task, and

- the sfrategies to configure an image analysis process.

The set of all meaningful sequences of image analysis operators is represented by an o p t o r me, The declarative knowledge content. Like which i m a g processing operators are available, the knowledge content

about their parameter values,etc. are represented asdata objects. The objects are interconnected by relations. The objects own methods which can be used to apply pme- dural knowledge content which are specific for an individual object or a class of objects in an object oriented manner.

The configuration is achieved in three steps. Based on the task specification the configuration system is instantiated. Tn a second step the userprovided reference knowiedge, the quality mquircmnts and boundary conditions are used in a direct configuration smt~ey to reduce the search space for the optimal configuration. The third step is called the adaptation phase. Here an indirect configuration strategy is used by specifying an optimization criterion on the basis of image quality in the test images and using search strategies to optimize the parameter adaptation.

Or1y1rx11 Start configurntion

Cycle 1 Cycle 2

Cycle 3 Cycle 4

Fig. 6: Original and processing results of consecu- tive parameter modification-, processing-, and evaluation cycles in C O W [ZI].

In the adaptation phase for all possibte processing paths the parametem are adapted and finally the path exhibiting the best processing result5 on the test images is selected as the final configuration result. The parameter adaptation requires several adaptation cycles. Each cycle includes the processing of a test imag with the sequence of operators under investigation using test set- tings for the parameter values. an evaluation of the pro-

cessing result and a modification of the parameter vnl- ues,

Fig6 shows an example for the intermediate results produced by the system in the parameter adaptation phase. Each image (except the original iconic input ima- g in the upper left comer) represents one result in the parameter modification-, processing-, and evaluation cycle.

A vision system which uses explicite knowledge representation forknowIedgcontent aboutthe scene under real time conditions is presently being developed [22]. The system isused for the analysis of aerial images and image sequences. The knowledge content about the observed objects and their xlations like lanes, highways, bridgs. etc., are represented explicitely as rules of a pduction ne!. The architecture of the analysis system is bawd on the blackboard principle. Basic image p h i - tives like line*lements. arcs. circles etc.. and their al-

tributes like position. direction, size, quality descrip- tors, etc. are extracted by standard image analysis methods from the aerial image under investigation and arc written onto the blackboard. The retations between the basic primitives and theobjects of higher absmction level (like lanes, bridges, etc.) are expressed as rules of the production net. The different rules are implemented on a number ofprocessors which have p m l [el access to the blackbod. The rules are used to develop a parse me by iterative modification of the blackboard content.

In order to s p e d up the system the blackboard has k n realized as associative memory in PLA technology. The size of the memory is presently 144 MByre and it permits entries of about 1 4 . M Bit length. The bit sequential and word parallel processing permits access times k t w c e n 1MI ps and 10 ms. A realization in ASrC technoIogy is in development and will permit access times of less than 10 ps.

Fig.7 illustrates an application, where a highway bridge has to be recognized From an aerial photo. The basic primitives which are extracted from the imag, are linc segments which are assembled within Ihe patsing process to structures like long linc segments. street patches, parallel s e t patches, lanes, highways, and bridgs. Images like the one in Fig.7 produce initially about 2OM basic primitives and 300 objects of higher abstraction level. Images of lower quality, obscured by noise, may resutt in up to 30.000 basic primitives. The parsing process for the type of application dexribcd here requires about 700 search passea through tht rrptmory. When for the implementation of the rules pro- cessors m used with an individual processing power of

investigation of more efficient learning prucedures under the consideration of more global criteria.

6. HUMAN INTERFACE

Rg. 7: Line extraction from aerial photos in a black- hard oriented vision system for real time applications.

10 MIPS, on overnll analysis time of 50 ms per image can be achieved.

A disadvantage of knowledge based systems is that. they m very complex. The maintenance of the systems requites seven1 experts like the knowledge engineer. who has his backpound in the AI ma nnd isresponsible for the system itse2f and the domain expert. who is an expert in the particular area of application. Knowledge based systems which can lcam the knowledge content autornaticatly and directly from an image analysis art still in a very early state of development.

A more convenient method for the automated Im- ing of knowledge content fot computer vision systems is provided by the concept of neural nets. Because of the inherent parallel structutc very high processing speeds can k obtained. The teaming is mostly achieved by iterative local methods on the basis of the presentation of input-output pairs of information. Applications arc frequently found in situations where a well defined rela- tion ktween input and output patterns exists but its panmetrical formulation is unknown. mere are presently numerous applications in this field to be observed like the sorting of particular typcs of bottles. the recognition of plant diseases, the evaluation of surface quali- ties of industrial materials, the segmentation of traffic s i p s . the correction of lense propenies in camera sys- tans. the interpretation of the license plates. etc.. An area of cutrent research in neural net technology is the

In many cases, the acceptance of new computer vision systems i s not only determined by its novel and suptrjor processing capabilities but rather by a user friendly inclusion in an existing and well defined workingenvimn- mcnt. Two examples will be given. one from the prcpa- ration phase in the professional printing indusay and one ftom the area of clinical cytology.

6.1 IMAGE ASSEMBLY FOR THE PRINTMC INDUSTRY

Specific tasks in the professional tepm industry q u i r e for a more efficient pnphic preparation step prior to the actual printing new high performance image scanning techniques.

The final image is typically composed of objects from xveral individua1 images which are usualIy deliv- ered as color slides. T h e layout for placing these objects in the final image maybe a scribble or an accurate scaled drawing done by the designer. The key probbm is the scanning of the slides with correct zooming and rotation. Theenormous amount of data of these irnagtsdoes presently not permit economical soIutions for computer based mution and zooming. Therefore these tasks are achieved by rotated mounting of each slide on the scanner and by appropriate choice of the sampling rate. Mounting the slides and calibrating the scanner consti- tute time consuminr operator controlled processes,

Higher efficiency can be obtained by using a vision hwd system made possible by a CCD camera with programmable resolution from video size up to 2300 X 3000 pel, The video size scanning which allows up to 25 scans/$ is used for slide positioning by displaying the scanned images on a monitor which also shows the de- signers layout in a transparent mode. Image rotation is achieved by physically rotating the slide and morning by camcra movements. When image objcct and layout are matching, thc final high resolution =an can be done by choosing the required programmable c a m m msolu- tion. Frequently additional color measurements atc necessary for con-ection ofcalor p d i o g . Since a low resolution scan with less image data provides the full color infomation the color measurements can be obtained at high spctd in a low resolution prescan.

In addition to a considerable speedup of these standard processing steps the system pemita the direct assessment of 3D objects wirhout using intermediate photographic processes.

6.2 SCREENING AND ANALYSIS SYSTEM FOR CYTOLOGICAL SPECIMEN

Due to the l a r ~ e numbers of specimen which have to be investigated every day and due to special quantification requirements implied by the new preparation techniques (immunochemical markers, ploidy estimation techniques) computer supponed waluation systems are applied to an increasing extent. especially in big cytology laborstories and hospitals.

The evaluation can either $e supervised and guided by a cytotechnicinn (interactive or half automatic system) or it can be run completely automatically (auro- matic sysrcm). In both cases i t is absolutely necessary, out of legal as we31 as ethical reasons. that the expert (cytologist, patholorist) who is responsible for the final diagnosis has the possibility tn analyze [he evaluation results and compare them with his own experience. re- spectively. with the experience of some other expert.

For that reason a special interface was developed which combines the presentation of the evaluation results with the direct access to an image data base, with the relocation possibility of the original cells via a scanning stage and with the access to different expen opinions in form of mined classifiers 1231.

If encertain borderfine cases occur it is often necessary to consult other experts. This can k done via a telecommunication module which enables theexchange of measurement results, cell images, whole cell scenes or even aElows the consultant to remote control the mi- cmscope and look at the stide parts which may k e s ~ - cially of interest to him.

A cytology evaluation system with the user inredace described above was successfully applied in [he grading of borderline lesions of bone turnon 1241 and in the recognition of early state bladder cancer. T h e interface is now integmted i n a commercial cytology evaluation system which is applied teal1 kindsof rourine ceEl evalu- atian tasks.

In industrial automation there is a strong request for the ~ n a l y s i s of 31) objects. Presently techniques based on

contmlled illumination are p r e f e d . Other methods like stereo are still in an experimental stage. Several prototypes of vision systems to support safety measures and automated guiding of vehicles for future public traffic are developed in the PROMETHEUS projecr.

Dynamic 3D scenes have for a long time been modelled by computer graphic techniques. Since there are still severc problems in obtaining realistic looking "natwral" scenes, there is an incseaswl request for 3D modelling using computer vision techniques.

Mechanisms for knowledge representation in vision systems play an important mle in the development of systems which are robust. reliable, adaptive. easy to handle and easy to teach. On one hand knowledge based systems are under investigation, and on the other hand neural nets show an increasing popularity for various applications.

The acceptance in practical applicarons is frequently not determined by new and superior processing capabilities but rather by a userfriendly inclusion in an existing and well defined working environment.

ACKNOWLEDGEMENTS

The author would like to thank rhc staff members of the institute for their suppor! to this paper. especialty Dr. Gahm and W. Wiilker for there contributions to the chapter HUMAN N E R F A C E , In additon I would like to thank Dr. Heuser (VISCOM). Dr. Zimrner and Mr. Ulrner (DAIMZER-BENZ AG). Dr. Doemens (SIE- MENS AG), Ms. Jansen (BREUCKMANN), and Dr. Lutjen (HM) for the vaIuable discussions and the provi- sion of material for this paper.

REFERENCES

I1 I B-Breukrnann: Optische 3D-Mellsysteme f i r Online-Anwendungen. Technisches Messen- tm, 57, Heft In, pp. TR9-394,1990.

[21 R. Gutsche, E Wahl: The Integration, of a 3D Sensor into a Robor Work Cell. IEEE Tnt. Conf. on Robtics and Automation. Sacramento/ California, USA, 1991.

[3] P. Rurnmel and P. KKllensperger: GSS-A fast, d e l - b a s d vision system for workpiece rez-

ognitien: International Journal of Pattern Recog-

nition and Anificial Intelligence, Vo1.2. No.3 pp. 543-556,1988.

M. Heuser and C.-E. Liedtke: Recognition of the

Spatial Position of Industrial 3 W b j c c u Using Relaxation Techniques; Prmeedinps of the lOrh ICPR. Vol 1, pp.191-193. Atlanric City, USA. 1990.

J.K. Aggatwal, N. Nwdhakurnnr: On the Com- putation of Motion from Sequences of Images - A Review. Proceedings of the IEEE, Vol. 76. No. 8. Aug. 1988.

K. Welz: Beobachtung von Vetkchrszeichen aus einem btwcgttn Fahmug, RoccEdings of the 7. Anchcncr Symposium fiir Signaltheorie. Aa- chen, Sept. 1990.

B. Ulrner: VITA- An Autonomous Road Vehicle for Collision Avoidance in Trafic. hoctcdings of &c conference on Intelligent Vehicles, De- troit. USA. 1992.

E.D. Dickmanns and V. Gratfe: Dynamic Mon- ocular Machine Vision and Applications of Dy- namic Monwular Machine Vision. Int. Journal of Machine Uson & Application, Vol.1, pp. 223-261. Springer-Internat.. N.Y., 1988.

E.D. Dickrnanns, Th. Christinns: Relative 3D- S a t e Estimation For Autonomous Visual Guid- ance of Road Vehicles. Intcl!ignt Autonomous Systcrn 2 (IAS-2), Amstdam, 11-14 Decem- ber 1989.

A. Kuehnle: Syrnmt~y-based rccopnition ofvt- hick rears. Pnttern Recognition Letters 12, pp.244-25%. North Holland. 1991.

H. Hanshima and F, Kishino: Intelligent Image Coding and Communications with Realistic Sensntions - Roctnt Trends. IEICE Transac- lions. Vo1.E 74, No.6, Junc 1991.

D. Pearson: Texture Mapping in model basad image coding. Signal Processing: lmape Com- munication, Vo1.2, N0.4, pp.377-395, Dee 1990.

R. Kwh: Dynamic 3D Scent Analysis through Synthesis Redback Control. Proceedings of rhe IEEE Pattern Analysis and Machine Tntelli- gence. to appear in a special issue on "3D Mdcl- ling for Image Analysis and Synthesis" in 1902/93.

C.-E. Liedtke, H. Buscich. R. Koch: Automatic Modelling of 3D Moving Objccts from a T T

Image Sequence. SPTE Conf. Sensing and Re- construction of 3D-Objecta and Scenes. Vol. 12h0. pp.236239, Santa Clara, USA. 1WO.

R. Koch: Model-Based 3 0 Scene Analysis from Stereoscopic Image Squcncts. Rocdings of the International Smiety of Photogmrwneby and Remote Sensing, ISPRS-Confcrence, Washington, D.C., USA, Aug. 1992.

H. Sacker, Kai Pfitcnn: XRAY - Konfigurier- ang automatischcr R&ntgenpr(lfsysteme. P u b lishcd in 'mas PLAKDN - Buch". Eds.: Cunis. Gilntcr, S ~ k e r . Infomatik Fachberichte 266, Springer Vcrlag. 1W.

T, Mcsstr, M. Schubert: Automatic Configurn- ton of Mcdium- h v e l Routines Using Domain Knowledge. In : Pmcctdings of Vision Inzdacc 91 Conference, Calgwy. f WZ. p. 5 M 3 . The Runge h s s Ltd., 1991.

H. Niemann t t aI.:A knowledge b d vision system for induspial applications. Machine Vi- sion and Applications, 3: pp.201-229, 1 m. H. Niemann et al.:ERNEST: A Semantic Net- work System for Pnrtcm Analysis. TEE& Tmns. Panern Analysis and Machine Vnteltipcnce PAMT-9. pp. RR3-WS. 19W,

B. Neurnann: Wsscnsbnsiene Konfiguration von Bildvtrarkitungssy~temen~ b. Muster- trkennung 1986, Infomatik Fachberichte 125. pp. 206-218, 19%.

C.4 . Liedkc, A. BlMrnes. 'Ih. Gahm: Knowl- edge Based Confi~uration of h a g Seghmnta- tion Rocessea. International Journal of Imaging Systems and Technology. Vol. 2. pp.285-295. 1990.

K. LUtjen: BPI: Ein blackboardbesiertes Roduk- tionssystcm fir die automatische Bildauswer- mnp. 1n:lnfonnatik Fachberichte 125. Editor: G. H m a n n : pp. 1W16R, Springer Vcrlag. 1986.

7%. Gahm, B. Aeikens: CESAR: A Computer Supported Measummtnt System for the En- hancement of Diagnostics and Quality in Cytol- om; Micron qnd Microscopica Acta, Vo1.21, No.lD, pp.29-55. 1990.

'Ih. Dreyer, Th. Gahm, G. DeFling: Significance of image analysis in the diagnosis of bone tu- mours and turnour like lesions: 1 RO/ SPIE Vol. 1357, Med Tech 89: Medical Imaging. pp.180 -1R7,1989.

new machine vision applications in germany - cvl - ikeuchi laboratory

Documents