Int. J. Modelling, Identification and Control, Vol. 1, No. 2, 2006

Copyright © 2006 Inderscience Enterprises Ltd.

94

A compact vision sensor for weld pool surface sensing

Gohar Saeed, Yu Ming Zhang* and Sarah Cook Department of Electrical and Computer Engineering, Center for Manufacturing, University of Kentucky, Lexington, KY 40506, USA E-mail: gmsaee0@uky.edu E-mail: ymzhang@engr.uky.edu E-mail: secook0@engr.uky.edu *Corresponding author

Abstract: The purpose of vision-based sensing devices in the welding industry is to electronically replicate the role of a skilled welder and emulate the human eye with a light-sensing device such as camera and the human brain with a computer algorithm that interprets the images. Just as an optical feedback from human eye guides the human welder, optical feedback in this electronic system would be used to control a welding process, in the case of this research, the Gas Tungsten Arc Welding (GTAW). Such sensing systems have been developed, but the purpose of this study is to build one using more commonly available elements and on a much smaller scale, as to be able to attach it to an already-existing welding system without imposing dramatic space requirements on the system. This paper discusses the procedure employed in developing the knowledge base and the experimental system used for building this compact sensor. Experiments have been performed to determine the positioning of the lens, its focal length and size. A study of the illuminating system is also documented towards understanding how light is dispersed under the welding environment. The illumination system is based on structured laser light, where a laser line is projected on the weld pool. The weld pool is divided into three parts: front (deepest), middle and back (shallow). Experiments have been performed to determine the position where the laser light needs to hit the weld pool and how it is reflected from various points of the weld pool.

Keywords: weld pool; surface; sensor; vision; image; weld penetration; joint penetration; welding; automation; manufacturing.

Reference to this paper should be made as follows: Saeed, G., Zhang, Y.M. and Cook, S. (2006) ‘A compact vision sensor for weld pool surface sensing’, Int. J. Modelling, Identification and Control, Vol. 1, No. 2, pp.94–100.

Biographical notes: Gohar Saeed obtained his initial education in the United Arab Emirates (UAE). He received his PhD in Electrical Engineering from the University of Kentucky focusing on machine vision applications to welding processes under Prof. Yu Ming Zhang for four years in the Center for Robotics and Manufacturing Systems. He also received his MS and BS in Electrical Engineering from the University of Kentucky. His research interests include machine vision, controls, automation, welding and image processing. Currently, he is working for Progressive Systems in Berea, Kentucky as a Project Engineer.

Yu Ming Zhang is a Professor and the Director of Graduate Studies in the Department of Electrical and Computer Engineering at the University of Kentucky, where he holds the James R. Boyd Professorship in Electrical Engineering and directs the Welding Research Laboratory and Applied Sensing and Control Laboratory. He received his PhD in Mechanical Engineering/Welding Major and MS and BS in Electrical Engineering/Control Major from the Harbin Institute of Technology, Harbin, China. He is a senior member of the IEEE and the SME and a member of the AWS and the ASME. He is the recipient of the Donald Julius Groen Prize from the Institution of Mechanical Engineers, UK, the AF Davis Silver Medal award and the Adams Memorial Membership Award from the American Welding Society and the 15th IFAC Triennial World Congress Poster Paper Prize and Application Paper Honorable Mention from the International Federation of Automatic Control.

Sarah Cook is a Senior Electrical Engineering student at the University of Kentucky. She worked for the Center for Robotics and Manufacturing Systems at the University of Kentucky under the supervision of Dr. Yu Ming Zhang for eight months before going on to two terms of co-op at Lexmark International. She is active in the Society of Women Engineers, serving as the Vice-President, Secretary and Career Fair Director in her time at the University of Kentucky.

Dynamic, capability-driven scheduling of DAG-based real-time jobs in heterogeneous clusters 95

1 Introduction

For critical and accurate joining where the weld quality must be ensured, such as for the root pass and for the welding of advanced materials, Gas Tungsten Arc Welding (GTAW) is often used to produce a high-quality weld because of its capability in precision control of the fusion process (Swaim, 1998). Its weld penetration control has been a fundamental subject in automated welding. Significant advances over time in the sensor development have allowed for limited automation of welding processes. Several methods have been studied, including pool oscillation (Renwick and Richardson, 1983; Xiao and den Ouden, 1993; Zacksenhouse and Hardt, 1983), ultrasound, infrared sensor (Beardsley et al., 1994; Nagarajan et al., 1992), radiography, etc. Acoustic emission sensing (Cannon et al., 1987) is capable of distinguishing between full penetration and partial penetration. Though many methods are available to measure the weld penetration, new or improved solutions are still strongly needed, because of the inherent restrictions associated with each of the above-mentioned methods. The difficulty associated with this problem is to find a precise and reliable way to measure the weld penetration using only topside sensors, which are attached to and move with the torch. With the advancement of machine vision, vision-based sensors have been used and studied to sense the penetration of the weld pool. Several techniques such as stereo vision (Mnich et al., 2004), pulsed laser and synchronised camera systems (Maqbool et al., 1998; Zhang et al., 1996; Zhao et al., 1998) have been used to observe the weld pool. However, the equipment involved with these sensing and measurement methods is expensive, as it requires multiple cameras or a specialised pulsed laser with synchronised cameras to observe the weld pool. These procedures also impose a large space requirement on the system making application harder.

In this paper, a novel vision-based sensor is presented. The sensor is compact and measures 45.72 mm × 32.512 mm × 54.102 mm (1.85′ × 1.28′ × 2.13′). The sensor is mounted directly on the existing GTAW torch without imposing any drastic space requirement. The sensor works based on structured light technique. A laser line is projected on the weld pool and the sensor is placed on the other end, which collects the reflected laser light from the mirror like molten weld pool. The deformation of the stripe is determined by the deformation of the weld pool surface. Thus, the sensed reflection of the stripe can be processed and used to compute the Three-Dimensional (3D) pool surface. The entire sensor is attached to and moves with the torch.

2 Design of optical parameters

The basic organisation for the apparatus described in this paper is shown in Figure 1. This sensing system has three major variables that need to be determined. Firstly, the angle and distance of the laser, secondly the angle of disbursement of the laser after being reflected off the

molten weld pool and finally the parameters of lens as the positioning, focal length and the place where the CCD sensor needs to be placed for imaging.

Figure 1 The compact sensor illustrated with the whole system

2.1 Determining the position of laser

The angle of the laser is an important variable concerning the positioning of the sensor. The angle at which the laser is reflected after hitting the weld pool depends entirely on the surface of the weld pool and the incident angle of the laser beam. Therefore, to determine the position of the sensor, where it can capture the reflected rays it is important to experiment and fix the incident angle. Varying over the range between 0° and 90° for the incident angle, it was determined that a too-steep angle (>60°) or a too-shallow angle (<20°) will cause a problem in the illumination and observation of the weld pool. A too-steep angle will cause the torch to cast a ‘shadow’ over the image of the weld pool as the laser will be reflected at too high of an angle to clear the far edge of the torch. A too-shallow angle will cause difficulty in taking useful data, or will cause the weld pool to cast a ‘shadow’ over itself (due to internal reflection from the edge of the weld pool, that is, the laser may get reflected back after hitting the edge of the weld pool). Also a too-shallow angle will cause the laser to cast a very elongated curve along the weld pool, and make the set of data taken from any position of the laser either unimportant (Figure 2) or very limited. The accuracy of the system will also be compromised at a shallow angle, as it will be harder to focus the laser line at a shallow angle. Another difficulty with shallow angle would be keeping the far edge of laser in the centre of the weld pool. The data from the positioning in Figure 2 would not be sufficient to determine the depth of the weld pool since, though the laser is positioned at the centre of the weld pool, the angle of the laser would cause the resulting image to be of the front edge of the weld pool, alone. Hence, the authors have experimented within the range of 15–55° and found 20–25° to be of sufficient acceptability. It is possible that different arrangements of hardware would require a different angle to optimise the data taken. Figure 3 shows shadowing of laser due to electrode at a too-steep angle.

As the shape of the surface of the weld pool is similar to a concave mirror, it is necessary to take into account how the laser is reflected from front, back and middle of the weld pool as shown in Figure 4. Each of these

96 G. Saeed, Y.M. Zhang and S. Cook

positions must be taken into account, as in the final application of this system, an array of laser lines will be used that will span the length of the weld pool, thus describing the entirety of the surface of the weld pool in a 3D coordinate system. Therefore, it is important to find out the range of angle over which the laser is reflected back from the weld pool. This range will help determine the position where the CCD sensor needs to be placed to collect the reflected laser. Figure 4 shows how the weld pool reflects the laser at various positions along its surface.

Figure 2 The elongated laser line due to shallow projection angle. The elliptical weld pool shape is an approximation to make theoretical demonstration easier

Figure 3 Shadowing caused at a steep laser projection angle

Figure 4 Illustration of how laser reflects from the surface of the weld pool

2.2 Determining the range of reflected laser

Another important parameter that needs to be determined is the position where the CCD sensor is to be placed. The CCD sensor should be placed in a place where it can capture the reflected laser in its entirety. Therefore, several

experiments need to be performed in which the incident laser angle is fixed and the range in which the laser is reflected is determined. Knowing this range, the optics and the CCD sensor can be engineered to capture the entire range of reflected laser. Hence, no occlusions, where the laser temporarily leaves the sensors field of view, will take place. In performing the experiment to determine the range of the reflected laser, the incident laser angle was fixed at 25° and a paper plane placed 85.725 mm (3.375″) away from the weld pool. Paper was used as an imaging plane to mark the areas where the laser line fell after being reflected from the weld pool. The laser was then scanned over the length of the weld pool. The resulting curve spanned ∼25.4 mm (1.0″) from top to bottom on the image plane. The bottom edge or the starting point was previously established to be at 20°. This indicates a range of 13.4°, starting at 20° and going up to 34.4°, making the centre line at an angle of 26.7° (see Figure 5). This will be used to align the CCD so that the centre lines go through the middle of the lens and hit the middle of the CCD, at an angle of 26.7° above the plane of the weld pool. These were the operating specifications for the lens/CCD system, as the lens needed to capture the entire image, no matter where the laser hit on the weld pool.

Figure 5 The range of laser reflected with centre line at 26.7°

2.3 Optics

The next system parameter that needed to be determined was the diameter of lens, its focal length and positioning. The lens had to be large enough in size to capture the reflected laser range and focus it on the CCD sensor placed behind the lens. Hence, the distance of lens from the weld pool, the distance of lens to the CCD sensor and the diameter of lens were three variables that had to be determined. A requirement imposed on the lens positioning was that the image had to be diminished in size, so that it would fit it on the CCD sensor placed behind it (since the laser line diverges as it travels away). This could be accomplished by placing the lens at a distance of 2f away from weld pool, where f is the focal length of the convex lens, and placing the CCD sensor between f and 2f behind

A compact vision sensor for weld pool surface sensing 97

the lens. Figure 6 shows the different positions of the lens with reference to the weld pool and how the resulting image would be affected.

The first placement of the lens (see Figure 6(a)), with the weld pool at a distance less than f shows the image forming far away from the lens, with magnification M > 1. This set-up is not desired, even though the lens being quiet close to the weld pool ensures that all the reflected lasers are captured before diverging away; however, the rays after passing through lens form a magnified image that may not fit on the CCD sensor. The second placement of the lens (see Figure 6(b)), with the weld pool at 2f shows the image forming at 2f with M = 1. This means that the size of the image formed is the same as the object. In our requirement, we need the image to be diminished or smaller than the object. Hence this set-up is also not desired. The third placement of the lens (see Figure 6(c)), with the weld pool beyond 2f shows the image formed in between f and 2f, with magnification less than 1 (M < 1). This third set-up is the desired set-up, as it allows for shrinking of the image to fit the CCD, and also allows, through a series of optimisations, a relatively close set-up of the lens and the sensor to the weld pool. The additional the lens is placed from the object, the smaller the image is going to be and the more likely it will all fit on the CCD, but at a larger distance, the lens is less likely to capture all of the reflected laser light. Both a larger diameter lens and a shorter focal length help fulfil these two requirements.

Figure 6 Lens diagram demonstrating object placement and image formation by a convex lens: (a) object placed between f and 2f forms a magnified image, (b) object at 2f produces an image at 2f of same height and (c) object beyond 2f produced a diminished image between f and 2f

The focal length of the system was set to be 15 mm (0.5906″) after giving consideration to the standard lens in market and their sizes. It was determined that with a focal length of 15 mm, placed at a distance of 33.116 mm (1.3038″) (>2f), the diameter of the lens had to be at least 11.453 mm (0.4509″) to capture the reflected laser in the range of 20–34.4°. Hence, a standard 15 mm (0.5906″) diameter × 15 mm (0.5906″) focal length lens was used placed at a distance of 33.116 mm (1.3038″) away from the weld pool. The distance of the CCD sensor from the

lens was determined based on the equation 1/f = ((1/u) + (1/v)), where u is the distance of the object from the lens and v is the distance of the image plane from the lens. Therefore, v, the distance of CCD sensor from the lens was computed to be 27.42 mm (1.0795″).

It should be noted that, given the angle of the laser, the distance of the laser from the weld pool is relatively unimportant as increasing the distance between the laser and the weld pool only expands the width of area covered by the laser on either side of the weld pool. The two extremes to be avoided, of course, are that where the laser does not completely span the weld pool and that where focusing the laser to a fine line across the weld pool becomes too difficult.

3 Hardware set-up

The sensor was built using the Kodak KAI-2001 CCD sensor. The sensor was placed on a custom-built PCB board to provide mobility and miniaturise the size of the sensor. The PCB board provided the timing and control signals to the CCD sensor from the evaluation board. The evaluation board for the KAI-2001 image sensor is divided into a timing board and an imaging board. The timing board generates clock and control signals based on the desired mode of operation for the KAI-2001 chip. The signals are low voltage differentially transmitted to the imaging board where they are conditioned and received by the KAI-2001. The output from the imaging chip is then coaxially transmitted to the timing board. The signal is then processed and LVDS sent to the PC performing control functions. The custom-built PCB board had hardware to drive the output signal directly to the imaging board. From Figure 7, noise reduction (ferrites and shunt capacitors) was a fairly major concern for the output line driver.

Figure 7 Output driving hardware from the PCB board to the imaging board

A driving board was built that attached to the timing board and had circuitry to drive the signals to the PCB board with the CCD sensor. The circuit used on the driver board that generated 20 MHz signals off the Kodak

98 G. Saeed, Y.M. Zhang and S. Cook

imaging board is shown in Figure 8. The operational amplifier used was selected for high bandwidth and skew rate. The gain on this was set to be slightly above 1 using components from the analogue driver circuit. The unbalanced line impedances allow a more readily available op-amp power source and variable line impedance. The lower frequency and DC signals were directly sent.

Figure 8 Signal driver hardware for driving signals from timing board to PCB board

For connection, an 18-conductor individually foil shielded cable with an outer foil shield was used. One end of all shield drain wires was grounded. Initially the conductors were physically connected to the boards with solder. This physical layer was chosen to avoid crosstalk and provide somewhat stable impedance to the line. Some of the DC signals were sent directly rather than made with voltage dividers to reduce component population.

Band-pass filters and neutral density filters were placed in front of the sensor. These filters were necessary

to block out the arc light and allow the light in the bandwidth of laser light to pass through. The band-pass filter blocks all light but the light in the bandwidth of 685 ± 10 nm to pass through. Some light from the arc in that range passes through the filter. However, the intensity of the arc light is much less compared to the intensity of the laser light, and therefore the neutral density filters block the arc light and allows the higher-powered laser light to go through. Two neutral density filters in the order of 1% and 10% transmission in series were used. The lens is mounted on a lens holder at a distance of 27.42 mm (1.0795″) away from the CCD sensor surface with filters in between. Figure 9(a) shows the CCD sensor placed on the custom CCD, Figure 9(b) and (c) show the compact sensor and the cross-sectional view of the sensor with its elements.

The filters to be used were determined through a series of experimentation. The optimal filter combination was selected based on the desired results of having the arc light blocked out and just the laser light pass through. Figure 10(a) shows the result of experiment conducted with no filter placed in front of the CCD sensor. Figure 10(b) shows the experimental result of placing a band-pass filter centred at 685 nm to block out some of the arc light. Figure 10(c) shows the results of placing band-pass filters and neutral density filters placed in series to filter out the remaining arc light.

Figure 9 The compact sensor: (a) the CCD sensor placed on the custom-designed PCB board (b) the vision-based compact sensor side view and (c) the cross-sectional view of the compact sensor with its elements

Figure 10 Test images acquired: (a) without filters, (b) with band-pass filter centred at 685 nm and (c) with a neutral density and band-pass filter in series

A compact vision sensor for weld pool surface sensing 99

4 Results

The sensor was used in several experiments to observe the weld pool. In the experimental results reported in this section, the laser line is projected at Position 2 as shown in Figure 4. The reflected laser light was successfully captured using the compact sensor and the distortion in the laser line distinctively showed the shape of the weld pool. Figure 11 shows some of the data captured by the compact sensor. An interesting finding was that the shape of the weld pool is like a convex mirror and shaped like a blob when the metal is fully penetrated (see Figure 11(a)). However, when the welding current is increased and the weld penetrates the metal, the weld pool is like a concave mirror and deepens as the penetration increases (see Figure 11(b)).

Figure 11 Partial and full penetration experiment: (a) partial penetration and (b) full penetration

The depth of the weld pool could also be observed using the compact sensor. Figure 12 shows a series of frames captured during an experimental run in which the welding current was slowly increased and as a result the weld pool depth increased. It can be observed from the images that the curvature of the laser line changes to reflect the depth of the weld pool. In Figure 12(a), the weld pool is not very deep; when the current is increased, the resulting laser line has a deeper curvature and is shown in Figure 12(b) because of the increased penetration. When the current is further increased, the weld pool penetration increases further, and the curvature of the captured laser line is even greater as shown in Figure 12(c).

Figure 12 Experiment at different penetration level: (a) shallow weld pool with little penetration (b) increased penetration and (c) further increased penetration

5 Conclusion

Sensing applications are present throughout the welding industry, giving engineers the tools necessary to build mechanical welding systems that produce higher and higher quality welds more efficiently and more predictably. A system such as this, with as much mobility as a compact sensor allows, lends itself to many applications with a very little or no mechanical alteration to the system. Any timed welding system that requires a human to make sure that the system is welding the right area would benefit from this system, which could be easily altered, through software, to detect seams or other visual characteristics that a human would use for reference to guide a machine. Alignment issues would be eliminated, though welding systems would need to have the ability to react to feedback added, and possibly would require a wider range of motion to react to data analysed by the sensor system described here. A visual system that relies on area light and photographic analysis software would struggle to function while welding is taking place, as brilliant light from the welder washes out the camera in just the area that is crucial for the application: the area being welded. This system eliminates that issue, and, whereas our application has been specifically interested in the surface of the weld pool, the range of this system can be easily broadened to encompass an area around the weld that a conventional camera system would find difficult to analyse.

Acknowledgement

This work is funded by the National Science Foundation under Grants DMI-0114982 and DMI-0527889.

References

Beardsley, H.E., Zhang, Y.M. and Kovacevic, R. (1994) ‘Infrared sensing of full penetration state in gas tungsten arc welding’, International Journal of Machine Tool Manufacturing, Vol. 34, pp.1079–1090.

Cannon, I., Maram, J. and Smith, M. (1987) ‘Weld penetration sensor for an experimental welder’, Transactions of NAMRC/SME, Vol. XV, p.12.

Maqbool, S., et al. (1998) ‘Neural networks for the control of welding systems’, Proceedings from the Eighth International Conference on Computer Technology in Welding, Liverpool University, UK, June.

Mnich, C., et al. (2004) ‘In situ weld pool measurement using stereovision, Paper #UL 082’, Proceedings of 2004 JUSFA 2004 Japan USA Symposium on Flexible Automation, Denver, Colorado, 19–21 July.

Nagarajan, S., Bangerjee, P., Chen, W. and Chin, B.A. (1992) ‘Control of the welding process using infrared sensors’, IEEE Transactions on Robotics and Automation, Vol. 8, pp.86–93.

Renwick, R.J. and Richardson, R.W. (1983) ‘Experimental investigation of GTA weld pool oscillations’, Welding Journal, Vol. 62, pp.29s–35s.

100 G. Saeed, Y.M. Zhang and S. Cook

Swaim, W. (1998) ‘Gas tungsten arc welding made easy’, Welding Journal, Vol. 77, pp.51–52.

Xiao, Y.H. and den Ouden, G. (1993) ‘Weld pool oscillation during GTA welding of mild steel’, Welding Journal, Vol. 72, pp.428s–434s.

Zacksenhouse, M. and Hardt, D.E. (1983) ‘Weld pool impedance identification for size measurement and control’, ASME Journal of Dynamic Systems Measurement and Control, Vol. 105, pp.179–184.

Zhang, Y.M., Kovacevic, R. and Li, L. (1996) ‘Characterization and real-time measurement of geometrical appearance of weld pool’, International Journal of Machine Tool and Manufacturing, Vol. 36, No. 7, pp.799–816.

Zhao, J., et al. (1998) ‘Real-time image processing for penetration control in automated welding’, Proceedings from the Eighth International Conference on Computer Technology in Welding, Liverpool University, UK, June.