![Page 1: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/1.jpg)
Image Processing Image Processing and Computer and Computer
Vision for RobotsVision for Robots
![Page 2: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/2.jpg)
![Page 3: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/3.jpg)
Convolution-based Convolution-based Image Processing, Image Processing, especially for texture especially for texture partitioning and robot partitioning and robot vision. vision.
![Page 4: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/4.jpg)
Convolution Application Convolution Application in Robot Visionin Robot Vision
• What is convolution
• Different filters formed with Convolution
• Convolution application examples
First few examples what can be achieved with convolution
![Page 5: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/5.jpg)
![Page 6: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/6.jpg)
![Page 7: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/7.jpg)
![Page 8: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/8.jpg)
![Page 9: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/9.jpg)
![Page 10: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/10.jpg)
![Page 11: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/11.jpg)
Binary Image CreationBinary Image Creation
Popularly used in industrial roboticsPopularly used in industrial robotics
![Page 12: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/12.jpg)
Pixels AveragingPixels Averaging
![Page 13: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/13.jpg)
Bit per PixelBit per Pixel
![Page 14: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/14.jpg)
Convolution KernelConvolution Kernel
•New Pixel Value Computed from Neighboring Pixel Values
•Convolution of an N x N Matrix (Kernel) with the Image
![Page 15: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/15.jpg)
Convolution can be done serially, in parallel or in mixed Convolution can be done serially, in parallel or in mixed wayway
No division operation here
![Page 16: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/16.jpg)
• Wi * F(f(x,y))– Function of one variable F can be nonlinear, realized in a lookup
table
• F(f(x1,y1), f(x2,y2),…,f(x9,y9)) can be highly complicated and nonlinear function
Convolution kernelConvolution kernel
![Page 17: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/17.jpg)
Example of a More general ConvolutionExample of a More general Convolution
m i,j is individual mask elements
pij is individual image elements in row i and column j
m1,1 m2,1 m3,1
m1,2 m2,2 m3,2
m1,3 m2,1 m3,3
Original Image Mask
p1,1 p1,2 p1,3 p1,4 p1,5 p1,6
p2,1 p2,2 p2,3 p2,4 p2,5 p2,6
p3,1 p3,2 p3,3 p3,4 p3,5 p3,6
p4,1 p4,2 p4,3 p4,4 p4,5 p4,6
p5,1 p5,2 p5,3 p5,4 p5,5 p5,6
p6,1 p6,2 p6,3 p6,4 p6,5 p6,6
![Page 18: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/18.jpg)
p1,1 p1,2 p1,3 p1,4 p1,5 p1,6
p2,1 p2,2 p2,3 p2,4 p2,5 p2,6
p3,1 p3,2 p3,3 p3,4 p3,5 p3,6
p4,1 p4,2 p4,3 p4,4 p4,5 p4,6
p5,1 p5,2 p5,3 p5,4 p5,5 p5,6
p6,1 p6,2 p6,3 p6,4 p6,5 p6,6
Animation of ConvolutionAnimation of Convolution
m1,1m1,2m1,3
m2,1m2,2m2,3
m3,1m3,2m3,3
Mask
Image after convolution
Original Image
To accomplish convolution of the whole image, we just Slide the mask
C4,2 C4,3 C4,4
![Page 19: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/19.jpg)
More general Convolution (continued)
At the heart of convolution operation is the convolution mask or kernel, shown as M(ask) or W(indow) in next figures
The quotient is known as the weight of the mask
n
j
ij
m
i
n
j
ijij
m
i
m
mp
xyC
11
11
Example of a More general ConvolutionExample of a More general Convolution
Requires division, too bad
![Page 20: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/20.jpg)
Filtering by convolutionFiltering by convolutionAlgorithm
1. Reads the DN of each pixel in array2. Multiplies DN by appropriate weight3. Sums the products of (DN x weight) for the nine pixels, and divides sum by 94. Derived value applied to center cell of array5. Filter moves one pixel to right, and operation is repeated, pixel by pixel, line by line
No. 3
![Page 21: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/21.jpg)
Detecting isolated points by convolution
![Page 22: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/22.jpg)
Requires addition and multiplications, in generalRequires addition and multiplications, in general
Detecting isolated points by convolution
![Page 23: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/23.jpg)
To find edges of objectsTo find edges of objects
Turned to 2-level with edges emphasizedTurned to 2-level with edges emphasized
![Page 24: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/24.jpg)
Image Image FrequenciFrequencies es and and FilteringFiltering
![Page 25: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/25.jpg)
Different Filters formed with Different Filters formed with ConvolutionConvolution
•Different filters can be formed by applying convolution
•Filters that modify their operation as the data elements change also be constructed which is defined as nonlinear filters, e.g. median filter.
•With the former equation, we get linear filters, which each is a summation of weighted pixel intensities and then is divided by a constant value, or weight
![Page 26: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/26.jpg)
Different Filters formed with ConvolutionDifferent Filters formed with ConvolutionBy the frequency-response characteristic, linear filter can be divided into
Frequency highlow
Response
pass
reject
Frequency highlow
Response
pass
reject
Filters (Mask) applied to edge detection or image sharpening are high-pass filter
-- low-pass filter -- high-pass filter
Filters (Masks) applied to zooming and noise elimination
are low pass filter
![Page 27: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/27.jpg)
Filtering Based on Filtering Based on ConvolutionConvolution
Filtering Based on Filtering Based on ConvolutionConvolution
![Page 28: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/28.jpg)
Image FrequenciesImage Frequencies
• Low Frequency Components = Slow Changes in Pixel Intensity
• regions of uniform intensity
![Page 29: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/29.jpg)
Low Frequency Low Frequency ContentContent
![Page 30: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/30.jpg)
High Frequency High Frequency component of image component of image
and filteringand filtering
•High Frequency High Frequency Components = Rapid Components = Rapid Changes in Pixel Changes in Pixel IntensityIntensity
•regions with lots of regions with lots of detailsdetails
![Page 31: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/31.jpg)
High High FrequencFrequency y ComponeComponentnt
![Page 32: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/32.jpg)
HIGH PASS FILTERS
Applied to digital data to remove slowly varying components, the low frequency changes in DNs from pixel to pixel, and to retain high frequency local variations
In general terms, fine detail and edges are emphasized - or enhanced - in the digital data
![Page 33: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/33.jpg)
High Pass FiltersHigh Pass Filters
![Page 34: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/34.jpg)
More About More About KernelsKernels• Coefficients of the Kernel determine its
function
• Color Images: Operate on luminance only, or R/G/B?
• Kernel Size
• smaller kernel = less computation
• larger kernel = higher quality results
![Page 35: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/35.jpg)
How to handle Edge Pixels of the How to handle Edge Pixels of the image?image?
•How to deal with edges (no How to deal with edges (no neighbors in some directions)?neighbors in some directions)?
•Zero fill (black border around Zero fill (black border around image),image),
oror
•duplicate edge pixelsduplicate edge pixels
oror
•don’t process the edges!don’t process the edges!
![Page 36: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/36.jpg)
Basic Low Pass Filters and High Pass Laplacian Filter Basic Low Pass Filters and High Pass Laplacian Filter
image image resultresult
![Page 37: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/37.jpg)
Smoothing or Smoothing or BlurringBlurring
Low-Pass Filtering: Eliminate Details (High Frequencies)
Eliminates Pixelation Effects, Other Noise
![Page 38: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/38.jpg)
Blurring ExampleBlurring Example
![Page 39: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/39.jpg)
Blur of Blur of noisenoise
![Page 40: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/40.jpg)
Blurring continuedBlurring continued
•Sum of Kernel Coefficients = 1
•preserves average image density
•Simple Averaging
•Gaussian Blurring
•coefficients approximate the normal distribution
![Page 41: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/41.jpg)
Gaussian Blur ExampleGaussian Blur Example
![Page 42: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/42.jpg)
Filter Design Example in MathlabFilter Design Example in Mathlab
![Page 43: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/43.jpg)
ZoomingZoomingZoomingZooming
![Page 44: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/44.jpg)
Zooming as a Convolution Application Zooming as a Convolution Application ExampleExample
1 3 2
4 5 6
Zerointerlace
1 0 3 0 2 0
0 0 0 0 0 0
4 0 5 0 6 0
0 0 0 0 0 0
Convolve
1 1 1
1 1 1
1 1 1
The value of a pixel in the enlarged image is the average of the value of around pixels. The difference between insert 0 and original value of pixels is “smoothed” by convolution
Original Image
(1+0+3+0+0+0+4+0+5) ÷(1+1+1+1+1+1+1+1+1) = 13/9
![Page 45: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/45.jpg)
Zooming as a Convolution Application Zooming as a Convolution Application ExampleExample
![Page 46: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/46.jpg)
1 1 1
1 1 1
1 1 1
245 237 243
251 238 239
242 244 247DN value(of cell)
Weight (or operator)
[(245 x 1) + (237 x 1) + (243 x 1) + (251 x 1) + (238 x 1) + (239 x 1) + (242 x 1) + (244 x 1) + (247 x 1)] / 9
New value of DN applied to centre cell
Low Pass Low Pass AverageAverage Filter Filter
Low Pass (Average) Filter
![Page 47: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/47.jpg)
IMAGE ENHANCEMENT BY IMAGE ENHANCEMENT BY LOW PASS FILTERSLOW PASS FILTERS
Filter out medium and high frequencies and’smooth' the image so that sharp edges or gradients, and random 'noise', are suppressed
Low frequency filter examines average brightness value of a number of pixels surrounding the pixel to be enhanced
Each pixel in the array is assigned a 'weight' or 'operator'
![Page 48: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/48.jpg)
Median Filters Median Filters
and Noise and Noise EliminationElimination
Median Filters Median Filters
and Noise and Noise EliminationElimination
![Page 49: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/49.jpg)
Noise Elimination as Convolution Application Noise Elimination as Convolution Application ExamplesExamples
There are many masks used in Noise Elimination
Median Mask is a typical one
23 65 64
120 187 90
47 209 72
J=1 2 3
I=1
2
3
Rank: 23, 47, 64, 65, 72, 90, 120, 187, 209
median
Masked Original Image
The principle of Median Mask is to mask some sub-image, use the median of the values of the sub-image as its value in new image
![Page 50: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/50.jpg)
Median FilteringMedian Filtering
![Page 51: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/51.jpg)
Convolution Application Examples--Noise Elimination
The noise is eliminated but the operation causes loss of sharp edge definition.
In other words, the image becomes blurred
Noise Elimination as Convolution Application Noise Elimination as Convolution Application ExamplesExamples
![Page 52: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/52.jpg)
Median FilteringMedian Filtering
![Page 53: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/53.jpg)
Median FilteringMedian Filtering
![Page 54: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/54.jpg)
SmartSmart Approaches to Approaches to RobotRobot Vision Vision
![Page 55: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/55.jpg)
• 1) We can have stored models of line-drawings of objects (from many possible angles, and at many different possible scales!), and then compare those with all possible combinations of edges in the image. – Notice that this is a very computationally intensive and expensive
process.
– This general approach, which has been studied extensively, is called model-based vision.
There are several good approaches to detect objects:There are several good approaches to detect objects:
Model-based vision. Model-based vision.
![Page 56: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/56.jpg)
• 2) We can take advantage of motion.
– If we look at an image at two consecutive time-steps, and we move the camera in between, each continuous solid objects (which obeys physical laws) will move as one, i.e., its brightness properties will be conserved.
– This gives us a hint for finding objects, by subtracting two images from each other.
– But notice that this also depends on knowing well:
• how we moved the camera relative to the scene (direction, distance),
• and that nothing was moving in the scene at the time.
– This general approach, which has also been studied extensively, is called
motion vision.
Motion vision. Motion vision.
![Page 57: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/57.jpg)
• 3) We can use stereo (i.e., binocular stereopsis, two eyes/cameras/points of view). – Just like with motion vision above, but without having to
actually move,
– we get two images,
– we subtract them from each other,
– if we know what the disparity between them should be, (i.e., if we know how the two cameras are organized/positioned relative to each other), we can find the information like in motion vision.
Binocular stereopsisBinocular stereopsis
![Page 58: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/58.jpg)
• 4) We can use texture.
– Patches that have uniform texture are consistent, and have almost identical brightness, so we can assume they come from the same object.
– By extracting those we can get a hint about what parts may belong to the same object in the scene.
TextureTexture
• 5) We can also use shading and contours in a similar fashion. – And there are many other methods, involving object shape and
projective invariants, etc.
Shading and contoursShading and contours
![Page 59: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/59.jpg)
Biologically MotivatedBiologically Motivated• Note that all of the above strategies are employed in
biological vision.
• It's hard to recognize unexpected objects or totally novel ones (because we don't have the models at all, or not at the ready).
• Movement helps catch our attention.
• Stereo, i.e., two eyes, is critical, and all carnivores use it – unlike herbivores, carnivores have two eyes pointing in the same
direction.
• The brain does an excellent job of quickly extracting the information we need for the scene.
![Page 60: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/60.jpg)
Clever Special Tricks that work:Clever Special Tricks that work:• Machine vision has the same task of doing real-time vision.
• But this is, as we have seen, a very difficult task.
• Often, an alternative to trying to do all of the steps above in order to do object recognition, it is possible to simplify the vision problem in various ways: – 1) Use color; look for specifically and uniquely colored objects,
and recognize them that way (such as stop signs, for example)
– 2) Use a small image plane; instead of a full 512 x 512 pixel array, we can reduce our view to much less.
• For example just a line (that's called a linear CCD).
• Of course there is much less information in the image, but if we are clever, and know what to expect, we can process what we see quickly and usefully.
![Page 61: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/61.jpg)
Smart Tricks continued:Smart Tricks continued:– 3) Use other, simpler and faster, sensors, and combine
those with vision. • For example, IR cameras isolate people by body-
temperature.
• Grippers allow us to touch and move objects, after which we can be sure they exist.
– 4) Use information about the environment; • if you know you will be driving on the road which has
white lines, look specifically for those lines at the right places in the image.
• This is how first and still fastest road and highway robotic driving is done.
![Page 62: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/62.jpg)
Vision as good sensor selectionVision as good sensor selection• Those and many other clever techniques have to be employed when
we consider how important it is to "see" in real-time.
• Consider highway driving as an important and growing application of robotics and AI.
– Everything is moving so quickly, that the system must perceive and act in time to react protectively and safely, as well as intelligently.
• Now that you know how complex vision is, you can see why it was not used on the first robots, and it is still not used for all applications, and definitely not on simple robots.
• A robot can be extremely useful without vision, but some tasks demand vision.
• As always, it is critical to think about the proper match between the robot's sensors and the task.
![Page 63: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/63.jpg)
• 1. Having talked about the type of sensors (external and proprioceptive), think about how they can be useful for general robotic tasks like navigation and manipulation.
• 2. Proprioceptive sensors sense the robot's actuators (e.g., shaft encoders, joint angle sensors, etc.); they sense the robot's own movements. – You can think of them as perceiving internal state instead
of external state.
• 3. External sensors are helpful but not necessary or as commonly used. Think of all.
Questions and ProblemsQuestions and Problems
![Page 64: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/64.jpg)
• 4. Before we will learn about these areas in more detail, without any external influence, try to think how you would write software for the following:– recognition of obstacles based on color,– recognition of obstacles based on shape,– recognition of moving targets based on motion,– recognition based on three-dimensional model of
obstacles.
• 5. Collect information on inexpensive computer cameras and analyze which of them is best for an eye of a robot head. Two such cameras are needed.
Questions and ProblemsQuestions and Problems
![Page 65: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/65.jpg)
Questions and ProblemsQuestions and Problems• 6. Write all applications of image processing and computer vision that you can think of
• 7. For some of them, that you are more familiar with, write what are the necessary stages of processing
• 8. Discuss convolution applications for color images.
• 9. How to remove high contrast details from an image
• 10. Apply your knowledge of filtering from circuit classes to images. How to design an image filter for a specific application
![Page 66: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/66.jpg)
Questions and ProblemsQuestions and Problems• 11. Write a Lisp program for zooming of a grey level image.
• 12. Write a Lisp program for a median filter
• 13. Design digital hardware of a pipelined median filter
• 14. Design a computer for convolution in which multiplying is done always by powers of two and implemented by shifting
• 15. Write Lisp program for noise elimination and experiment with different kernels
![Page 67: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/67.jpg)
SourcesSources
• Maja Mataric
• Dodds, Harvey Mudd College
• Damien Blond
• Alim Fazal
• Tory Richard
• Jim Gast
• Bryan S. Morse• Gerald McGrath
• Vanessa S. Blake
• Many sources of slides from Internet
http://www.cheng.cam.ac.uk/seminars/imagepro/
•Bryan S. Morse•Many WWW sources•Anup Basu, Ph.D. Professor, Dept of Computing Sc. University of Alberta• Professor Kim, KAIST• Computer science, University of Massachusetts, Web Site: www-edlab.cs.umass/cs570
![Page 68: Image Processing and Computer Vision for Robots](https://reader031.vdocuments.site/reader031/viewer/2022032105/56649dbc5503460f94aaec0e/html5/thumbnails/68.jpg)
• 533 Text book• http://sern.ucalgary.ca/courses/CPSC/533/W99/presentations/L2_24A_Lee_Wang/ http://sern.ucalgary.ca/courses/CPSC/533/W99/presentations/L1_24A_Kaasten_Steller_Hoang/main.htm http://sern.ucalgary.ca/courses/CPSC/533/W99/presentations/L1_24_Schebywolok/index.html http://sern.ucalgary.ca/courses/CPSC/533/W99/presentations/L2_24B_Doering_Grenier/• http://www.geocities.com/SoHo/Museum/3828/optical.html• http://members.spree.com/funNgames/katbug/
SourcesSources