preliminary lect - biomisabiomisa.org/wp-content/uploads/2020/02/pre-lect.pdf · machine learning...
TRANSCRIPT
1
Digital Image Processing
Preliminary Lect
3
Image Processing & Machine Vision
Low Level Process
Input: Image
Output: Image
Examples: Noise
removal, image
sharpening
From Image Processing to Machine Vision: low, mid and high-level processes
Image Processing
4
Example: Low Level Processing
5
Image Processing & Machine Vision
Low Level Process
Input: Image
Output: Image
Examples: Noise
removal, image
sharpening
Mid Level Process
Input: Image
Output: Attributes
Examples: Object
recognition,
segmentation
From Image Processing to Machine Vision: low, mid and high-level processes
Image Processing
6
Example: Mid Level Processing
Segmentation of image into regions
7
Image Processing & Machine Vision
Low Level Process
Input: Image
Output: Image
Examples: Noise
removal, image
sharpening
Mid Level Process
Input: Image
Output: Attributes
Examples: Object
recognition,
segmentation
High Level Process
Input: Attributes/Image
Output: Understanding
Examples: Scene
understanding,
autonomous navigation
From Image Processing to Machine Vision: low, mid and high-level processes
Image Processing Machine Vision
8
Example: High Level Processing
Object Classification
9
Image Processing & Machine Vision
Low Level Process
Input: Image
Output: Image
Examples: Noise
removal, image
sharpening
Mid Level Process
Input: Image
Output: Attributes
Examples: Object
recognition,
segmentation
High Level Process
Input: Attributes/Image
Output: Understanding
Examples: Scene
understanding,
autonomous navigation
From Image Processing to Machine Vision: low, mid and high-level processes
Image Processing Machine Vision
In this course
Some of this as well
Why Image Processing?• Images and video are everywhere!
Personal photo albums
Surveillance and security
Movies, news, sports
Medical and scientific images
Slide credit; L. Lazebnik
11
Problem Domain Application Input Pattern Output Class
Document Image
Analysis
Optical Character
Recognition
Document Image Characters/words
Document
Classification
Internet search Text Document Semantic categories
Document
Classification
Junk mail filtering Email Junk/Non-Junk
Multimedia retrieval Internet search Video clip Video genres
Speech Recognition Telephone directory
assistance
Speech waveform Spoken words
Natural Language
Processing
Information extraction Sentence Parts of Speech
Biometric Recognition Personal identification Face, finger print, Iris Authorized users for
access control
Medical Computer aided
diagnosis
Microscopic Image Healthy/cancerous cell
Military Automatic target
recognition
Infrared image Target type
Industrial automation Fruit sorting Images taken on
conveyor belt
Grade of quality
Bioinformatics Sequence analysis DNA sequence Known types of genes
Summary of Applications
Text Book & References:
• Rafael C. Gonzalez and Richard E. Woods, Digital Image Processing, 4rd Edition, 2018 (available from local market)
• Rafael C. Gonzalez and Richard E. Woods, Digital Image Processing using MATLAB, 3rd Edition, 2020 (available from local
market) (available from local market)
• Class slides & selected research papers to be shared by the instructor
Course Information• Course Material
– Lectures slides, assignments (computer/written), solutions to problems, projects, and announcements will be uploaded on course web page.
http://biomisa.org/usman/digital-image-processing-sp-20.html
Course Contents
• Introduction to Image processing (Chapter – 1, 2)
• Image processing Fundamentals (Chapter-2)
• Image Enhancement (Spatial & Frequency Domain) (Chapter – 3, 4)
• Morphological operations (Chapter – 9)
• Segmentation (Chapter – 10, Online material, David Forsyth)
• Texture analysis (Chapter – 11, Online material, David Forsyth )
• Image Pyramids & Wavelets (Chapter – 07, Online material, David Forsyth )
• Image representation and description (Chapter – 11)
• Introduction to Machine Learning and Convolutional Neural Networks (Chapter – 12 & Stanford course on CNN)
Contents Breakdown1. Introduction to image processing and it fundamentals
a. Basics of image processingb. Image resolutionc. Connected component analysis
4 hrs
2. Image enhancement in spatial domaina. Intensity transformationsb. Histogram and its analysisc. Convolution and spatial filtering
5 hrs
3. Edge Detection and analysisa. Edge segmentation (magnitude and Phase Analysis)b. Hough Transform
2 hrs
4. Segmentation using thresholding and clusteringa. Global, local and adaptive thresholdingb. K-means clusteringc. Region growing and splitting based segmentation
2 hrs
5. Color image processinga. Formation of color imageb. Different color modelsc. Analysis of colored images
1.5 hrs
6. Morphological operations for binary and gray imagesa. Introduction to morphological operationsb. Morphological operation for binary images
c. Gray level morphological operations
2 hrs
Contents Breakdown7. Image enhancement in frequency domain
a. Basic concepts related to Fourier transformb. Sampling in frequency domain and introduction to DFT
c. Filtering in frequency
3 hrs
8. Texture Analysisa. Statistical descriptorsb. GLCM analysisc. Spectral descriptors
2 hrs
9. Convolution Neural Networksa. Basic concepts of CNNS related to Fourier transform
b. Calculating parameters and layer size for CNN architectures
2 hrs
10. Introduction to wavelets a. motivation of waveletsb. Wavelet decompositionc. wavelet energy
2 hrs
11. Descriptors a. Local Binary Patternb. Histogram of oriented gradients
2 hrs
12. Revision 2 hrs13. Classification Outside
study
Lab BreakdownLab 01 Installation & Introduction to Python and OpenCV
Lab 02 Basic Image Processing & Connected component analysis
Lab 03 Transformation Operations (Assignment-1: Extraction of features and their visualization)
Lab 04 Histogram Equalization and other transformations (Project Idea Presentations)
Lab 05 Spatial Filtering
Lab 06 Edge Detection
Lab 07 Open Lab-1
Lab 08 Image Segmentation (Assignment-2: Segmentation of skin Lesions)
Lab 9 Morphological Operations
Lab 10 Introduction to Machine Learning and Convolutional Neural Network
Lab 11 Image Segmentation using CNN (Assignment-3: Classification of dermoscopic images)
Lab 12 Image Classification using CNN
Lab 13 Frequency Analysis
Lab 14 Texture Based Descriptors
Lab 15 Open Lab -2 & Lab Final (Project Final Presentations)
Lab RubricsUnsatisfactory (0-3 marks) Satisfactory (4-6 marks) Good (7-8 marks) Excellent (9-10 marks)
SpecificationsWeight = 35%
Completed less than 70% of the requirements.
Completed between 70-80% of the requirements.
Completed between 80-90% of the requirements.
Completed between 90-100% of the requirements.
Coding Standards/ Time EfficiencyWeight = 20%
No name, date, or assignment/task title included
Poor use of white space (indentation, blank lines).
Disorganized and messy Poor use of variables (many
global variables, ambiguous naming).
Not delivered on time or not in correct format (disk, email, etc.)
Includes name, date, and assignment/task title.
White space makes program fairly easy to read.
Organized work. Good use of variables (few
global variables, unambiguous naming).
Delivered on time, and in correct format (disk, email, etc.)
Includes name, date, and assignment/task title.
Good use of white space. Organized work. Good use of variables (no
global variables, unambiguous naming)
Delivered on time, and in correct format (disk, email, etc.)
Includes name, date, and assignment/task title.
Excellent use of white space. Creatively organized work. Excellent use of variables (no
global variables, unambiguous naming).
Delivered on time, and in correct format (disk, email, etc.)
DocumentationWeight = 15%
No documentation included. Basic documentation has been completed including descriptions of all variables.
Purpose is noted for each function.
Clearly documented including descriptions of all variables.
Specific purpose is noted for each function and control structure.
Clearly and effectively documented including descriptions of all variables.
Specific purpose is noted for each function, control structure, input requirements, and output results.
Runtime/OutputWeight = 15%
Does not execute due to errors. User prompts are misleading or
non-existent. No testing has been completed.
Executes without errors. User prompts contain little
information, poor design. Some testing has been
completed.
Executes without errors. User prompts are
understandable, minimum use of symbols or spacing in output.
Thorough testing has been completed
Executes without errors excellent user prompts, good use of symbols, spacing in output.
Thorough and organized testing has been completed and output from test cases is included.
EfficiencyWeight = 15%
A difficult and inefficient solution.
A logical solution that is easy to follow but it is not the most efficient.
Solution is efficient and easy to follow (i.e. no confusing tricks).
Solution is efficient, easy to understand, and maintain.
Project RubricsTrait Unsatisfactory
(0-3)Satisfactory
(4-6)Good(7-8)
Exceptional(9-10)
Understanding of Problem 25%Problem
UnderstandingWeight = 15%
There is no knowledge of the problem depth.
The depth of the problem is very weak.
The depth of problem is understood to a sufficient
level.
The depth of the problem is exceptionally
understood.Review
Weight = 10%No effort made to cover
related work of the selected problem.
Related work covered is very sketchy.
Related work of the problem is covered
sufficiently except for the latest developments.
Related work of the problem is covered to an
excellent extent.
Presentation and Demonstration Skills 25%DemonstrationWeight = 25%
The demonstration is poor with no organization and
fails at the cross questioning session.
The demonstration is unorganized and cross
questioning session is sketchy.
Good skills while demonstrating project and fairing fairly good in cross
questioning
Excellent command of project is shown with exceptional question
answer session.Coding and Documentation 50%
SpecificationsWeight = 30%
The program is producing incorrect results.
The program produces correct results but does not display them
correctly.
The program works and produces the correct
results and displays them correctly. It also meets
most of the other specifications.
The program works and meets all of the specifications.
ReadabilityWeight = 10%
The code is poorly organized and very difficult
to read.
The code is readable only by someone who knows what it is
supposed to be doing.
The code is fairly easy to read.
The code is exceptionally well organized and very
easy to follow.Report
(Documentation)Weight = 10%
The documentation is simply comments
embedded in the code and does not help the reader
understand the code.
The documentation is simply comments embedded in the
code with some simple header comments separating routines.
The documentation consists of embedded
comment and some simple header documentation that
is somewhat useful in understanding the code.
The documentation is well written and clearly explains
what the code is accomplishing and how.
Grading PolicyCourse Assessment
Exam: 2 Sessional and 1 FinalHome work: 3 graded Assignments, 04 home tasks (non graded)Lab reports: 12-13 reports, 02 open Labs
Design reports: 1 Design report and 2 presentations based on Semester Project
Quizzes: 6-8 Quizzes
Grading:
Theory (67%) Lab (33%)
2 One Hour Tests (OHTs): 30%
Lab Work/Tasks (Every week) 40%
Open Lab 1 12%
Quizzes: 10% Open Lab 2 12%
Assignments 10% Home Tasks 06%
Final Exam 50% Final Project 30%
Course Learning Outcome
Course Learning Outcome (CLOs) PLOs
Learning
Level Assessments
CLO 1
Understanding the fundamentals and basic concepts
of image processing related to image enhancement,
filtering and segmentation etc
PLO 1 C2
Q1, Q2, A1,
OHT-1, Final
1-question
CLO 2
Performing different mathematical transformations,
histogram based operations and filtering concepts
for solving image enhancement and feature
extraction problems
PLO 2 C3
Q3, Q4, A2,
OHT2, Final 1-
question
CLO 3
Combining the concepts of image processing with
machine learning to analyze and design decision
support systems for image processing based
applications
PLO 3 C4, C6
Q6, A3, Final
1-question
CLO 4
Learning the use of Python and OpenCV to
implement basic image processing algorithms and to
solve real life and open ended problems
PLO 5 P4
Labs, Open
Labs, Project,
Presentation
CODE OF ETHICS
• All students must come to class on time (Attendance will be taken in first 5 to 10 mins)
• Students should remain attentive during class and avoid use of Mobile phone, Laptops or any gadgets
• Obedience to all laws, discipline code, rules and community norms
• Respect peers, faculty and staff through actions and speech
• Student should not be sleeping during class
• Bring writing material and books
• Class participation is encouraged
Policies• No extensions in assignment deadlines.
• Quizzes will be unannounced.
• Exams will be closed book but formula sheets will be allowed
• No Attendance and marking of lab tasks if late more than 15 min
• Never cheat.
– “Better fail NOW or else will fail somewhere LATER in life”
• Plagiarism will also have strict penalties.
Adapted from What is Plagiarism PowerPoint
http://mciu.org/~spjvweb/plagiarism.pptCourtesy Dr. Khawar
Image Sources
• Electromagnetic (EM) band imaging
– Gamma ray band images
– X-ray band images
– Ultra violet band images
– Visual light and infra-red images
– Images based on micro waves or radio waves
• Non-EM band imaging
– Acoustic and ultrasonic images
– Electron microscopy
– Computer generated images (synthetic)
Light & EM Spectrum
• EM Waves
– A stream of mass less particles each travelling in a
wave like pattern, moving at the speed of light and
contains a certain bundle of energy
– The electromagnetic spectrum is split up in to bands
according to the energy per photonHighest Energy Lowest Energy
Light & EM Spectrum
Vis
ible
lig
ht
is j
ust
a p
arti
cula
r p
art
of
the
elec
tro
mag
net
ic s
pec
tru
m t
hat
can
be
sen
sed
by t
he
hu
man
eye
Examples: Imaging in EM bands
Spectral Band Example
Gamma-Rays Nuclear Medicine (Radioactive isotope
injected in the patient)
X-Rays Medical Diagnostic
Ultraviolet Fluorescence microscopy
Visible & Infrared Remote sensing, industrial inspection,
…
Microwave Radar
Radio Medicine – MRI
Examples: Imaging other Modalities
• Sound
– Geological Applications – Oil and Gas Exploration
– Medicine – Ultrasound Imaging
• Synthetic Images
– Computer generated
A synthetic image
Key Stages in DIP
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Key Stages in DIP
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Key Stages in DIP
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Key Stages in DIP
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Key Stages in DIP
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Key Stages in DIP
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Key Stages in DIP
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Key Stages in DIP
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Key Stages in DIP
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Key Stages in DIP
Image
Acquisition
Image
Restoration
Morphological
Processing
Segmentation
Object
Recognition
Image
Enhancement
Representation
& Description
Problem Domain
Colour Image
Processing
Image
Compression
Readings from Book (3rd Edn.)
• Chapter - 1
41
Acknowledgements
Statistical Pattern Recognition: A Review – A.K Jain et al., PAMI (22) 2000
Pattern Recognition and Analysis Course – A.K. Jain, MSU
Pattern Classification” by Duda et al., John Wiley & Sons.
Digital Image Processing”, Rafael C. Gonzalez & Richard E. Woods, Addison-Wesley, 2008
Machine Vision: Automated Visual Inspection and Robot Vision”, David Vernon, Prentice Hall, 1991
www.eu.aibo.com/
Advances in Human Computer Interaction, Shane Pinder, InTech, Austria, October 2008
Mat
eria
l in
th
ese
slid
es h
as b
een
tak
en f
rom
, th
e fo
llow
ing
reso
urc
es