why d.i.p.?
DESCRIPTION
Why D.I.P.?. Reasons for compression Image data need to be accessed at a different time or location Limited storage space and transmission bandwidth Reasons for manipulation - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/1.jpg)
Why D.I.P.?Why D.I.P.?
Reasons for compressionReasons for compression– Image data need to be accessed at a different time or Image data need to be accessed at a different time or
locationlocation– Limited storage space and transmission bandwidthLimited storage space and transmission bandwidth
Reasons for manipulationReasons for manipulation– Image data might experience nonideal acquisition, Image data might experience nonideal acquisition,
transmission or display (e.g., restoration, enhancement transmission or display (e.g., restoration, enhancement and interpolation) and interpolation)
– Image data might contain sensitive content (e.g., fight Image data might contain sensitive content (e.g., fight against piracy, conterfeit and forgery) against piracy, conterfeit and forgery)
– To produce images with artistic effect (e.g., pointellism)To produce images with artistic effect (e.g., pointellism)
Reasons for analysisReasons for analysis– Image data need to be analyzed automatically in order Image data need to be analyzed automatically in order
to reduce the burden of human operators to reduce the burden of human operators – To teach a computer to “see” in A.I. tasksTo teach a computer to “see” in A.I. tasks
![Page 2: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/2.jpg)
Beyond Image ProcessingBeyond Image Processing
The way of thinkingThe way of thinking– From art (heuristics) to science (principles)From art (heuristics) to science (principles)– The key is mathematics (I will write a separate The key is mathematics (I will write a separate
blog about the role of mathematics in DIP)blog about the role of mathematics in DIP)
The holistic viewThe holistic view– Everything is connected (recall the six-degree Everything is connected (recall the six-degree
phenomenon in social science)phenomenon in social science)
The “Google”-style re-searchThe “Google”-style re-search– Ability to search is a basic part of learning Ability to search is a basic part of learning
![Page 3: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/3.jpg)
Image Compression
Image Manipulation
Image Analysis
Image Acquisition
Image Perception
Image Display
Image Generation
D.I.P.ThemePark
DIP is also about connecting dots – in image compression, you will see why you need to learn matrix theory and statistics
![Page 4: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/4.jpg)
The Art of Image CompressionThe Art of Image Compression
Why are images compressible?Why are images compressible?– Redundancy in images (NOT random)Redundancy in images (NOT random)
How data compression works?How data compression works?– Probability theory and statisticsProbability theory and statistics– Shannon’s information theoryShannon’s information theory
What about the future of image What about the future of image compression?compression?– I will discuss this in my weblog and facebookI will discuss this in my weblog and facebook
(Google “(Google “what happened to Iterated Systems Incorporatedwhat happened to Iterated Systems Incorporated?”) ?”)
![Page 5: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/5.jpg)
Shannon’s Picture on Shannon’s Picture on Communication (1948)Communication (1948)
sourceencoder
channel
sourcedecoder
source destination
Examples of source: Human speeches, photos, text messages, computer programs …
Examples of channel: storage media, telephone lines, wireless transmission …
super-channel
channelencoder
channeldecoder
The goal of communication is to move informationfrom here to there and from now to then
![Page 6: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/6.jpg)
Lossless vs. Lossy CompressionLossless vs. Lossy Compression
Lossless: zero error toleranceLossless: zero error tolerance– No information lossNo information loss– Shannon’s Shannon’s entropyentropy formula formula– For photographic images, compression ratio is For photographic images, compression ratio is
modest (about 2:1)modest (about 2:1)
Lossy: the goal is to preserve the visual quality of Lossy: the goal is to preserve the visual quality of imagesimages– Information loss Information loss visuallyvisually acceptable acceptable– Shannon’s rate-distortion functionShannon’s rate-distortion function– For photographic images, compression ratio is For photographic images, compression ratio is
typically around 10-100typically around 10-100
![Page 7: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/7.jpg)
Popular Lossless Image Compression Techniques
WinZip
- Based on the celebrated Lempel-Ziv algorithminvented nearly 30 years ago
-Based on an enhanced version of LZ algorithmby Welch in 1983-Was introduced by CompuServe in 1987 and madepopular until it was not royalty-free in 1994
GIF (Graphic Interchange Format)
PNG (Portable Network Graphics)
GIF Liberation Day: June 20, 2003
![Page 8: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/8.jpg)
Lossy Image Compression
JPEGdecoder
original raw image (262,144 bytes)
compressed JPEG file (20,407 bytes)
decompressed image
high compression ratio
low compression ratio
low quality
high quality
QQ100
0
![Page 9: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/9.jpg)
From JPEG to JPEG2000
JPEG (CR=64) JPEG2000 (CR=64)
discrete cosine transform based wavelet transform based
![Page 10: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/10.jpg)
Image Compression
Image Manipulation
Image Analysis
Image Acquisition
Image Perception
Image Display
Image Generation
D.I.P.ThemePark
DIP is also about connecting dots – in image manipulation, you will see why you need to learn calculus and Fourier transform
![Page 11: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/11.jpg)
salt and pepper (impulse) noise
Image Manipulation (I): Noise Removal
Noise contamination is often inevitable during the acquisition
additive white Gaussian noise
You will learn how to design image filter in a principled way
![Page 12: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/12.jpg)
License plate is barely legible due to motion blurring
Image Manipulation (II): Deblurring
You will learn the use of FT and the necessity of regularization
![Page 13: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/13.jpg)
overly-exposed image
Image Manipulation (III): Contrast Enhancement
under-exposed image
You will learn how to modify the histogram of an image
![Page 14: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/14.jpg)
Example: aliasing artifacts in MRI image acquisition
Tradeoff between scanning time and image quality(image reconstruction is covered by EE425)
Ideal quality, slow scanning
nonideal quality,fast scanning
Image Manipulation (IV): Aliasing Reduction
![Page 15: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/15.jpg)
small
large
digital zooming
1M pixels
4M pixels
Resolution enhancement can be obtained by common imageprocessing software such as Photoshop or Paint Shop Pro
Image Manipulation (V): Image Interpolation
You will learn the difference between digital and optical zooming
![Page 16: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/16.jpg)
F.Y.I.: search “Gigapixel images” by Google
http://triton.tpd.tno.nl/gigazoom/Delft2.htm
=+
Merge multiple images of the same scene into one with larger FOV
Image Manipulation (VI): Image Mosaicing
There exist several mosaicing software for automatic stitching
![Page 17: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/17.jpg)
blocks contaminated by channel errors(this problem is covered in EE565)
Image Manipulation (VII): Error Concealment
![Page 18: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/18.jpg)
Block artifacts
Image Manipulation (VIII): Deblocking/Deringing
Ringing artifactsYou will learn how to suppress those artifacts by nonlinear diffusion
![Page 19: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/19.jpg)
jittering noise (you will see it in either bonus assignment or final project)
Image Manipulation (IX): Dejittering
![Page 20: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/20.jpg)
Image Manipulation (X): Image Inpainting
You will learn how to do image inpainting in EE565
![Page 21: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/21.jpg)
Image Inpainting Application: Restore Old Photos
![Page 22: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/22.jpg)
25,680 colors (24 bits) 256 colors (8 bits)
Applications: video cell-phone, gameboy, portable DVD
Image Manipulation (XI): Color Quantization
![Page 23: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/23.jpg)
grayscale: 0-255 halftoned: 0/255
Image Manipulation (XII): Image Halftoning
You will learn the famous Floyd-Steinberg diffusion in CA
![Page 24: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/24.jpg)
original scrambled
Image Manipulation (XIII): Image Scrambling/Hashing*
![Page 25: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/25.jpg)
Original image Modified image
Image Manipulation (XIV): Image Watermarking*
![Page 26: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/26.jpg)
Image Manipulation (XV): Image Stylization*
![Page 27: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/27.jpg)
Abysscomputergenerated
Image Manipulation (XVI): Image Rendering*
![Page 28: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/28.jpg)
Image-based Rendering*
![Page 29: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/29.jpg)
Image Compression
Image Manipulation
Image Analysis
Image Acquisition
Image Perception
Image Display
Image Generation
D.I.P.ThemePark
DIP is also about connecting dots – in image analysis, you will see why you need to know about neuroscience and psychology
![Page 30: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/30.jpg)
Image Analysis (I): Edge Detection
You will learn basic edge detectors based on derivatives
![Page 31: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/31.jpg)
Image Analysis (II): Face Detection
Deceivingly simple for humans but notoriously difficult for machines
![Page 32: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/32.jpg)
Image Analysis (III): Change Detection
![Page 33: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/33.jpg)
Change Detection in Medical Application
![Page 34: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/34.jpg)
Image Analysis (IV): Image Matching
Antemortem dental X-ray record Postmortem dental X-ray record
![Page 35: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/35.jpg)
Image Matching in Biometrics
Two deceivingly similar fingerprints of two different people
![Page 36: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/36.jpg)
Image Analysis (V): Image Segmentation
![Page 37: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/37.jpg)
License number can be automaticallyextracted from the image of license plate
Image Analysis (VI): Object Recognition
![Page 38: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/38.jpg)
Object Recognition in Military Applications
![Page 39: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/39.jpg)
Image-based monitoring system prevents drowning
Image Analysis (VII): Event Recognition
![Page 40: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/40.jpg)
Only send out “important” motion pictures such as home-runs
Image Analysis (VIII): Video Summarization
![Page 41: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/41.jpg)
retrieved building images
Image Analysis (IX): Content-based Image Retrieval
![Page 42: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/42.jpg)
SummarySummary
In EE465, you will learnIn EE465, you will learn– Image compression: Lempel-Ziv, Huffman coding, run-length Image compression: Lempel-Ziv, Huffman coding, run-length
coding and JPEGcoding and JPEG– Image manipulation: linear/nonlinear filtering, histogram-based Image manipulation: linear/nonlinear filtering, histogram-based
processing, linear interpolationprocessing, linear interpolation– Image analysis: edge/corner detection, circle/ line/ellipse Image analysis: edge/corner detection, circle/ line/ellipse
detection, chain codes/shape numbersdetection, chain codes/shape numbers
In EE565, you will learnIn EE565, you will learn– Advanced algorithms/techniques with stronger mathematical Advanced algorithms/techniques with stronger mathematical
emphasisemphasis
Not covered by the courses I offeredNot covered by the courses I offered– CBIR (multimedia database), face/pedestrian detection CBIR (multimedia database), face/pedestrian detection
(Advanced biometrics), PDE-based image processing (medical (Advanced biometrics), PDE-based image processing (medical image analysis) image analysis)
![Page 43: Why D.I.P.?](https://reader035.vdocuments.site/reader035/viewer/2022062807/568150e0550346895dbefef0/html5/thumbnails/43.jpg)
General Cooking RecipeGeneral Cooking Recipe
MotivationsMotivationsProblem statementProblem statementHeuristic observationsHeuristic observationsExamples/IllustrationsExamples/IllustrationsPrincipled approaches (via mathematics)Principled approaches (via mathematics)MATLAB implementationsMATLAB implementationsComputer assignment reviewsComputer assignment reviewsCurrent state-of-the-art and future Current state-of-the-art and future directionsdirections