features for handwriting recognition
DESCRIPTION
Features for handwriting recognition. The challenge. “Rappt JD 10 Feb no 175, om machtiging om af”. Short processing pipeline. Learning. “machtiging”. Feature extraction. Classification. 82,34,66,…. “machtiging”. 0.12. Processing pipeline. Preprocessing. Feature extraction. - PowerPoint PPT PresentationTRANSCRIPT
Features for handwriting recognition
| 2
The challenge
“Rappt JD 10 Feb no 175, om machtiging om af”
| 3
Short processing pipeline
“machtiging”
Feature extraction
Classification
82,34,66,…0.12
“machtiging”
Learning
| 4
Processing pipeline
Feature extraction
Classification
Preprocessing
| 5
Input image types
› Color:
› Grayscale:
› Binary:
Preprocessing
› Goal: enhance the foreground while reducing other visual symptoms (stains, noise, pictures, ...)
› Methods:• Contrast stretching• Highpass filtering• Despeckling• Change color representation (RGB, HSV,
grayscale, black/white, …)• Remove selected connected components ()• …
| 6
| 7
Connected components
| 8
Processing pipeline
Segmentation
Feature extraction
Classification
Preprocessing
| 9
Object of classification› Sentences› Words› Characters
(use grammar)(use dictionary)(use alphabet)
| 10
Object representations
› Image› Unordered vectors (in a coco)› Contour vectors› On-line vectors› Skeleton image› Skeleton vectors
(x, y)i
(x, y)k
(x, y)k
(x, y)k
I(x, y)
I(x, y)
| 11
A full processing pipeline
Segmentation
Normalization
Feature extraction
Classification
Preprocessing
| 12
Invariance
› Luminance / contrast› Position› Size› Rotation› Shear› Writer style› Ink thickness› …
| 13
Invariance by normalization
› Luminance / contrast› Position› Size› Rotation› Shear› Writer style› Ink thickness› …
Center on center of gravity
Contrast stretching
Scale to standard
size
| 14
Invariance by trying many deformations› Luminance / contrast› Position› Size› Rotation› Shear› Writer style› Ink thickness› …
Try different scale factors
Try different rotations
… and use the best recognition result
Try different deformation
s
| 15
Invariance by using invariant features
› Luminance / contrast› Position› Size› Rotation› Shear› Writer style› Ink thickness› …
Zernike invariant moments
| 16
A full processing pipeline
Segmentation
Normalization
Feature extraction
Classification
Preprocessing
82,34,66,…
| 17
Feature ROI types
› Whole object› Zones› Windowing
| 18
Whole object (“wholistic”)
| 19
Zones
| 20
Windowing
| 21
Feature types
› Image itself› Statistical› Structural› Abstract
› Image (off-line) features (1—20)› Contour / on-line features (21 – 28)
| 22
Feature 1 – 3
› Connected component images
› Scaled image
› Distance transform (on whiteboard)
| 23
Feature 4: density histogram
| 24
Feature 5: radon transform
| 25
Feature 6: run count pattern
3
6
2 3
| 26
Feature 7: run length pattern
avg
stdev
avg
stdev
| 27
Feature 8: Autocorrelation
| 28
Feature 9: Polar zones
| 29
Feature 10: radial zones (tip!)
| 30
Feature 11: zone histograms
Feature 12: Hinge
| 31
(By Marius Bulacu)
Feature 13: Fraglets
| 32
| 33
Regelmatigheden
Singulariteiten
Feature 14: J.C. Simon (1/2)
| 34
"million" ==> convex:concave:3(north:concave) :(north:LOOP):concave:(north:LOOP) :concave:north :concave:HOLE :2(convex:concave)
(J.-C. Simon, 1989)
Feature 14: J.C. Simon (2/2)
| 35
Feature 15: Structure of background (1/3)
| 36
Feature 15: Structure of background (2/3)
| 37
Feature 15: Structure of background (3/3)
| 38
Feature 16: Structure of foreground + background
| 39
Feature 17: Fourier transform (1/2)
From: http://ccp.uchicago.edu/~dcbradle/pages/5.23.06.html
| 40
Feature 17: Fourier transform (2/2)
Fig. 1 and 3 from: http://www.csse.uwa.edu.au/~wongt/matlab.html
Fig. 2 from: http://www.chemicool.com/definition/fourier_transform.html
| 41
Feature 18: Wavelet transform
From: http://www.regonaudio.com/Audio%20Measurement%20via%20Wavelets.html
| 42
Feature 19: Hu invariant moments
dxdyyxyxiM q
D
pqp ).,(,
0,0M area of the object
0,11,0 ,MM center of mass
Slide from: http://www.cedar.buffalo.edu/~govind/CSE717/lectures/CSE717_3.ppt
› Invariant for scale, position and rotation
› Derived from moments› Moments describe the image distribution with
respect to its axes › Works on (x, y) vectors
| 43
Feature 20: Zernike moments
From: Trier, O. D., Jain, A. K., and Taxt, T. (1996). Feature extraction methods for character recognition - a survey. Pattern Recognition,29:641–662.
| 44
Feature 21 – 28: Contour features
› (cos, sin) of running angle› (cos, sin) of running angular difference› Angular difference› Fourier transform› Ink density (horizontal or vertical)› Radon transform: (ink density, computed radially from
the c.o.g.)› Angular histogram› Curvature scale space ()
| 45
Feature 28: Curvature scale space
From: http://www.christine.oppe.info/blog/category/formen-und-farben/formenvergleich/
pos
itera
tion
End