traditional image processing
DESCRIPTION
Traditional Image Processing. Data Structures Images, Palettes , Histograms , Profiles , etc. For many datatypes (8bit, 16 bit , float , etc.) Algorithms Dependent on Data Structures Dependent on datatypes Combinatorial Explosion -> unmanageable - PowerPoint PPT PresentationTRANSCRIPT
www.ngi-central.org 129.04.2010
www.ngi-central.org 2
Traditional Image Processing
• Data Structures– Images, Palettes, Histograms, Profiles, etc.– For many datatypes (8bit, 16 bit, float, etc.)
• Algorithms– Dependent on Data Structures– Dependent on datatypes
• Combinatorial Explosion -> unmanageable– To implement one new algorithm, it must be
written many times – for each type
29.04.2010
www.ngi-central.org 3
Generic Image Processing• Generic Programming comes to the rescue
• Three key design techniques– Iterators decouple data structures and
algorithms– Functors allow to exchange parts of
computation– Generic algorithms in terms of
abstract iterators and functors
• Enabling technology of compilers is the template<> mechanism
29.04.2010
Algorithms
Iterators
Data
www.ngi-central.org 4
Standard Template Library
• STL is an implementation of generic programming concepts
• STL is available with all compilers since it is part of the C++ standard
• STL is around 10 years old and considered mature
• Educational material to learn generic programming in general and STL in particular is available
29.04.2010
www.ngi-central.org 5
Example: RGB to Gray
• Using STL style programming
struct RGBValue {unsigned char red, green, blue;
};
struct RGBToGray {unsigned char operator()(const RGBValue& rgb) const {
return 0.3*rgb.red + 0.59*rgb.green + 0.11*rgb.blue;}
};
vector<RGBValue> rgb;vector<unsigned char> gray;...transform(rgb.begin(), rgb.end(), gray.begin(), RGBToGray());
29.04.2010
www.ngi-central.org 6
New: works for all types!
Better example• Templated on the datatype<template class T>struct RGBValue {
T red, green, blue;};
<template class T>struct RGBToGray {
T operator()(const RGBValue<T>& rgb) const {return 0.3*rgb.red + 0.59*rgb.green + 0.11*rgb.blue;
}};
vector<RGBValue<float> > rgb;vector<float> gray;...transform(rgb.begin(), rgb.end(), gray.begin(), RGBToGray<float>());
29.04.2010
www.ngi-central.org 7
STL leaves things to desire
• STL is designed to work with 1D data• Images are inherently two(multi)dimensional• Some algorithms do not need the
dimensionality– Use STL in this case
• Some algorithms need the dimensionality– Cannot use STL, need something different
29.04.2010
www.ngi-central.org 8
Multidimensional Locators
• A locator is a multidimensional iterator
• Logical extension to an iterator
Move the iterator++it;--it;it+=100;it-=50;
Move the locator++it.x;--it.y;it.x+=100;it.y-=50;
29.04.2010
www.ngi-central.org 9
Algorithms change as well
// STL implementationtemplate<class IT, class F>IT transform(IT First, IT Last, IT Dest, F Func){
for (; First != Last; ++First, ++Dest)*Dest = Func(*First);
return (Dest);}
// 2D implementationtemplate<class IT, class F>IT transform(IT First, IT Last, IT Dest, F Func){
for (; First.y != Last.y; ++First.y, ++Dest.y)for (; First.x != Last.x; ++First.x, ++Dest.x)
*Dest = Func(*First);return (Dest);
}
29.04.2010
www.ngi-central.org 10
What do we gain?• An algorithm like transform replaces many
functions in traditional programming style
• It can do this, because it is templated on the function– We still need to program the functionality,
but decoupled from navigation– It saves us to rewrite the loops many times– Bonus: the function objects can be reused in
completely different algorithms as well
• It can do this, because it is templated on the datatype (by iterator indirection)– It saves us to rewrite the complete thing for each datatype
• The gain is tremendous (time, functionality, flexibility)
29.04.2010
www.ngi-central.org 11
NGI - Goals• Image Processing
– Image -> Image
• Image Analysis– Image -> Numbers
• Generic Library– C++– Templates– Source code– Independence of type
• High Performance– no penalty for generic code
• Portability– clean source code for easy portability
29.04.2010
www.ngi-central.org 12
Directory Structure• Code (include/)
• Third party code (toolkits/)
• Documentation (book/, reference/, presentations/)
• Sample code (samples/) and sample images (images/)
• Automated tests (tests/)29.04.2010
www.ngi-central.org 13
NGI - Tests
• Unit Tests– Features of a class/function
are tested in isolation
• Regression Tests– Outcome of a function is tested for regressions– Image/Text comparisons (now == previous)
• Benchmarks– Measured in clocks per pixel
29.04.2010
www.ngi-central.org 14
NGI - Documentation
• User Documentation– NGI Book
• Reference Documentation– Built with Doxygen from source code (HTML)– Always up-to-date
29.04.2010
www.ngi-central.org 15
Third Party Code
• Mandatory• boost (www.boost.org)
• Optional• FreeImage (freeimage.sourceforge.net)• Cairo (www.cairographics.org)
29.04.2010
www.ngi-central.org 16
NGI – Build System• CMake is used as build
engine
• modular– select compiler
(VS2005, VS2008, VS2010, Intel)– run tests– build documentation
• Currently setting up a continuous build machine with various virtual build environments
29.04.2010
www.ngi-central.org 17
Sample Code
• Many code samples • Console samples• MFC samples• .NET samples
• Tests are also samples• Using Boost.Test• Unit tests, benchmarks,
regression tests29.04.2010
www.ngi-central.org 18
NGI – Guidelines
• Header only library• Source code• Heavy use of templates• Concentration on core issues• using other Open Source libraries where
applicable– Boost, FreeImage, Cairo, Qt
29.04.2010
www.ngi-central.org 19
NGI – Concepts• Buffer
– stores data– handles allocation and deallocation– can use external buffers
• View– level of indirection– light, just a pointer and a few pitches– all accesses through views– can handle foreign data buffers– view manipulations
• sub-views (moving_view.exe)• rotated views (dimension_swap_view.exe)• reverse scanning directions (reverse_view.exe)• subsampling (subsampling_view.exe)
29.04.2010
www.ngi-central.org 20
NGI – Functionality Overview (1)
• Basic functions used everywhere else– Interpolation
• linear, spline– Geometry
• point, line_segment, ray, line, triangle, quadrilateral, rectangle, polygon, circle, ellipse
• widgets for all geometric elements– Other examples
• Base64, ZIP/GZIP Compression/Decompression, Fixed point, Matrix solver, solving quadratic and cubic equations, etc.
29.04.2010
www.ngi-central.org 21
NGI – Functionality Overview (2)
• STL like functions– copy, count, equal, find, ...
• Image Processing functions– Point, Filters, Morphology, ...
• Image Analysis functions– Blob, Pattern match, OCR/OCV, ...
• Display Engine– Images, Palettes, Curves, Widgets, Geometry, ...
• Framegrabber/Camera interface
29.04.2010
www.ngi-central.org 22
NGI – STL Like Functions• accumulate• copy• count• equal• fill• find• generate• inner_product• max_element• min_element• replace• transform
29.04.2010
1D2D 3D
www.ngi-central.org 23
NGI – Image Processing Functions (1)
• Point functions (point_operations.exe, profile.exe)• Filter and morphological functions (
neighborhood_operations.exe)– 1D, 2D, 3D– Separable– Fast box filters – independent of kernel size– Many predefined kernels, user can easily add– Framing (framing.exe)– Erode, Dilate, Open, Close, Gradient– Gray and binary behavior
29.04.2010
www.ngi-central.org 24
NGI – Image Processing Functions (2)
• Binning (binning.exe)– binning method selectable/user writable
• Geometric Transformation (resample.exe)– affine / polar– choice of resampling filter (nearest neighbor, box,
triangle, quadratic, cubic, bspline, sinc, kaiser, lanczos)
• Autofocus– based on variance, gradient
29.04.2010
www.ngi-central.org 25
NGI – Image Processing Functions (3)
• Color Processing– Color models (color_transform.exe)
• RGB, CMY(K), HLS, HSI, Lab, Lchab, Luv, Lchuv
– Color twist (color_twist.exe)– Bayer demosaic
• Statistic Functions– min, max, average,
variance, percentiles• Camera Calibration
29.04.2010
www.ngi-central.org 26
NGI – Image Analysis Functions (1)
• Blob analysis– Thresholding via Otsu‘s method– 2D, 3D Labelling– Fläche/Volumen– Chain Code (inner, outer),
perimeter– Moments• binary, gray• ordinary, central, Hu, Flusser
29.04.2010
www.ngi-central.org 27
NGI – Image Analysis Functions (2)
• Pattern matching and searching (search.exe)– gray or color– using pyramid for acceleration
• OCR/OCV– train a character set– read/verify text
29.04.2010
www.ngi-central.org 28
NGI - Acceleration• Written for Multi-Core
– Uses OpenMP– Most functions scale
nicely, can be seen with the benchmarks
• Working on SIMD acceleration– Vector Processing (128 byte chunks)– 16 Pixels at a time– First for point functions
• These two methods are orthogonal
29.04.2010
www.ngi-central.org 29
NGI – Display Engine (1)
• Display Engine– Direct2D, GDI Plus, OpenGL, Cairo– Alpha channel support (blit.exe)– Images (file_access.exe) – Palettes (palette.exe, false_color.exe)– Curves (histogram.exe)
29.04.2010
www.ngi-central.org 30
NGI – Display Engine (2)
– Widgets (scale.exe)
– Hierarchical widget composition(widget_box.exe)
– Callbacks for smart interaction (horizontal_cursor.exe)
29.04.2010
www.ngi-central.org 31
NGI – Framegrabber/Camera
• Grabbing into ringbuffer• asynchronous on different
thread• live/snapshot• pre-trigger/post-trigger• Gen<i>cam support planned
29.04.2010
01
2
3
4
567
8
9
10
11
Acquisition
Display
www.ngi-central.org 32
More Information
29.04.2010
www.ngi-central.org