On Universal Restart Strategies for Backtracking Search

An Extensive Empirical Evaluation of Focus Measures for Digital PhotographyHashim Mir, Peter Xu, and Peter van BeekUniversity of Waterloo

Digital cameras are equipped with contrast-based autofocusContrast-detection autofocus makes use of a focus measuremaps an image to a value that represents degree of focus of imageSpeed and accuracy of autofocus crucial to user satisfactionsearch for an image with maximal focus according to focus measurechoice of focus measure is an important factor in performance of autofocusFebruary 4, 2014Focus Measures for Digital Photography2IntroductionPrevious studies on focus measures:small number of benchmarks images in their evaluation,directed at microscopy and not digital cameras, orbased on ad hoc evaluation criteriaHere: extensive empirical evaluation of focus measures for digital photographyadvocate using three standard statistical measures of performanceprecision, recall, and mean absolute erroras evaluation criteriaFebruary 4, 2014Focus Measures for Digital Photography3IntroductionIntroductionFocus MeasuresRelated WorkExperimental MethodologyExperimental ResultsConclusionFebruary 4, 2014Focus Measures for Digital Photography4

OutlineMany focus measures have been proposed in the literatureLet f (x, y) be the luminance or grayscale at pixel (x, y) in an image of size M N pixelsExample: squared gradient focus measure for an image acquired when the lens is at position p is given by,February 4, 2014Focus Measures for Digital Photography5Focus measures

(a) Focus measures of images at each lens position; (b) flower in focus

February 4, 2014Focus Measures for Digital Photography6(a)(b)(a) Focus measures of images at each lens position; (b) fern & grasses in focus

February 4, 2014Focus Measures for Digital Photography7

(a)(b)We evaluated more than 30 focus measures in our studyThe focus measures can be classified under six types:first-order differentiationsecond-order differentiationimage histogramimage statisticscorrelationdata compressionFebruary 4, 2014Focus Measures for Digital Photography8Focus measures: CategorizingExample: squared gradientExample: focus measure based on 33 Sobel operators,February 4, 2014Focus Measures for Digital Photography9Focus measures: First-order differentiation

(horizontal)(vertical)Example: focus measure based on 33 Laplacian operator,February 4, 2014Focus Measures for Digital Photography10Focus measures: Second-order differentiation

Example: variance focus measure is given by,February 4, 2014Focus Measures for Digital Photography11Focus measures: Image statistics

where is the mean luminance of the imageExample: Vollaths F4 focus measure is given by,February 4, 2014Focus Measures for Digital Photography12Focus measures: Correlation

February 4, 2014Focus Measures for Digital Photography13Related work: Evaluating focus measuressettingcitationconclusionsMicroscopy applicationsGroen, Young & Ligthart, 1985squared gradientFirestone et al. 1991vertical Brenner, varianceSantos et al. 1997Vollath's F4, F5Liu, Wang & Sun 2007varianceDigital photographySubbarao and Tyan 1998Laplacian Ng, Neow & Ang 2001first-order 33 Sobel, varianceShih 2007first-order 33 Sobel & PrewittYousefi et al. 2011Vollath's F4Step 1: Implemented a camera remote control application

Step 2: Using app, gathered 25 sets of benchmark images cover a range of common photography settings 4,303 jpeg images (23 167 + 2 231, one for each focus position)Canon EOS 550D/Rebel T2i cameraFebruary 4, 2014Focus Measures for Digital Photography14Experimental methodology: Comparative evaluation of focus measures for digital photography

Step 3: Determined ground truthtrue lens position (or image) where an object is in best focusStep 4: Implemented more than 30 focus measuresextensive survey of the literature Step 5: Given a benchmark set of images and a focus measureapplied focus measure to each jpeg image in benchmark focus measure predicts where an object will be in best focusgiven predictions of focus measure and ground truth, calculated precision, recall, and mean absolute errorFebruary 4, 2014Focus Measures for Digital Photography15Experimental methodology: Comparative evaluation of focus measures for digital photographyground truthtruefalsepredictedtruetp(true positive)fp(false positive)falsefn(false negative)tn(true negative)February 4, 2014Focus Measures for Digital Photography16Experimental methodology: Precision and recall

February 4, 2014Focus Measures for Digital Photography17

squared gradientLaplacianFebruary 4, 2014Focus Measures for Digital Photography18

squared gradientLaplacian

MAE chosen over mean squared error MAE prefers errors that are occasionally large but small on averagemean square error prefers errors that avoid large errors while still possibly being quite unsatisfactory overallFebruary 4, 2014Focus Measures for Digital Photography19Experimental methodology: Mean absolute error (MAE)

absolute error = where p is lens position of image in maximal focusis lens position that focus measure predicts

Focus measureprecisionrecallMAEBrenner 100.0099.000.00squared gradient100.0099.000.003 3 difference98.0097.000.023 3 Sobel98.0097.000.023 3 Scharr 98.0097.000.023 3 Roberts98.0097.000.023 3 Prewitt98.0097.000.02Gaussian 95.0092.000.05vertical sq. gradient91.3392.000.55vertical Brenner 91.3390.000.23February 4, 2014Focus Measures for Digital Photography20Experimental results: First-order differentiationFocus measureprecisionrecallMAELaplacian of Gaussian98.0099.000.325 5 vertical Sobel96.6799.000.655 5 Laplacian96.6799.000.465 5 cross Sobel90.3393.000.663 3 cross Sobel89.6793.001.183 3 vertical Sobel88.7396.003.943 3 Laplacian 85.0091.001.415 5 horizontal Sobel83.3388.001.083 3 horizontal Sobel73.2778.002.95February 4, 2014Focus Measures for Digital Photography21Experimental results : Second-order differentiationFocus measureprecisionrecallMAEnormalized variance43.5753.0026.36variance32.3335.0020.80threshold pixel count4.2418.0052.38threshold content2.134.0036.31power0.532.0043.80February 4, 2014Focus Measures for Digital Photography22Experimental results: Image statisticsFocus measureprecisionrecallMAEautocorrelation80.4099.0026.87Vollath's F461.7487.0024.60Vollath's F533.0035.0019.00February 4, 2014Focus Measures for Digital Photography23Experimental results: CorrelationResults:some popular focus measures perform poorly when applied to autofocusing in digital photography or to more representative sets of imagessimple focus measures based on taking the first derivative of an image perform exceedingly well in digital photography, although care must be taken in which direction derivative is takenApril 8th 2013Combinatorial Problems in Compiler Optimization24Experimental results: DiscussionThe speed and accuracy of a digital camera's contrast-based autofocus algorithm are crucial to user satisfactionWe advocate using three standard statistical measures of performanceprecision, recall, and mean absolute erroras accuracy criteriaOur experimental results indicate that two simple focus measures based on taking first derivative of an image perform exceedingly well in digital photographyFebruary 4, 2014Focus Measures for Digital Photography25Conclusion