Finger Detection for Multi-Touch Tabletop Display System

Finger Detection for Multi-Touch Tabletop Display System

Su-ting, Chuang2010/8/21OutlineIntroductionRelated WorkSystem and MethodExperimentConclusion & Future Work22OutlineIntroductionRelated WorkSystem and MethodExperimentsConclusion & Future Work33Introduction44OutlineIntroductionRelated WorkSystem and MethodExperimentsConclusion & Future Work55Related WorkFTIR (Frustrated Total Internal Reflection)

J. Y. Han, Low-cost multi-touch sensing through frustrated total internal reflection," in Proceedings of the 18th annual ACM symposium on User interface software and technology (UIST '05). New York, NY, USA: ACM Press, 2005, pp. 115-118.

66Related WorkDI (Diffused Illumination)J. Rekimoto and N. Matsushita, Perceptual surfaces: Towards a human and object sensitive interactive display," Workshop on Perceptural User Interfaces (PUI'97), 1997.

7An IR camera with IR illuminators to observe hands

7Related WorkTouchLibA multi-touch development kit

Finger detection processing flow chart8

BackgroundSubtractionSimpleHighpassScaleThreshold

8OutlineIntroductionRelated WorkSystem and MethodExperimentsConclusion & Future Work99Hardware Configuration

(2) IR Camera(3) IR Illuminator(1) Peripheral Projector1010Hardware ConfigurationOrder of diffuser layer and touch-glass layer11Diffuser layerIR illuminatorIR cameraspotIR illuminatorIR cameraTouch-glass layerIR cameraspotIR camera

2143The surface of this kind of tabletop system is usually composed of a diffuser layer and a touch-glass layer.11Hardware ConfigurationProblem: IR rays will be reflected by the touch-glass and resulting hot spot regions in camera viewsSolution:Use other cameras to recover the regions which are sheltered by IR spots

12

camera12Software ArchitectureDetection systemImage StitchingFinger DetectionFinger Tracking

Parameter determination13ImageStichingFingerDetectionFinger TrackingImage StitchingCombine multi-camera view into a virtual camera viewFinger DetectionRecognize touched fingertipFinger TrackingSmooth the trajectory of finger and fix lost results13Software ArchitectureDirectShowFilter-based framework GShowGPU-accelerated framework

14

Dshow: [media-streaming] Provide high-quality capture and playback of multimedia streams can perform high-quality video and audio playback or captureIt supports capture from digital and analog devices based on the Windows Driver Model (WDM) or Video for Windows. It automatically detects and uses video and audio acceleration hardware when available, but also supports systems without acceleration hardware.

Gshow: combination of directX and directShow14Image Stitching15ImageBlendingIR CameraIR cameraUndistortionUndistortionHomoWarpHomoWarp

15Image StitchingHomoWarpGoalUnify finger size among different position of table16

16Image StitchingHomoWarpMethodCalculate homography in advance

17521625431436

25Calculate homography between cameras and virtual space in advance17Image StitchingHomoWarpWarp image from each camera to virtual space18

18Image StitchingImage Blending

1919Parameter DeterminationRequirements of ideal finger detection systemSensitive Noise-freeEvaluation of parametersData CollectionDepict traceMeasurementMinimize miss and false alarm20MissFalse Alarmrequirmentsmiss & false alarm20Parameter DeterminationIdeal finger detection Only one fingertip landing on traceContinuity among frames21

Good frameOnly one finger landing on traceContinuity

21Parameter Determination22

22Parameter Determination23Parameters DeterminatorParameter CombinationDetection ResultApplicable set of ParametersTestSetTouch DataGround Truth(Trace)Detection SystemParameters Determinator : generate parameter combinations & evaluate parameter combinations by calculating miss and false alarm

23OutlineIntroductionRelated WorkSystem and MethodExperimentsConclusion & Future Work2424ExperimentsPerformance evaluation25

25ExperimentsParameter determinationDecide parameters in our systemAdopt sampling-based parameter search technique

26NormalizationDifference of GaussianBackground SubtractionBinaryFinger AnalysisSubtract valueSmoothkernelThresholdFingerSize26ExperimentsParameter determinationExhaustive searchParameter combination5 (step) *5 (step) *5 (step) *5 (step) = 625Applicable parameter num16/625 = 2.56%

27Subtract valueSmooth kernelThresholdFinger sizeLow bound051010Step55510High bound1517305027ExperimentsParameter determinationParticle filtering28SamplingMeasure

The key idea is to represent the required posterior function by a set of random samples with associated weights and to compute estimates based on these samples and weights.

28Experiments2929Experiments3030OutlineIntroductionRelated WorkSystem and MethodExperimentsConclusion & Future Work31313232