dip intro

8/12/2019 DIP Intro

http://slidepdf.com/reader/full/dip-intro 1/63

Digital Image Processing 1

Digital Image Processing

Introduction

Object tracking

& Motion detection

in video sequenceshttp://cmp.felk.cvut.cz/~hlavac/TeachPresEn/17CompVision3D/1!ma"e#otion.p$f

%ecomen$e$ link:

8/12/2019 DIP Intro



8/12/2019 DIP Intro



DYNMI! "!#N# N$Y"I"

The input to the $'namic scene

anal'sis is a se(uence of ima"e

frames F ) x, y, t * taken from the

chan"in" +orl$.

x, y are spatial coor$inates.

,rames are usuall' capture$ at

fi-e$ time intervals.

t represents t th frame in the

se(uence.

8/12/2019 DIP Intro


Digital Image Processing %

Motion detection. ften from a static camera. Common in

surveillance s'stems. ften performe$ on the pi-el level onl'

)$ue to spee$ constraints*.

Object localization. ,ocuses attention to a re"ion of interest inthe ima"e. Data re$uction. ften onl' interest points foun$

+hich are use$ later to solve the correspon$ence pro0lem.

Motion segmentation !ma"es are se"mente$ into re"ion

correspon$in" to $ifferent movin" o0ects.

Three-dimensional shape from motion. Calle$ also structure

from motion. 2imilar pro0lems as in stereo vision.

Object tracking. sparse set of features is often tracke$4 e.".4

corner points.

'(ical ((likations

8/12/2019 DIP Intro


Digital Image Processing )

Pedestrians in underground

station

8/12/2019 DIP Intro


Digital Image Processing *

ssumin" that the scene illumination $oes not

chan"e4 the ima"e chan"es are $ue to relative

motion 0et+een the scene o0ects an$ thecamera )o0server*.

There are three possi0ilities:

• 2tationar' camera4 movin" o0ects.• #ovin" camera4 stationar' o0ects.

• #ovin" camera4 movin" o0ects.

Problem de+inition

8/12/2019 DIP Intro


Digital Image Processing ,

Motion +ield

5#otion fiel$ is a 6D representation in the ima"e planeof a )"enerall'* 3D motion of points in the scene)t'picall' on surfaces of o0ects*.

5To the each point is assigned a velocity vectorcorresponding to the motion direction and velocitymagnitude.

8/12/2019 DIP Intro


Digital Image Processing -

Motion +ield

#.amle/

8/12/2019 DIP Intro



#stimating motion +ield

1. #atchin" of 0locks.

6. #atchin" of o0ects. !nterpretation nee$e$. sparsemotion fiel$.

3. #atchin" of interest points. 0it $enser motion fiel$. more $ifficult interpretation. Pro0lems +ith repetitivestructures.

. ptic flo+4 a $ifferential techni(ue matchin" intensities

on a pi-ellevel4 e-plorin" spatial an$ time variations.!$eall'4 a $ense motion fiel$. Pro0lems for te-turelessparts of ima"es $ue to aperture pro0lem )to 0ee-plaine$*.

8/12/2019 DIP Intro


Digital Image Processing

"egmentation in ideosequences

ack"roun$ 2u0traction

first must 0e $efine$ a mo$el of the

0ack"roun$ this background model is compare$ a"ainst

the current ima"e an$ then the kno+n0ack"roun$ parts are su0tracte$ a+a'.

The o0ects left after su0traction arepresuma0l' ne+ fore"roun$ o0ects.

8/12/2019 DIP Intro



#stablising a 4e+erence

Images#stablising a 4e+erence Images/

5 Di++erence 6ill erase static object/

7 8en a d'namic object move out com(letel' +rom its

(osition9 te back ground in re(laced:

8/12/2019 DIP Intro



Object tracking

0ect trackin" is the pro0lem of $eterminin")estimatin"* the positions an$ other relevantinformation of movin" o0ects in ima"e se(uences.

T+oframe trackin" can 0e accomplishe$ usin": correlation0ase$ matchin" metho$s

feature0ase$ metho$s optical flo+ techni(ues

chan"e0ase$ movin" o0ect $etection metho$s

8/12/2019 DIP Intro



Main di++iculties in reliabletracking o+ moving objects

%api$ appearance chan"es cause$ 0'

ima"e noise4

illumination chan"es4 nonri"i$ motion4

...

8onsta0le 0ack"roun$

!nteraction 0et+een multiple multiple o0ects

...

8/12/2019 DIP Intro


Digital Image Processing 1%

;lock matcing metod

Correlation0ase$ trackin" is # metho$

,or a "iven re"ion in one frame4 fin$ the

correspon$in" re"ion in the ne-t frame 0'fin$in" the ma-imum correlation score )orother 0lock matchin" criteria* in a searchre"ion

locks are t'picall' s(uare$ an$ overlappe$

8/12/2019 DIP Intro


Digital Image Processing 1)

;lock matcing metod

8/12/2019 DIP Intro


Digital Image Processing 1*

isible Motion and rue Motion

9enerall'4 opticalflo+ correspon$s tothe motion fiel$4 0ut

not al+a's. ,or e-ample4 the

motion fiel$ an$optical flo+ of a

rotatin" 0ar0erspole are $ifferent...

8/12/2019 DIP Intro


8/12/2019 DIP Intro


Digital Image Processing 1-

O(tical +lo6

OPTIC FO! " apparent motion of the same #similar$intensit% patterns

!$eall'4 the optic flo+ can appro-imate proections of the

three$imensional motion )i.e.4 velocit'* vectors onto theima"e plane.

=ence4 the optic flo+ is estimate$ instea$ of the motionfiel$ )since the latter is uno0serva0le in "eneral*.

8/12/2019 DIP Intro



(erture (roblem

' anal'zin" proecte$ ima"es4 +e al+a's o0serve alimite$ area of visi0le motion onl' )$efine$ 0' apertureof the camera or of the area of influence of the use$

al"orithm for motion anal'sis*.

8/12/2019 DIP Intro


Digital Image Processing 2=

(erture (roblem e.am(le

5 ,or e-ample4 assume that +e either onl' see theinner rectan"les4 or the +hole three ima"es taken attimes t4 t > 14 t > 6. ,or the inner rectan"les4 +e ma'conclu$e an up+ar$ translation +ith a minor rotation....

8/12/2019 DIP Intro



(erture (roblem

+e are onl' a0le tomeasure thecomponent of optical

flo+ that is in the$irection of the intensit'"ra$ient. ?e areuna0le to measure the

component tan"entialto the intensit' "ra$ient.

8/12/2019 DIP Intro



O(tical +lo6

tar"et features aretracke$ over timean$ their movement

is converte$ intovelocit' vectors)upper ri"ht*@

lo+er panels sho+ asin"le ima"e of thehall+a' )left *an$flo+ vectors )ri"ht* asthe camera moves$o+n the hall

)ori"inal ima"es courtes' of AeanBves ou"uet*

8/12/2019 DIP Intro



O(tical +lo6

Common ass&mption -Common ass&mption - Brightness constancy Brightness constancy ''

The appearance of the image patchesThe appearance of the image patches

do not change #brightness constanc%$do not change #brightness constanc%$

)1,(),( ++= t v p I t p I iii

pi-el from the ima"e of an o0ect in the scene $oes not

chan"e in appearance as it )possi0l'* moves from frame

to frame. ,or "ra'scale ima"es this means +e assume

that the 0ri"htness of a pi-el $oes not chan"e as it is

tracke$ from frame to frame.

8/12/2019 DIP Intro



O(tical >lo6 ssum(tions/

;rigtness constanc'

8/12/2019 DIP Intro




8/12/2019 DIP Intro




em(oral (ersistence

Temporal persistence or “small movements” . The ima"e motion of asurface patch chan"es slo+l' in time. !n practice4 this means thetemporal increments are fast enou"h relative to the scale of motionin the ima"e that the o0ect $oes not move much from frame to

frame.

8/12/2019 DIP Intro


Digital Image Processing 2,

O(tical >lo6/ 1D !ase

ri"htness Constanc' ssumption:

)),(()),(()( dt t dt t x I t t x I t f ++=≡

0)( =∂

∂

+

∂

∂

∂

∂

t xt t

I

t

x

x

I

x v t

x

t

I

I v =⇒

{0

)(=

∂

∂

t

x f ecause no chan"e in 0ri"htness +ith time

8/12/2019 DIP Intro



v

x I

(patial derivative(patial derivative

Temporal derivativeTemporal derivativet I

racking in te 1D case/

x

),( t x I )1,( +t x I

p

t

x x

I I

∂

∂=

p x

t t

I I

=

∂

∂=

x

t

I

I v −≈

)ss&mptions')ss&mptions'

5 *rightness constanc%*rightness constanc%5 (mall motion(mall motion

8/12/2019 DIP Intro



racking in te 1D case/

x

),( t x I )1,( +t x I

p

x I

t I

Temporal derivative at +Temporal derivative at +ndnd iterationiteration

Iterating helps refining the velocit% vector Iterating helps refining the velocit% vector

Can keep the same estimate for spatial derivativeCan keep the same estimate for spatial derivative

x

t previous

I

I vv −←

Converges in abo&t , iterationsConverges in abo&t , iterations

8/12/2019 DIP Intro


Digital Image Processing 3=

>rom 1D to 2D tracking

The Math is ver% similar'The Math is ver% similar'

v

x

),( t x I )1,( +t x I

1v

2v

3v

4v

x

y)1,,( +t y x I

),,( t y x I

x

t

I I v −≈

bGv 1−−≈

∑

=

p y y x

y x x

I I I

I I I G

aroundwindow2

2

∑

=

p t y

t x

I I

I I b

aroundwindow

8/12/2019 DIP Intro



?$ in P'ramids

Pro0lem:

+e +ant a lar"e +in$o+ to catch lar"e motions4

0ut a lar"e +in$o+ too oft en 0reaks the coherent

motion assumption

To circumvent this pro0lem4 +e can track first over

lar"er spatial scales usin" an ima"e p'rami$ an$

then refine the initial motion velocit' assumptions

0' +orkin" our +a' $o+n the levels of the ima"e

p'rami$ until +e arrive at the ra+ ima"e pi-els.

8/12/2019 DIP Intro


! t +i ti l +l

8/12/2019 DIP Intro


image Iimage J

Gaussian pyramid of image It-1 Gaussian pyramid of image I

image Iimage It-1

!oarseto+ine o(tical +lo6estimation

run iterative

run iterative

+arp F upsample

.

.

.

8/12/2019 DIP Intro



#dge

; lar"e "ra$ients4 all the same

; lar"e λ14 small λ6

G ,rom hurram =assan2hafi(ue CPH1H Computer Vision 6II3

8/12/2019 DIP Intro



$o6 te.ture region

; "ra$ients have small ma"nitu$e

; small λ14 small λ6


8/12/2019 DIP Intro



@ig te.tured region

; "ra$ients are $ifferent4 lar"e ma"nitu$es

; lar"e λ14 lar"e λ6


8/12/2019 DIP Intro


Digital Image Processing 3,

$ocal +eatures +or tracking

...There are many kinds of local features that onecan track...

!f stron" $erivatives are o0serve$ in t+oortho"onal $irections then +e can hope that thispoint is more likel' to 0e uni(ue.

<J man' tracka0le features are calle$ corners.!ntuitivel'4 cornersKnot e$"esKare the points

that contain enou"h information to 0e picke$ outfrom one frame to the ne-t.

8/12/2019 DIP Intro



@arris corner detection

The $efinition relies on the matri- of the secon$or$er $erivatives )L6 x 4 L6 y 4 L x Ly * of the ima"eintensities.

=essian matri- aroun$ a

point:

!utocorrelation matrix of the secon$ $erivativeima"es over a small +in$o+ aroun$ each point:

8/12/2019 DIP Intro



@arris corner detection

5

=arrisMs $efinition:

Corners are places in the ima"e +here theautocorrelation matri- of the secon$ $erivatives hast+o lar"e ei"envalues.

<J there is te-ture )or e$"es* "oin" in at least toseparate directions

these t+o ei"envalues $o more than $etermine if apoint is a "oo$ feature to track@ the' also provi$e ani$entif'in" si"nature for the point.

8/12/2019 DIP Intro


Digital Image Processing %=

"i and omasi/Aood >eatures to rack

2hi an$ Tomasi: 9oo$ corner results as lon" as the smaller of

the t+o ei"envalues is "reater than aminimum threshol$.

2hi an$ TomasiMs metho$ +as not onl'sufficient 0ut in man' cases "ave more

satisfactor' results than =arrisMs metho$.

8/12/2019 DIP Intro


Digital Image Processing %1

Alobal a((roac/@orn"cunck

metho$ for fin$in" the optical flo+ pattern +hichassumes that the apparent velocit' of the0ri"htness pattern varies smoothl' almost

ever'+here in the ima"e. n iterative implementation computes the optical

flo+ of each pi-el.

8/12/2019 DIP Intro


8/12/2019 DIP Intro


Digital Image Processing %3

dvantages < Disadvantageso+ te @orn5"cunck algoritm

dvantages / it 'ields a ig densit' o+ +lo6 vectors9 i:e: te +lo6

in+ormation missing in inner (arts o+ omogeneousobjects is +illed in +rom te motion boundaries:

Disadvantages/ it is more sensitive to noise than local metho$s.

The =orn2chunck metho$ is ver' unsta0le in caseof illumination artifacts in ima"e se(uences )$ue tothe assume$ 0ri"htness constanc'*.

8/12/2019 DIP Intro


!om(uter ision >all 2==1Digital

Image Processing %%

Mean"i+t racking

Mean-(hift in tracking task'

track the motion of a cluster of interestin"

features. 1. choose the feature $istri0ution to represent

an o0ect )e.".4 color > te-ture*4

6. start the meanshift +in$o+ over the feature

$istri0ution "enerate$ 0' the o0ect 3. finall' compute the chosen feature

$istri0ution over the ne-t vi$eo frame.

8/12/2019 DIP Intro



Image Processing %)

Mean"i+t racking

2tartin" from the current +in$o+ location4 themeanshift al"orithm +ill fin$ the ne+ peak ormo$e of the feature $istri0ution4 +hich

)presuma0l'* is centere$ over the o0ect thatpro$uce$ the color an$ te-ture in the fi rstplace.

<J !n this +a'4 the meanshift +in$o+ tracksthe movement of the o0ect frame 0' frame.

8/12/2019 DIP Intro



Image Processing %*

Meansi+t algoritm in action

n initial +in$o+ isplace$ over a t+o$imensional arra' of$ata points an$ issuccessivel'recentere$ over themo$e )or local peak*of its $ata $istri0utionuntil conver"ence

8/12/2019 DIP Intro



Image Processing %,

Meansi+t algoritm

The meanshift al"orithm runs as follo+s:

1. Choose a search +in$o+:

5 its initial location@

5 its t'pe )uniform4 pol'nomial4 e-ponential4 or9aussian*@

5 its shape )s'mmetric or ske+e$4 possi0l' rotate$4roun$e$ or rectan"ular*@

5 its size )e-tent at +hich it rolls off or is cut off *.

6. Compute the +in$o+Ms )possi0l' +ei"hte$* center ofmass.

3. Center the +in$o+ at the center of mass.

. %eturn to step 6 until the +in$o+ stops movin" )it al+a's

+ill*.G

8/12/2019 DIP Intro



Image Processing %-

!amsi+t continuousl' a$aptive meanshift

!t $iffers from the meanshift in that the search+in$o+ a$usts itself in size.

8/12/2019 DIP Intro


l

8/12/2019 DIP Intro



Image Processing )=

Motion em(latesObject silouettes/

Object silho&ettes can 0e o0taine$ in a num0er of+a's:

1. use a reasona0l' stationar' camera an$ then

emplo' frametoframe $ifferencin" . Th is +ill "ive'ou the movin" e$"es of o0ects4 +hich is enou"hto make motion templates +ork.

6. Bou can use chroma ke'in". ,or e-ample4 if 'ou

have a kno+n 0ack"roun$4 'ou can simpl' take asfore"roun$ an'thin" that is not 0ack"roun$.

l

8/12/2019 DIP Intro



Image Processing )1

Motion em(latesObject silouettes/

To learn a backgro&nd model from hich %o& can

isolate ne foregro&nd objectspeople as

silho&ettes.

1. Bou can use active silhouettin" techni(uesKfor

e-ample4 creatin" a +all of nearinfrare$ li"ht an$havin" a nearinfrare$sensitive camera look at the

+all. n' intervenin" o0ect +ill sho+ up as a

silhouette.

6. Bou can use thermal ima"es@ then an' hot o0ect)such as a face* can 0e taken as fore"roun$.

3. ,inall'4 'ou can "enerate silhouettes 0' usin" the

se"mentation techni(ues ....

http://+++.'outu0e.com/+atchOv<?n7$vnz0

ki ! H i O i l

8/12/2019 DIP Intro


Digital Image Processing )2

racking !ars Hsing O(tical>lo6 4esults Mat6orks

The mo$el

locates the carsin each 0inar'feature ima"eusin" the lo0 nal'sis 0lock.

The mo$el uses an optical flo+ estimation techni(ue to

estimate the motion vectors in each frame of the vi$eo

se(uence.

' threshol$in" an$ performin" morpholo"ical closin" on themotion vectors4 the mo$el pro$uces 0inar' feature ima"es.

8/12/2019 DIP Intro



!angebased blob tracking

l"orithm

Calculate the 0ack"roun$

!ma"e su0traction to $etection motion 0lo0s

Compute the $ifference ima"e 0et+een t+o frames

Threshol$in" to fin$ the 0lo0s

ocate the 0lo0 center as the position of the o0ect

8/12/2019 DIP Intro


Digital Image Processing )%

#stimation Prediction

!n a lon" ima"e se(uence4 if the $'namics of themovin" o0ect is kno+n4 pre$iction can 0e ma$ea0out the positions of the o0ects in the current

ima"e. This information can 0e com0ine$ +ith theactual ima"e o0servation to achieve more ro0ustresults

8/12/2019 DIP Intro


8/12/2019 DIP Intro


Digital Image Processing )*

Model o+ s'stem

Model o+ s'stem

ki + l t d i t

8/12/2019 DIP Intro


Digital Image Processing ),

racking +ormulated using te@idden Markov model B@MMC

=i$$en #arkov mo$el )=##*:

ki + l t d i t

8/12/2019 DIP Intro


Digital Image Processing )-

racking +ormulated using te@idden Markov model The state xk usuall' consists of the position4 the

velocit'4 the acceleration4 an$ other features of theo0ect

The o0servation zk is usuall' the vi$eo frame atcurrent time instance.

The transition pro0a0ilit' is mo$ele$ usin" $'namicmo$el such as constant velocit' mo$el4 or constantacceleration mo$el

E-ample: constant velocit' $'namic mo$el

8/12/2019 DIP Intro



e d'namic s'stem

8/12/2019 DIP Intro


Digital Image Processing *=

e d'namic s'stem

"eneral pro0lem of tr'in" to estimate the state x of a $iscretetime controlle$process is "overne$ 0' the linear stochastic $ifference e(uation

+ith a measurement z that is:

The ran$om varia0les "k an$ vk represent the process an$ measurement noise

)respectivel'*.

The' are assume$ to 0e in$epen$ent )of each other*4 +hite4 an$ +ith normalpro0a0ilit' $istri0utions

!n practice4 the process noise covariance Q

an$

measurement noise covariance # matrices mi"ht change

"ith each time step or measurement, ho"ever here "e

assume they are constant.

8/12/2019 DIP Intro


Digital Image Processing *1

?alman +ilter

5The alman filter is a set of mathematicale(uations that provi$es an efficientcomputational )recursive* means to estimate the

state of a process4 in a +a' that minimizes themean of the s(uare$ error.

8/12/2019 DIP Intro


Digital Image Processing *2

?alman +ilter

T+o "roups of the e(uations for the alman filter:

time update e(uations

an$ measurement update e(uations.

The time up$ate e(uations are responsi0le for proectin" for+ar$ )in time* thecurrent state an$ error covariance estimates to o0tain the a priori estimates for

the ne-t time step.

The measurement up$ate e(uations are responsi0le for the fee$0ackKi.e. for

incorporatin" a ne+ measurement into the a priori estimate to o0tain an

improve$ a posteriori estimate.

8/12/2019 DIP Intro


?alman +ilter

the time up$ate e(uations can also 0e thou"ht of as predictor e(uations

the measurement up$ate e(uations can 0e thou"ht of as correctore(uations.

predictor$corrector al"orithm <J solvin" numerical pro0lems:

Discrete alman filter

time up$ate e(uations:

Discrete alman filter

measurement up$ate e(uations:

time update equations project the state and covariance estimates

forward from time step k-1 to step k .

dip intro

Documents