relational dynamic bayesian networks to improve multi-target tracking. cristina manfredotti and enza...

Download Relational Dynamic Bayesian Networks to improve Multi-Target Tracking. Cristina Manfredotti and Enza Messina DISCo, University of Milano-Bicocca

Post on 31-Mar-2015

216 views

Category:

Documents

2 download

Embed Size (px)

TRANSCRIPT

  • Slide 1

Relational Dynamic Bayesian Networks to improve Multi-Target Tracking. Cristina Manfredotti and Enza Messina DISCo, University of Milano-Bicocca Slide 2 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 2 Relations to improve tracking Slide 3 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 3 Complex activity recognition Y.Ke, R.Sukthankar, M.Hebert; Event Detection in Crowed Videos Slide 4 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 4 Objectives Goals: 1.To model relations and 2.To maintain beliefs over particular relations between objects In order to simultaneously: Improve tracking with informed predictions and Identify complex activities based on observations and prior knowledge Slide 5 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 5 Relational Domain Relational Domain: set of objects characterized by attributes 1 and with relations 1 between them Car Id color position(t) velocity(t) direction(t) DecreasingVelocity(t) A SameDirection(t) distance(t) Before(t) Car B Id color position(t) velocity(t) direction(t) DecreasingVelocity(t) SameDirection(t) distance(t) Before(t) 1 Attributes and relations are predicate in FOL. Slide 6 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 6 Relational State The State of a Relational Domain is the set of the predicates that are true in the Domain. Relational state State of attributes State of relations Slide 7 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 7 Relational Bayesian Networks: Uncertainty in a Relational Domain Relational (Dynamic) Bayesian Networks Syntax RBN: a set of nodes, one for each variable a directed, acyclic graph a conditional distribution for each node given its parents This distribution must take into account the actual complexity of the nodes! Syntax RBN: a set of nodes, one for each predicate a directed, graph a conditional distribution for each node given its parents Slide 8 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 8 Dynamics The State of a Relational Domain is the set of the predicates that are true in the Domain. State evolves with time We extend a RBN to a RDBN as we are used to extend a BN to a DBN. Slide 9 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 9 Inference Markov assumption and Conditional independence of data on state. bel(s t ) = p(z t |s t ) s p(s t |s t-1 )bel(s t-1 )ds t-1 Bayesian Filter The problem of computing: bel(s t ) = p(s t |z 1:t ) Slide 10 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 10 Inference Relations in the State result in correlating the State of different objects between them p(x t-1 |z 1:t-1 )p(x t |z 1:t-1 )p(x t |z 1:t ) Bel(x t-1 ) Bel(x t ) Transition model Sensor model t = t+1 Slide 11 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 11 Sensor model (1 st assumption) part of the state relative to relations, s r, not directly observable p(z t |s t ) = p(z t |s a t ) observation z t independent by the relations between objects. Intuitively: Travelling Together vs Being Close Slide 12 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 12 Transition model: a trick p(s t |s t-1 ) = p(s a t,s r t |s a t-1, s r t-1 ) S a t-1 S r t-1 SatSat SrtSrt Intuitive Slide 13 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 13 p(s a t,s r t |s a t-1,s r t-1 )= But s r t independent by s a t-1 given s r t-1 and s a t p(s a t,s r t |s a t-1,s r t-1 ) = p(s a t |s a t-1,s r t-1 ) p(s r t |s r t-1, s a t ) bel(s t ) = p(s t |z 1:t ) = p(s a t,s r t |z 1:t ) bel(s t )=p(z t |s a t,s r t ) s p(s a t,s r t |s a t-1,s r t-1 )bel(s t-1 )ds t-1 p(z t |s a t,s r t ) = p(z t |s a t ) Relational Inference p(s a t |s a t-1,s r t-1 ) p(s r t |s a t-1,s r t-1, s a t ) Transition model (2 nd assumption) Slide 14 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 14 * It is a technique that implements a recursive Bayesian Filter through a Monte Carlo simulation. The key idea is to represent the posterior pdf as a set of samples (particles) paired with weights and to filter the mesurament based on these weights.. Particle Filtering* (general case) Slide 15 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 15 Relational Particle Filter Slide 16 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 16 RPF: extraction X a t,(m) X r t,(m) X a t,(m) ~ p(x a t,(m) |s a t-1,s r t-1 ) X a t,(m) ~ p(x r t,(m) |s a t = x a t,(m),s r t-1 ) X r t,(m) Slide 17 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 17 RPF: weighting The consistency of the probability function ensures the convergence of the algorithm. X a t,(m) X r t,(m) Weight ( ) ~p(z t |x a t ) The weighting step is done according to the attributes part of each particle only, the relational part follows. Slide 18 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 18 Experiments: FOPT Slide 19 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 19 Experiments: Transition Model If relation true If relation false Slide 20 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 20 Experiments: Results Slide 21 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 21 Further experiments Data: 15 simulated objects. From each cell, an object can jump to one of the n next cells where n depends by the cell. Objects can move together. If traveling together, two (or more) objects will always be in cells from which it is possible for one to reach the other or vice-versa. If traveling together, two objects will behave similarly (i.e. if one turns left, the other will follow). Slide 22 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 22 Tracking AND activity Recognition X a t,(m) X r t,(m) X a t,(m) X r t,(m) X a t,(m) X a {t,(m)} X o {t,(m)} X r t,(m) X a t+1,(m) 1 step of sampling: prediction of the state of attributes X a t,(m) X a {t,(m)} X o {t,(m)} X r t,(m) X a t+1,(m) X a {t,(m)} X o {t,(m)} X r t+1,(m) 2 step of sampling: prediction of the state of relations Or activity prediction Slide 23 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 23 step 12 step 24 True Positive Rate False Positive Rate The worst (time step 24) and the best (time step 12) ROC curve for the relation recognition task. Further Results 01 1 Slide 24 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 24 PF: 4.6500 2.2333 3.7333 2.7667 RPF: 4.6000 2.4667 1.3333 2.1000 PF : 4.7000 3.6667 5.6667 2.6000 RPF: 4.6000 3.5333 5.2667 2.5333 Further Results (cont.) Tracking error (distance) for each of the 15 objects. Comparable behaviour of the errors BUT for related objects RPF trackes always better than PF. PF : 4.6667 4.6667 3.8333 1.9333 RPF: 4.7667 2.7667 3.5333 1.5333 PF: 2.0667 5.9000 1.6000 RPF: 2.0333 5.8333 2.2333 Slide 25 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 25 To conclude... Modeling Relations dynamically: To improve multi target tracking To recognize complex activities Inference in Dynamic Relational Domain In theory complex BUT Simplified by smart decomposition of the transition model non-relational sensor model Showed promising results Slide 26 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 26 Related works Complex tracking tasks: Heuristics M.Isard and J. MacCormick BraMBLe, A Bayesian Multi-Blob Tracker. Slide 27 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 27 Related works Complex tracking tasks: Heuristics Mixed-States models Complex activity recognition: Stochastic grammar Free Y.A.Ivanov and A.F.Bobick Recognition of Visual Activities and Interactions by Stochastic Parsing Slide 28 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 28 Related works Complex tracking tasks: Heuristics, Mixed-States models Complex activity recognition: Stochastic grammar Free, First Order Logic S. Tran and L. Davis, Visual Event Modeling and Recognition using Markov Logic Networks Slide 29 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 29 p(x t-1 |z 1:t-1 )p(x t |z 1:t-1 )p(x t |z 1:t ) Bel(x t-1 ) Bel(x t ) Transition model Sensor model t = t+1 ~ Transition model Sensor model Inference Relations in the State result in correlating the State of different objects between them p(x t-1 |z 1:t-1 )p(x t |z 1:t-1 )p(x t |z 1:t ) Bel(x t-1 ) Bel(x t ) Transition model Sensor model t = t+1 Slide 30 C.Manfredotti, E.Messina: R