Upload File

real%time)detec-on)and)tracking))...

  • Home
  • Documents
  • Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)
25
RealTime Detec-on and Tracking for Augmented Reality on Mobile Phones D. Wagner, A. Mulloni and D. Schmals-eg Presented by Yuansi Chen and Lingqi Yan

Upload: others

Post on 25-May-2020

6 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Real-­‐Time  Detec-on  and  Tracking    for  Augmented  Reality  on  Mobile  Phones    

 D.  Wagner,  A.  Mulloni  and  D.  Schmals-eg  

Presented  by    Yuansi  Chen  and  Lingqi  Yan  

Page 2: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Outline  

1. Mo-va-on  and  Related  Work  2. Modified  Features  Detectors  3.  Performance  and  Analysis  

Page 3: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Mo-va-on  

Page 4: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Mo-va-on  

•  Limited  computa-onal  resources  (speed  and  memory)  on  Mobile  devices  

•  Natural  feature  tracking  infeasible:  SIFT  and  Ferns  

Page 5: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Goal  

•  Enough  speed  improvement  for  real-­‐-me  AR  processing    

•  with  Limited  memory    •  without  losing  too  much  quality  •  on  real  phones  (<33ms/frame)  

Page 6: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Related  Works  

•  General  Feature  Detectors  for  PCs  (slow)  •  Outsource  the  tracking  task  to  PCs  via  wifi.  (AR-­‐PDA  project:  10s  per  frame  is  s-ll  slow)  

•  Marker  tracking:  restricted  applica-ons  

Page 7: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Detec-on  and  Tracking  Rou-ne  

Keypoint  Dectec-on  

Feature  Matching  

Outliner  Removal  

Pose  Es-ma-on  and  Refinement  

Recogni-on  

Tracking  

Local-­‐Search-­‐Based  Tracker    

Ac-vate  if  target  is  found  and  sure  

Ac-vate  when  lost  

Page 8: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Scale  Invariant  Feature  Transform  (SIFT)  

Page 9: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Ferns  

•  Feature  detec-on  as  classifica-on  •  Binary  Feature  F(p)  •  C  =  argmax  P(Ci|F)  •  Instead  of  storing  full  joint  distribu-on,  add  independence:    P(F|C)  =  ΠP(F_S|C)  

 

Page 10: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

FAST  Corner  Detector  

Ref  from:  <Machine  learning  for  high-­‐speed  corner  detec-on>  By  Edward  Rosten  and  Tom  Drummond,  University  of  Cambridge  •  Features  from  accelerated  segment  test  (FAST)  •  A  corner  detector  many    -mes  faster  than  DoG  but    not  very  robust  to  the    presence  of  noise  •  Based  on  intensity  level  tests  

Page 11: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

FAST  Corner  Detector  

Page 12: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

SIFT  to  PhonySIFT  

Main  Modifica-ons:  •  Uses  FAST  corner  detector  to  all  scaled  images  to  detect  feature  points  instead  of  scale-­‐crossing  DoG  

•  Only  3x3  subregions,  4bins  each  ,  creates  36-­‐d  vector  •  Using  a  Spill  tree    

Page 13: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Ferns  to  PhonyFerns  

Main  Modifica-ons  •  Uses  FAST  detector  to  increase  detec-on  speed  •  Reduces  each  ferns  size  •  Uses  8-­‐bit  size  to  store  probability  instead  of  using  4  bytes  float  point  value  

•  modifying  the  training  scheme  to  use  all  FAST  responses  within  the  8-­‐neighborhood  

Page 14: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Outliner  Removal  

•  Orienta-on  Es-ma-on    •  Homography  verifica-on  based  on  RANSAC/PROSAC  

Page 15: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

PatchTracker  

Ideas:      1.  Both  the  scene  and  the  camera  pose  change  

only  slightly  between  two  successive  frames  2.  New  feature  posi-ons  can  be  successfully  

predicted  by  old  one  with  defined  range  search  

Page 16: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Combined  Tracking  

Page 17: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Performance  &  Analysis  

•  Plaiorm:Asus  P552W  (Cellphone)  – 624Mhz  CPU  – 240x320  screen  resolu-on  – No  float  point  unit  – No  3D  accelera-on  

•  Plaiorm:Dell  Notebook  (PC)  – 2.5Ghz  ,  limited  to  use  single  core  – With  float  point  support  

Page 18: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Speed  

Page 19: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Robustness  over  different  objects  

Page 20: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Robustness  over  different  objects  

Page 21: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Typical  Situa-ons  of  Switch  

Page 22: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Typical  Situa-ons  of  Switch  

•  (Show  paper  Figure  7)  

Page 23: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Ferns  vs  SIFT  vs  PatchTracker  

Page 24: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Detailed  Speed  Analysis  

•  PhonySIFT:  –  Corner  Detec-on(FAST)                                  :    ~14%              –  Feature  descriptor  and  Matching  :    ~74%  – Outlier  Removal                                                            :    ~  9%  –  Pose  Refinement                                                          :    ~  3  %  

•  PhonyFerns:  –  Corner  detec-on(FAST)                                    :  ~  22%  –  Second  Octave  and  Blurring                    :  ~  17%  –  Classifica-on                                                                        :  ~    59%  – Outlier  Removal                                                            :  ~  2%  

Page 25: Real%Time)Detec-on)and)Tracking)) …ee225b/sp14/lectures/Presentation_AR.pdfReal%Time)Detec-on)and)Tracking)) for)Augmented)Reality)on)Mobile)Phones))) D.)Wagner,)A.)Mulloni)and)D.)Schmals-eg)

Conclusion  

•  Successfully  worked  with  tracking  system  on  phones  

•  Beper  CPU  would  come  out  in  the  future.  The  choice  of  the  next  genera-on  feature  is  unknown  


Document Title illwerke vkw - DETEC

-T - users.encs.concordia.cai-sip/Docs/Pres/MWSCAS_2017... · of an Event-Based surveillance system for real-time detec-tion, recognition, and tracking of a Person-of-Interest (PoI)

communication DETEC Office fédéral de l¶environnement OFEV

User Req Mold Detec

Detec%ng(and(Defending(Against Third2Party(Tracking… · Detec%ng(and(Defending(Against Third2Party(Tracking(on(the(Web(Franziska)Roesner, Tadayoshi(Kohno,(David(Wetherall(April(26,2012

Detec Cancer

deTec C4P-PA0603S001, Produktdatenblatt

ferramentas para detec de problemas em redes

detec - SICK Germany | SICK

AselfcalibrangreceiverforEOR detec-on - caastro.org · AselfcalibrangreceiverforEOR detec-on ... [email protected]" 24Aug2012" CSIROASTRONOMYANDSPACESCIENCE(CASS)" • Alex"Dunning

(발제)Handheld Augmented Reality Indoor Navigation with Activity-based Instructions+Mobile HCI 2011-Alessangdro Mulloni /이재홍 x2012 spring

FAAC City folder spa - Du-DETEC

deTec C4P-EA07531C00, Tuotetietolehtinen

Lecture9 Detec Tors

Handheld Augmented Reality Indoor Navigation with … · Handheld Augmented Reality Indoor Navigation with Activity-Based Instructions Alessandro Mulloni, Hartmut Seichter, Dieter

deTec | 8019085 - SICK

Det-tronics detec de gas.pdf

Ueda2015 early detec of dm dr.ihab salem

Protein Detec tion by SDS-PAGE & Western Blot

deTec | 8015525 - SICK

Detec Ńia unor markeri bacterieni prin spectrometrie de ...enviro.ubbcluj.ro/site enviro maghiara/studenti/inf doctorats/Ratiu... · 3 Abstract . Motiva Ńia alegerii temei „Detec

Fire Related Crime Investigation and Detec. Course

Fast data acquisition for silicon tracking detectors …...This thesis covers high-speed data acquisition for silicon tracking detec-tors in the context of the Mu3e experiment and

Annual Report 2002 Rheinmetall DeTec AGir.rheinmetall.com/download/companies/rheinmetall/Annual Reports/… · Rheinmetall DeTec at a glance 1) German Commercial Code as primary accounting

Document Title prego services - DETEC

REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,

¿SON LOS PADRES Y PROFESORES BUENOS DETEC- TORES DE …

deTec HG, 8023148 - SICK

Detec;ng Cars Using Gaussian Mixture Modelspeople.isasi.cnr.it/~p.mazzeo/downloads-3/files/DetectingCarsWithMoG.pdf · Detec;ng cars using Gaussian mixture model This example shows

Detec apostila complementar

12.13.01 - Performance Requirement for Combustible Gas Detec

Stochastic Petri Net Analysis of Deadlock Detec- tion

AI-powered Image Processing SoC - iCatch Technology · Edge enhancement over-shoot control Mo on-based object tracking engine Face detec on and tracking engine Processor Cores Arm

Sikkerhet detec

SlidesbyJeffreyD.Ullman · Highdim. data Locality) sensive) hashing) Clustering) Dimensional ity) reducon) Graph’’ data PageRank,) SimRank) Community) Detec-on) Spam) Detec-on)