High-Level Computer Vision - Final Project 1

Logistics •  Start date: 13. 06 •  Proposal presentation 5+5min on 20. 06: ‣  5 slides due 20.06 – task / motivation / method / dataset

•  Interim presentation 5+5min on 11. 07: ‣  progress report / problems encountered / feedback

•  Final presentation 15+5min on 25. 07 ‣  Progress and presentation evaluation

•  Written report submitted on 28. 07 ‣  Report evaluation

•  Reports to indicate assignments of each group member •  Honor Code: clearly cite your sources in your code and your

report.

High-Level Computer Vision - Final Project 2

Project goal

•  Choose a dataset and task: ‣  datasets: Caltech4, Caltech101, Buffy Stickmen, HOI,

UKBench, MPII Human Pose, ImageNet etc. ‣  tasks: object class recognition, object detection/localization,

person identification, gender recognition, scene classification

•  Apply methods to the task, present the analysis of your results ‣  necessary simplifications are OK (e.g. additional annotations) ‣  can you think of a new twist to the method?

High-Level Computer Vision - Final Project 3

Project goal

•  Application: ‣  Apply computer vision techniques to a real-world problem.

•  Model: ‣  Build a new model/algorithm or a new variant of existing

models for an existing computer vision task.

•  Apply your methods to the task, present the analysis of your results.

High-Level Computer Vision - Final Project

Proposal Slides Structure •  Slide 1/2 – Task and motivation ‣  Task statement and definitions ‣  Motivation ‣  Related work

•  Slide 3/4 – Models, tools, novelty ‣  Tentative material and methods

-  From coursework, open source and research code

‣  Investigation -  Feature, model, etc.

•  Slide 5 – Analysis ‣  Benchmark

-  Evaluation dataset and metric

High-Level Computer Vision - Final Project

Project report structure

•  Title •  Abstract •  Introduction •  Related work •  Proposed method explained •  Experimental results •  Conclusions and Future work •  References

High-Level Computer Vision - Final Project

Conferences

•  CVPR •  ICCV •  ECCV •  NIPS •  ICRL •  BMVC

•  ACCV •  GCPR

High-Level Computer Vision - Final Project 7

Dataset

•  Caltech-101 ‣  object class recognition ‣  object localization

High-Level Computer Vision - Final Project 8

Dataset

•  Pascal VOC: ‣  Object detection ‣  Segmentation

High-Level Computer Vision - Final Project 9

Dataset

•  RGB-D Object Dataset (www.cs.washington.edu/rgbd-dataset/) ‣  Object instance retrieval ‣  Object category classification

High-Level Computer Vision - Final Project 10

Dataset

•  RGB-D Indoor Scenes Dataset (http://cs.nyu.edu/~silberman/datasets/) ‣  Scene classification ‣  Object detection, recognition, segmentation

High-Level Computer Vision - Final Project 11

Dataset

•  Leeds Sports Pose Dataset ‣  sport scene recognition

High-Level Computer Vision - Final Project

Dataset

•  ImageNet •  SUN Database •  Places Database •  MPII Human Pose •  Microsoft COCO •  Labeled Faces in the Wild

•  …

High-Level Computer Vision - Final Project 13

Dataset

•  Or capture your own ‣  Digital camera, mobile phone, Google glasses.. ‣  Microsoft Kinect ‣  Web / Google image search

High-Level Computer Vision - Final Project 14

Methods

•  Bag-of-Words (BoW) ‣  Tools: VLFeat, SVM, Random Forests

•  Relevant paper ‣  G. Csurka, C. Bray, C. Dance, and L. Fan. Visual

categorization with bags of keypoints. In Workshop on Statistical Learning in Computer Vision (ECCV), 2004.

•  Publicly available SVM implementations ‣  http://svmlight.joachims.org/ ‣  http://www.csie.ntu.edu.tw/~cjlin/liblinear/

High-Level Computer Vision - Final Project 15

Methods

•  Histograms of Oriented Gradients (HOG) ‣  Tools: HOG/SVM, Random Forests

•  Relevant paper ‣  N. Dalal and B. Triggs. Histograms of Oriented Gradients for

Human Detection. In CVPR, 2005.

•  An optimized HOG implementation may be found at ‣  http://www.cs.brown.edu/~pff/latent/

High-Level Computer Vision - Final Project 16

Methods

•  Interest points, vocabulary trees and geometry ‣  Tools:

-  Hessian, Harris detectors

-  SIFT descriptor

-  Vocabulary tree

•  Relevant paper ‣  D. Nister and H. Stewenius. Scalable recognition with a

vocabulary tree. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2006.

High-Level Computer Vision - Final Project 17

Methods

•  Implicit Shape Model (ISM) ‣  Tools: VLFeat

•  Paper: ‣  B. Leibe, A. Leonardis, B. Schiele. Robust Object Detection

with Interleaved Categorization and Segmentation. In Special Issue on Learning for Recognition and Recognition for Learning (IJCV), 2008.

High-Level Computer Vision - Final Project 18

Methods

•  Image (IS) and video segmentation (VS) ‣  Tools:

-  IS (http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/resources.html)

-  VS (http://www.d2.mpi-inf.mpg.de/content/video-segmentation-superpixels-0)

-  Interactive (GrabCut)

‣  Papers -  P. Arbelaez, M. Maire, C. Fowlkes and J. Malik. Contour Detection and

Hierarchical Image Segmentation. IEEE TPAMI 2011.

-  F. Galasso, R. Cipolla and B. Schiele. Video Segmentation with Superpixels. In ACCV 2012

-  C. Rother, V. Kolmogorov and A. Blake. GrabCut. In SIGGRAPH 2004

High-Level Computer Vision - Final Project 19

Methods

•  Convolutional Neural Networks ‣  Tools:

-  Caffe: http://caffe.berkeleyvision.org/

-  Torch: http://torch.ch/

-  Theano: http://deeplearning.net/software/theano/

‣  Papers -  R. Girshick et al. Rich feature hierarchies for accurate object detection and

semantic segmentation. CVPR 14.

-  …

High-Level Computer Vision - Final Project 20

Features

•  RGB, RG color histograms •  Harris, Hessian, Laplacian, DoG keypoints •  SIFT descriptors •  HOG descriptors •  CNN features •  …

High-Level Computer Vision - Final Project 21

For a good project 5 W’s • What? (a problem) • Why? (motivation)

• How? (proposed strategy) • Where? (dataset and benchmark)

• Who? (team assignments)

It is recommended

•  Baseline

It is desired.. your considerations on

•  Influence of parameter and dataset choice •  Results: what is expected and what is surprising.. not just numbers!

Observations must be substantiated by results or references

High-Level Computer Vision - Final Project 22

Example Projects •  Gender Recognition •  Age Recognition •  Good vs. bad fruit recognition

•  other recognition tasks.... •  Object recognition or detection with the Kinect (RGB + Depth)

•  Image retrieval for 3D objects •  Image retrieval, keypoints and invariance •  Object retrieval in videos, on a mobile phone

•  Person identification •  Image and video segmentation •  Detection and segmentation

•  Tracking

High-Level Computer Vision - Final Project

Proposal Slides Structure - 20.06 •  Slide 1/2 – Task and motivation ‣  Task statement and definitions ‣  Motivation ‣  Related work

•  Slide 3/4 – Models, tools, novelty ‣  Tentative material and methods

-  From coursework, open source and research code

‣  Investigation -  Feature, model, etc.

•  Slide 5 – Analysis ‣  Benchmark

-  Evaluation dataset and metric

High-Level Computer Vision - Final Project

•  Questions?

24