structured forests for fast edge detection [paper presentation]
TRANSCRIPT
Dollár, Piotr, and C. Lawrence Zitnick. "Structured forests for fast edge detection.“
Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE, 2013.
Main
Contribution
Compute edge maps in realtime,
faster than the competing state-of-the-art
Proposed
Method
Structured Random Forests
This presentation is inspired by the talk: http://techtalks.tv/talks/structured-forest-for-fast-edge-detection/59412/
Edge Definition
Source: http://upload.wikimedia.org/wikipedia/en/8/8e/EdgeDetectionMathematica.png
Edge Definition
Source: http://upload.wikimedia.org/wikipedia/en/8/8e/EdgeDetectionMathematica.png
Where this work excels
A c c u r a c y & S p e e d [ re a l t i m e ]
Where this work excels
A c c u r a c y & S p e e d [ re a l t i m e ]
Where this work excels
A c c u r a c y & S p e e d [ re a l t i m e ]
Where this work excels
A c c u r a c y & S p e e d [ re a l t i m e ]
Edge Detection
as
Classification Problem
Edge Detection as Classification Problem
• {0, 1}
Edge Detection as Classification Problem
• {0, 1}
• Binary classification ignoring the local structures of the edges
Edges have Structures
Clustering Sketch Tokens
Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection, Joseph J. Lim et al. 2013
Random Forests
Random Forests
ℎ 𝑥, 𝜃 = 𝑥 𝑘1 − 𝑥 𝑘2 < 𝜏
Random Forests
Random Forests
Random Forests
Random Forests For Edge Detection
Random Forests For Edge Detection
Random Forests For Edge Detection
Random Forests For Edge Detection
Random Forests For Edge Detection
Random Forests For Edge Detection
Decision:
Structured Random Forests
The Output Space
{0, 1} 2
{ ,….} 151
DimensionalityInput Space
The Output Space
{0, 1} 2
{ ,….} 151
2256
DimensionalityInput Space
Node Split
Low entropy split
Training Model
Bad split
Training Model
Go od split
Training Model
Cluster the
structured labels
Training Model
Just one difference to random forests:
cluster the output into a binary or multiclass output using distance function
Clustering
𝑌: Structured space where information gain not well defined
𝐶: Discrete space where information space is good defined
𝑍: Intermediate space where similarity measurement is easy to compute
Π ∶ 𝑌 → 𝑍 , 𝑍 → 𝐶
Training Model
• Computing information gain
– Labels 𝐶 are discrete, standard entropy criterions used.
• Combining predictions
– To combine 𝑦1… 𝑦𝑛 ∈ 𝑌 into a prediction:
• Compute 𝑧𝑖 = Π𝜑(𝑦𝑖) of dimension 𝑚
• Select 𝑦𝑘 , whose 𝑧𝑘 = 𝑎𝑟𝑔𝑚𝑖𝑛𝑧𝑘 𝑖,𝑗(𝑧𝑘𝑗 − 𝑧𝑖𝑗)2
(medoid)
+ Computing medoids is fast, 𝑂(𝑛𝑚)
Training Structured Forests For
Edge Detection
Training Structured Forests For Edge Detection
32x32 RGB image patch
→ 7228 features
Training Structured Forests For Edge Detection
32x32 RGB image patch
→ 7228 features
Π ∶ 𝑌 → 𝑍
Dimension of 𝑍 = 2562
Down-sampled to m = 256
Training Structured Forests For Edge Detection
32x32 RGB image patch
→ 7228 features
Π ∶ 𝑌 → 𝑍
Dimension of 𝑍 = 2562
Down-sampled to m = 256
Edge Detection with Structured Forests
32x32 RGB image patch
→ 7228 features
Edge Detection with Structured Forests
32x32 RGB image patch
→ 7228 features
𝑌 is a 16x16 segmentation
mask
Multi-scale Detection
Multi-scale Detection
Multi-scale Detection
Results
• BSDS 500 image set
– Multi-scale ties or outperforms the accuracy of the state of the art.
– Single-scale improves runtime by 5x to 10x
Results
• BSDS 500 image set
– Multi-scale ties or outperforms the accuracy of the state of the art.
– Single-scale improves runtime by 5x to 10x
Results
• BSDS 500 image set
– Multi-scale ties or outperforms the accuracy of the state of the art.
– Single-scale improves runtime by 5x to 10x
Results
• NYU image set
– Multi-scale is slightly better than the state of the art.
– Improved performance by multiple orders of magnitude
Conclusions
• Realtime structured learning method for edge detection
• General purpose method for learning structured random forests
• Real time + state of the art accuracy → new applications possible
• Novel learning approach may be applicable to other problems.
T h a n k yo u