Sychronising event photos by approximating a set of pairwise time constraints

Paweł Nowak, Marcus Thaler, Harald Stiegler, Werner BailerMediaEval Workshop, Barcelona, Oct. 2014

2Approach: Determine offsets

determine the visual similarity of image pairs

based on full matching of SIFT descriptors and based on VLAD

GPS is used optionally (if available)

Assuming constant offsets between galleries, a matching pair contributes a constraint δp- δq = τij

Iteratively solve equation system using Jacobi method, and discard up to 10% largest outliers

3Approach: Determine offsets

Gallery 2 Gallery 3 Gallery 4 ... Gallery n

𝛿2

𝛿3

𝛿𝑛

𝛿𝑛−1

¿

. . .

. . .

. . .

. . .

. . .

0

0 0

0 τij

τkl

. . .

-

-

4Challenges of visual matching

5Approach: Cluster sub-events

Initialize from subsequent images with high similarity: oversegmentation

Merge clusters based on visual similarity, weighted by time distance

6Results: Offsets

7Results: Clustering

8Conclusion

Winter Olympics is more challenging

VLAD useful for events clustering, but not discriminative enough for determining offsets

If galleries are aligned, accuracy is quite high

Clustering often oversegments with our methods – not all measures seem to handle this equally well

Dependency of correct segmentation from correct offsets not well reflected in results

The research leading to these results has received funding from the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 610370, ICoSOLE (Immersive Coverage of Spatially Outspread Live Events“ (http://www.icosole.eu/).

Werner BailerJOANNEUM RESEARCH – DIGITAL

werner.bailer@joanneum.athttp://www.joanneum.at/digital