CERTH/CEA LIST at MediaEval Placing Task 2015 Giorgos Kordopatis-Zilos1, Adrian Popescu2, Symeon Papadopoulos1 and Yiannis Kompatsiaris1

1 Information Technologies Institute (ITI), CERTH, Greece

2 CEA LIST, 91190 Gif-sur-Yvette, France

MediaEval 2015 Workshop, Sept. 14-15, 2015, Wurzen, Germany

Summary

#2

Tag-based location estimation (2 runs) • Based on a geographic Language Model • Built upon the scheme of our 2014 participation [2] (Kordopatis-Zilos et

al., MediaEval 2014) • Extensions from [3]: improved feature selection and weighting

(Kordopatis-Zilos et al., PAISI 2015)

Visual-based location estimation (1 run) • Geospatial clustering scheme of the most visually similar images

Hybrid location estimation (2 run) • Combination of the textual and visual approaches

Training sets • Training set released by the organisers (≈4.7M geotagged items) • YFCC dataset, excl. images from users in test set (≈40M geotagged items)

Tag-based location estimation

#3

• Processing steps of the approach – Offline: language model construction – Online: location estimation

Language Model (LM)

#4

Feature Selection and Weighting

#5

accuracy locality

Accuracy

#6

Estimated

Locations

Locality

#7

Locality – value distribution

#8

london (6975), paris (5452), nyc (3917)

luminancehdr (0.0035), dsc6362 (0.003), air photo (0.002)

Extensions

• Spatial Entropy (SE) function – calculate entropy values applying the Shannon entropy formula in the tag-cell

probabilities – build a Gaussian weight function based on the values of the tag SE

#9

• Internal Grid – Built an additional LM using a finer grid, cell side length of 0.001° – combine the MLC of the individual language models

Visual-based location estimation

#10

Hybrid-based location estimation

#11

Confidence

#12

Runs and Results

#13

measure RUN-1 RUN-2 RUN-3 RUN-4 RUN-5

acc(1m) 0.15 0.01 0.15 0.16 0.16

acc(10m) 0.61 0.08 0.62 0.75 0.76

acc(100m) 6.40 1.76 6.52 7.73 7.83

acc(1km) 24.33 5.19 24.61 27.30 27.54

acc(10km) 43.07 7.43 43.41 46.48 46.77

m. error (km) 69 5663 61 24 22

RUN-1: Tag-based location estimation + released training set

RUN-2: Visual-based location estimation + released training set

RUN-3: Hybrid location estimation + released training set

RUN-4: Tag-based location estimation + YFCC dataset

RUN-5: Hybrid location estimation + YFCC dataset

Thank you!

• Code:

https://github.com/MKLab-ITI/multimedia-geotagging

• Get in touch:

@sympapadopoulos / papadop@iti.gr

@georgekordopatis / georgekordopatis@iti.gr

#14

References

#15

[1] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.

[2] G. Kordopatis-Zilos, G. Orfanidis, S. Papadopoulos, and Y. Kompatsiaris. Socialsensor at mediaeval placing task 2014. In MediaEval 2014 Placing Task, 2014.

[3] G. Kordopatis-Zilos, S. Papadopoulos, and Y. Kompatsiaris. Geotagging social media content with a refined language modelling approach. In Intelligence and Security Informatics, pages 21–40, 2015.

[4] A. Popescu. CEA LIST's participation at mediaeval 2013 placing task. In MediaEval 2013 Placing Task, 2013.

[5] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.

[6] O. Van Laere, S. Schockaert, and B. Dhoedt. Finding locations of Flickr resources using language models and similarity search. ICMR ’11, pages 48:1–48:8, New York, NY, USA, 2011. ACM.