Proceedings of the 2019 EMNLP Workshop W-NUT: The 5th Workshop on Noisy User-generated Text, pages 231–236Hong Kong, Nov 4, 2019. c©2019 Association for Computational Linguistics

231

Identifying Linguistic Areas for Geolocation

Tommaso Fornaciari, Dirk HovyBocconi University, Milan, Italy

{fornaciari|dirk.hovy}@unibocconi.it

Abstract

Geolocating social media posts relies on theassumption that language carries sufficient ge-ographic information. However, locations areusually given as continuous latitude/longitudetuples, so we first need to define discrete geo-graphic regions that can serve as labels. Moststudies use some form of clustering to dis-cretize the continuous coordinates (Han et al.,2016). However, the resulting regions do notalways correspond to existing linguistic areas.Consequently, accuracy at 100 miles tends tobe good, but degrades for finer-grained dis-tinctions, when different linguistic regions getlumped together. We describe a new algo-rithm, Point-to-City (P2C), an iterative k-dtree-based method for clustering geographiccoordinates and associating them with towns.We create three sets of labels at different levelsof granularity, and compare performance of astate-of-the-art geolocation model trained andtested with P2C labels to one with regular k-dtree labels. Even though P2C results in sub-stantially more labels than the baseline, modelaccuracy increases significantly over using tra-ditional labels at the fine-grained level, whilestaying comparable at 100 miles. The resultssuggest that identifying meaningful linguisticareas is crucial for improving geolocation at afine-grained level.

1 IntroductionPredicting the location of a Social Media postinvolves first and foremost ways to identify thewords that indicate geographic location. Secondly,and perhaps even more fundamentally, though, wealso need to determine an effective notion of whata “location” is, i.e., what do our labels represent:a state, a city, a neighborhood, a street? In manyNLP tasks, labels are ambiguous and open to inter-pretation (Plank et al., 2014). In geolocation, theinformation initially given is an unambiguous lati-tude/longitude pair, but this format captures a level

of detail (precise down to a centimeter) that is bothunnecessary and unrealistic for most practical ap-plications. Collapsing coordinates to geographiccategories is therefore a common step in geoloca-tion. However, this discretization step is open tointerpretation: what method should we choose?

Previous work includes three different ap-proaches to discretizing continuous values into lo-cation labels (see also Section 2):

1.) Geodesic grids are the most straightforward,but do not “lead to a natural representation ofthe administrative, population-based or languageboundaries in the region” (Han et al., 2012).

2.) Clustering coordinates prevents the iden-tification of (nearly) empty locations and keepspoints which are geographically close togetherin one location. Unfortunately, in crowded re-gions, clusters might be too close to each other,and therefore divide cultural/linguistic areas intomeaningless groups.

3.) Predefined administrative regions, likecities, can provide homogeneous interpretable ar-eas. However, mapping coordinates to the clos-est city can be ambiguous. Previous work typi-cally considered cities with a population of at least100K (Han et al., 2012, 2014). This approach hasthe opposite problem of clustering: different lin-guistic areas might be contained within a singleadministrative region.

Here, we propose Point-To-City (P2C), a newmethod mapping continuous coordinates to loca-tions. It combines the strengths of the last two ap-proaches, keeping coordinates which appear closeto each other in the same location, while also rep-resenting them in terms of meaningful administra-tive regions, with adjustable granularity. We showthat these two criteria also result in superior pre-diction performance for geolocation.

Relying on k-d trees (Maneewongvatana andMount, 1999), P2C iteratively clusters points

232

within a specified maximum distance d, and mapsthem to the coordinates of the closest town with aminimum population size.

We evaluate P2C on two data sets commonlyused for geolocation. We create three differentconditions by using three different values for d asmaximum distance between points, and comparethe results to those obtained using k-d tree labels(as used in the W-NUT shared task (Han et al.,2016)). For all four labeling schemes, we train anattention-based convolutional neural network, andevaluate mean and median distance between targetand predicted point, and accuracy within 161 km(Acc@161). We also show the standard accuracyscore relative to the specific labels, usually muchworse than Acc@161, and often not reported inthe literature.

Our results show that P2C reliably producesAcc@161 performance which is comparable withstate-of-the-art models. For exact accuracy, how-ever, P2C labels always result in substantially bet-ter performance than previous methods, in spiteof the larger set of classes. This suggests thatP2C captures more meaningful location distinc-tions (backed up by a qualitative analysis), and thatprevious labels capture only broader, linguisticallymixed areas. More generally, our results show thatlanguage reflects social and geographical distinc-tions in the world, and that more meaningful real-world labels help language-based prediction mod-els to perform their task more efficiently.

Contributions The contributions of this paperare the following: 1.) we propose P2C, a k-d treebased procedure to cluster geographic points as-sociated with existing towns within a certain dis-tance between town and cluster centroid. 2.) weshow that P2C produces more meaningful, inter-pretable cultural and linguistic locations 3.) weshow that P2C labels substantially improve modelperformance in exact, fine-grained classification

2 Related work

Geolocation prediction can, in principle, be mod-eled both as regression and as classification prob-lem. In practice, however, given the difficulty ofpredicting continuous coordinate values, regres-sion is often carried out in conjunction with theclassification (Eisenstein et al., 2010; Lourentzouet al., 2017; Fornaciari and Hovy, 2019b). In gen-eral, however, the task is considered a classifi-cation problem, which requires solutions for the

identification of geographic regions as labels.Geodesic grids were used for the geolocation of

posts on Flickr, Twitter and Wikipedia (Serdyukovet al., 2009; Wing and Baldridge, 2011).

Hulden et al. (2015) noticed that “using smallergrid sizes leads to an immediate sparse data prob-lem since very few features/words are [selectively]observed in each cell”.

In order to enhance the expressiveness of the ge-ographic cells, Wing and Baldridge (2014), con-structed both flat and hierarchical grids relying onk-d tree, and testing their methods at different lev-els of granularity. The same labels were used inthe study of Rahimi et al. (2018).

Han et al. (2012, 2014), who releasedTWITTER-WORLD, use the information providedby the Geoname dataset1 in order to identify a setof cities around the world with at least 100K in-habitants. Then they refer their geo-tagged textsto those cities, creating easily interpretable ge-ographic places. Cha et al. (2015) proposed avoting-based grid selection scheme, with the clas-sification referred to regions/states in US.

Most works use deep learning techniques forclassification (Miura et al., 2016). Often, theyinclude multi-view models, considering differ-ent sources (Miura et al., 2017; Lau et al.,2017; Ebrahimi et al., 2018; Fornaciari and Hovy,2019a). In particular, Lau et al. (2017) imple-mented a multi-channel convolutional network,structurally similar to our model. Rahimi et al.(2018) proposes a Graph-Convolutional neuralnetwork, though the text features are representedby a bag-of-words, while we rely on word embed-dings.

The ability of the labels to reflect real anthropo-logical areas, however, affects primarily the mod-els which rely on linguistic data. This is thecase of the studies of Han et al. (2012) and Hanet al. (2014) who based their predictions on theso-called Location-Indicative Words (LIW). Re-cently, neural models have been built with thesame purpose (Rahimi et al., 2017; Tang et al.,2019).

3 MethodsData sets We apply our method to two widelyused data sets for geolocation: TWITTER-US(Roller et al., 2012), and TWITTER-WORLD (Hanet al., 2012). They are all collections of En-

1http://www.geonames.org

233

glish tweets aggregated by author and labeledwith geographic coordinates. TWITTER-US andTWITTER-WORLD contain 450K and 1.39M texts,respectively. They are each divided into their owntraining, development, and test sets. Readers arereferred to the respective papers for additional de-tails. We round the coordinates to the second dec-imal number. A distance of 0.01 degrees corre-sponds to less than 1.1 km on the longitude axis(the distance is not constant on the latitude axis).Smaller distinctions are not relevant for any com-mon NLP task.

Data set d labels mean median

TW.-US .1 1554 7.07 3.81.25 914 9.10 5.64.5 418 15.54 12.21

W-NUT 256 – –

TW.-WORLD .1 3047 0.45 0.00.25 2818 1.77 0.00.5 2350 3.28 2.39

W-NUT 930 – –

Table 1: Number of labels and mean/median distancein km between instances and the cluster town center.For W-NUT, distance can not be computed, as cen-troids are not close to meaningful places

Point-To-City (P2C) For the classification, weneed to identify labels corresponding to existingcultural/linguistic areas, so that the geographic in-formation conveyed through language can be fullyexploited.To this end, P2C iteratively creates clus-ters of points, and afterwards associates the finalclusters with specific towns.

The parameter d controls the maximum spheri-cal distance we allow between points assigned tothe same cluster at the initialization step. We runP2C considering three values: 0.1, 0.25, and 0.5coordinate decimal points, which correspond to11.12 km (6.91 miles), 27.80 km (17.27 miles),and 55.60 km (34.55 miles) on the longitude axis.We use these values to explore the feasibilityof finer (and more challenging) predictions thanthose usually accepted in the literature.

One of the most popular metrics in previousstudies (see Section 2 and 4) is the accuracy ofthe predictions within 161 km, or 100 mi, fromthe target point. In contrast, we are interested inthe accuracy relative to the precise prediction ofthe labels, and we want labels representing pointsaggregated according to a distance much smallerthan 161 km/100 mi: even the highest value we

choose for d, 0.5, is about one third the distance ofaccuracy at 161 km (Acc@161). However, sinceP2C iteratively creates clusters of clusters, it ispossible that the original points belonging to dif-ferent clusters are further apart than the thresh-old of d. For this reason, we selected values of dwhich are about three to fifteen times smaller than161 km/100 mi.

Given d and a set of coordinate points/instancesin the data set, P2C iterates over the followingsteps until convergence:

1. for each point, apply k-d trees to find clustersof points where each pair has a reciprocal dis-tance less than d;

2. remove redundant clusters by ordering theirelements (e.g., (A,B) vs. (B,A));

3. remove subsets of larger clusters (e.g. (A,B)vs. (A,B,C));

4. compute clusters’ centroids as the averagecoordinates of all points belonging to thecluster;

5. assign the points which fall into more thanone cluster to the one with the nearest cen-troid;

6. substitute each instance’s coordinates withthe centroid coordinates of the correspondingcluster.

The algorithm converges when the final number ofpoints cannot be further reduced, since they all arefarther apart from each other than the maximumdistance d. After assigning each instance its newcoordinates, we follow Han et al. (2012, 2014) inusing the GeoNames data set to associate clus-ters with cities, by substituting the instance coordi-nates with those of the closest town center. In ourcase, however, rather than collecting cities with apopulation of at least 100K, we consider all townswith a population of at least 15K.

This last step further reduces the set of pointsassociated with our instances. Table 1 shows theresulting number of labels, and the mean distancein km between the new instance coordinates andthe respective town center.

This choice of 15K inhabitants is coherent withthe settings of d: we aim to account for lin-guistic/social environments more specific than thebroad and compound communities of densely pop-ulated cities. This is helpful for high resolution

234

method model # labels Acc Acc@161 mean median

TWITTER-US

Han et al. (2014) NB + IGR 378 26% 45% - 260Wing and Baldridge (2014) HierLR k-d “fewer classes” - 48% 687 191Rahimi et al. (2017) MLP + k-d tree 256 - 55% 581 91

Att-CNN + k-d tree 256 26.17% 55.27% 580.7 93.02Att-CNN + P2C .1 1554 44.04%∗ 59.76%∗ 544.35∗ 47.19∗

Att-CNN + P2C .25 914 49.08%∗ 60.4%∗ 537.0∗ 39.71∗

Att-CNN + P2C .5 418 54.73%∗ 58.56% 537.79∗ 0∗

TWITTER-WORLD

Han et al. (2014) NB + IGR 3135 13% 26% - 913Wing and Baldridge (2014) HierLR k-d “fewer classes” - 31% 1670 509Rahimi et al. (2017) MLP + k-d tree 930 - 36% 1417 373

Att-CNN + k-d tree 930 18.35% 33.85% 1506.33 469.48Att-CNN + P2C .1 3047 22.57%∗ 39.41%∗ 1372.3∗ 328.42∗

Att-CNN + P2C .25 2818 26.68%∗ 39.94%∗ 1269.13∗ 299.04∗

Att-CNN + P2C .5 2350 32.64%∗ 41.8%∗ 1257.36∗ 292.09∗

Table 2: Performance of prior work and of the proposed model with W-NUT and P2C labels. ∗ : p ≤ 0.01.

geolocation both in the case of crowded regionsand of areas with low density of inhabitants. How-ever, we found that in spite of qualified informa-tion, such as the annual Worlds Cities report ofthe United Nations, it is actually difficult to setan optimal threshold. In fact, not even that doc-ument provides a detailed profile of small towns ata global level. Therefore we rely on the format ofthe information offered by Geonames.

The code for computing P2C is available atgithub.com/Bocconi-NLPLab.

Feature selection The two corpora have verydifferent vocabulary sizes. Despite fewer in-stances, TWITTER-US contains a much richer vo-cabulary than TWITTER-WORLD: 14 vs. 6.5 mil-lions words. This size is computationally infeasi-ble. In order to maximize discrimination, we filterthe vocabulary with several steps.

In order not to waste the possible geographic in-formation carried by the huge amount of low fre-quency terms, we use replacement tokens as fol-lows: We again take only the training data intoaccount. First, we discard the hapax legomena,that is the words with frequency 1, as there is noevidence that these words could be found else-where. Then, we discard words with frequencygreater than 1, if they appear in more than oneplace. We replace low frequency terms which ap-pear uniquely in on place with a replacement tokenspecific for that place, i.e., label. Finally, we sub-stitute these words with their replacement token inthe whole corpus, including development and test

set. Since the word distribution follows the Zipfcurve (Powers, 1998) we are able to exploit thegeographic information of millions of words usingonly a small number of replacement tokens. Theuse of this information is fair, as it relies on the in-formation present in the training set only. In termsof performance, however, the effect of the replace-ment tokens is theoretically not different from thatresulting from the direct inclusion of the singlewords in the vocabulary.The benefit is in terms ofnoise reduction, for the selective removal of geo-graphically ambiguous words, and computationalaffordability.

Following Han et al. (2012), we further filterthe vocabulary via Information Gain Ratio (IGR),selecting the terms with the highest values untilwe reach a computationally feasible vocabularysize: here, 750K and 460K for TWITTER-US andTWITTER-WORLD.

Attention-based CNN For classification, weuse an attention-based convolutional neuralmodel. We first train our own word embeddingsfor each corpus, and feed the texts into two con-volutional channels (with window size 4 and 8)and max-pooling, followed by an overall attentionmechanism, and finally a fully-connected layerwith softmax activation for prediction.

For evaluation, as discussed in Section 3, weuse the common metrics considered in literature:acc@161, that is the accuracy within 161 km(100 mi) from the target point, and mean and me-dian distance between the predicted and the target

235

points. We are also interested in the exact accu-racy. This metric is often not shown in literature,but is important for the geolocation in real casescenarios. We evaluate significance via bootstrapsampling, following Søgaard et al. (2014).

(a) W-NUT labels

(b) P2C labels

Figure 1: Example of cumulative point accuracy withthe two label sets for gold label Washington DC (flag).Circles are predictions, diameter represents percentageof predictions on that point.

4 ResultsThe model performance is shown in table 2. Whenapplied to the W-NUT labels, our model repli-cates the results of Rahimi et al. (2017): inTWITTER-US the values correspond perfectly, inTWITTER-WORLD the Att-CNN performance isslightly lower. Compared to the W-NUT labels,the P2C labels are much more granular in everycondition and, in spite of their apparent greaterdifficulty, they help to reach better performance inall metrics, with very high levels of significance.Such differences are surprisingly wide with re-spect to the accuracy: in TWITTER-US, for P2Cwith d = .5, the performance is more than doubledcompared to the same model with the W-NUT k-d

tree labels (54% vs. 26%).Figure 1 shows the coordinates of the W-NUT

(1a) and of the P2C cluster centroids (1b). The di-ameter of the circles represent the rate correct pre-diction for those points. As can be seen, P2C iden-tifies a unique linguistic region around Washing-ton, while different W-NUT labels cover more orless the same area. P2C labels also allow a muchbetter concentration of predictions in the same ad-ministrative/linguistic area.

5 ConclusionP2C is a method for geographic labeling that dy-namically clusters points and links them to spe-cific towns. The aims are 1) to gather the pointsbelonging to the same linguistic areas; 2) to asso-ciate such areas with distinct, existing administra-tive regions; 3) to improve the models’ effective-ness, training them with texts showing consistentlinguistic patterns. Compared to the W-NUT k-dtree labels, P2C leads to remarkably higher per-formance in all metrics, and in particular in theaccuracy, even in spite of the higher number of la-bels identified. This suggests that techniques likeP2C might be particularly useful when high per-formance at high levels of granularity is required.

AcknowledgmentsThe authors would like to thank the reviewers ofthe various drafts for their comments. Both au-thors are members of the Bocconi Institute forData Science and Analytics (BIDSA) and the Dataand Marketing Insights (DMI) unit. This researchwas supported by a GPU donation from Nvidia, aswell as a research grant from CERMES to set up aGPU server, which enabled us to run these experi-ments.

ReferencesMiriam Cha, Youngjune Gwon, and HT Kung. 2015.

Twitter geolocation and regional classification viasparse coding. In Ninth International AAAI Confer-ence on Web and Social Media.

Mohammad Ebrahimi, Elaheh ShafieiBavani, Ray-mond Wong, and Fang Chen. 2018. A unified neuralnetwork model for geolocating twitter users. In Pro-ceedings of the 22nd Conference on ComputationalNatural Language Learning, pages 42–53.

Jacob Eisenstein, Brendan O’Connor, Noah A Smith,and Eric P Xing. 2010. A latent variable model forgeographic lexical variation. In Proceedings of the2010 conference on empirical methods in natural

236

language processing, pages 1277–1287. Associationfor Computational Linguistics.

Tommaso Fornaciari and Dirk Hovy. 2019a. DenseNode Representation for Geolocation. In Proceed-ings of the 5th Workshop on Noisy User-generatedText (WNUT).

Tommaso Fornaciari and Dirk Hovy. 2019b. Geoloca-tion with Attention-Based Multitask Learning Mod-els. In Proceedings of the 5th Workshop on NoisyUser-generated Text (WNUT).

Bo Han, Paul Cook, and Timothy Baldwin. 2012. Ge-olocation prediction in social media data by findinglocation indicative words. Proceedings of COLING2012, pages 1045–1062.

Bo Han, Paul Cook, and Timothy Baldwin. 2014. Text-based twitter user geolocation prediction. Journal ofArtificial Intelligence Research, 49:451–500.

Bo Han, Afshin Rahimi, Leon Derczynski, and Timo-thy Baldwin. 2016. Twitter Geolocation PredictionShared Task of the 2016 Workshop on Noisy User-generated Text. In Proceedings of the 2nd Workshopon Noisy User-generated Text (WNUT), pages 213–217.

Mans Hulden, Miikka Silfverberg, and Jerid Francom.2015. Kernel density estimation for text-based ge-olocation. In Twenty-Ninth AAAI Conference on Ar-tificial Intelligence.

Jey Han Lau, Lianhua Chi, Khoi-Nguyen Tran, andTrevor Cohn. 2017. End-to-end network for twittergeolocation prediction and hashing. arXiv preprintarXiv:1710.04802, pages 744–753.

Ismini Lourentzou, Alex Morales, and ChengXiangZhai. 2017. Text-based geolocation prediction ofsocial media users with neural networks. In 2017IEEE International Conference on Big Data (BigData), pages 696–705. IEEE.

Songrit Maneewongvatana and David M Mount. 1999.It’s okay to be skinny, if your friends are fat. In Cen-ter for Geometric Computing 4th Annual Workshopon Computational Geometry, volume 2, pages 1–8.

Yasuhide Miura, Motoki Taniguchi, Tomoki Taniguchi,and Tomoko Ohkuma. 2016. A simple scalable neu-ral networks based model for geolocation predictionin twitter. In Proceedings of the 2nd Workshop onNoisy User-generated Text (WNUT), pages 235–239.

Yasuhide Miura, Motoki Taniguchi, Tomoki Taniguchi,and Tomoko Ohkuma. 2017. Unifying text, meta-data, and user network representations with a neu-ral network for geolocation prediction. In Proceed-ings of the 55th Annual Meeting of the Associationfor Computational Linguistics (Volume 1: Long Pa-pers), volume 1, pages 1260–1272.

Barbara Plank, Dirk Hovy, and Anders Søgaard. 2014.Linguistically debatable or just plain wrong? InProceedings of the 52nd Annual Meeting of the As-sociation for Computational Linguistics (volume 2:Short Papers), pages 507–511.

David MW Powers. 1998. Applications and expla-nations of zipf’s law. In Proceedings of the jointconferences on new methods in language process-ing and computational natural language learning,pages 151–160. Association for Computational Lin-guistics.

Afshin Rahimi, Trevor Cohn, and Tim Baldwin.2018. Semi-supervised user geolocation viagraph convolutional networks. arXiv preprintarXiv:1804.08049, pages 2009–2019.

Afshin Rahimi, Trevor Cohn, and Timothy Baldwin.2017. A neural model for user geolocation and lex-ical dialectology. arXiv preprint arXiv:1704.04008,pages 209–216.

Stephen Roller, Michael Speriosu, Sarat Rallapalli,Benjamin Wing, and Jason Baldridge. 2012. Super-vised text-based geolocation using language modelson an adaptive grid. In Proceedings of the 2012 JointConference on Empirical Methods in Natural Lan-guage Processing and Computational Natural Lan-guage Learning, pages 1500–1510. Association forComputational Linguistics.

Pavel Serdyukov, Vanessa Murdock, and RoelofVan Zwol. 2009. Placing flickr photos on a map. InProceedings of the 32nd international ACM SIGIRconference on Research and development in infor-mation retrieval, pages 484–491. ACM.

Anders Søgaard, Anders Johannsen, Barbara Plank,Dirk Hovy, and Hector Martınez Alonso. 2014.What’s in a p-value in nlp? In Proceedings of theeighteenth conference on computational natural lan-guage learning, pages 1–10.

Haina Tang, Xiangpeng Zhao, and Yongmao Ren.2019. A multilayer recognition model for twitteruser geolocation. Wireless Networks, pages 1–6.

Benjamin Wing and Jason Baldridge. 2014. Hierar-chical discriminative classification for text-based ge-olocation. In Proceedings of the 2014 Conferenceon Empirical Methods in Natural Language Pro-cessing (EMNLP), pages 336–348.

Benjamin P Wing and Jason Baldridge. 2011. Sim-ple supervised document geolocation with geodesicgrids. In Proceedings of the 49th annual meeting ofthe association for computational linguistics: Hu-man language technologies-volume 1, pages 955–964. Association for Computational Linguistics.