Edinburgh Research Explorer

Universal Semantic ParsingCitation for published version:Reddy, S, Täckström, O, Petrov, S, Steedman, M & Lapata, M 2017, Universal Semantic Parsing. inProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association forComputational Linguistics, Copenhagen, Denmark , pp. 89–101, EMNLP 2017: Conference on EmpiricalMethods in Natural Language Processing, Copenhagen, Denmark, 7/09/17.https://doi.org/10.18653/v1/D17-1009

Digital Object Identifier (DOI):10.18653/v1/D17-1009

Link:Link to publication record in Edinburgh Research Explorer

Document Version:Peer reviewed version

Published In:Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

General rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.

Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.

Download date: 04. Apr. 2021

https://doi.org/10.18653/v1/D17-1009https://doi.org/10.18653/v1/D17-1009https://www.research.ed.ac.uk/portal/en/publications/universal-semantic-parsing(09d7a4a4-486a-4fdd-870c-4075f9bd6acf).html

Universal Semantic Parsing

Siva Reddy† Oscar Täckström‡ Slav Petrov‡ Mark Steedman† Mirella Lapata††ILCC, School of Informatics, University of Edinburgh

‡ Google, Inc.siva.reddy@ed.ac.uk, {oscart, slav}@google.com, {steedman, mlap}@inf.ed.ac.uk

Abstract

Universal Dependencies (UD) provides across-linguistically uniform syntactic rep-resentation, with the aim of advancing mul-tilingual applications of parsing and natu-ral language understanding. Reddy et al.(2016) recently developed a semantic in-terface for (English) Stanford Dependen-cies, based on the lambda calculus. In thiswork, we introduce UDEPLAMBDA, a simi-lar semantic interface for UD, which allowsmapping natural language to logical formsin an almost language-independent frame-work. We evaluate our approach on seman-tic parsing for the task of question answer-ing against Freebase. To facilitate multilin-gual evaluation, we provide German andSpanish translations of the WebQuestionsand GraphQuestions datasets. Results showthat UDEPLAMBDA outperforms strongbaselines across languages and datasets.For English, it achieves the strongest resultto date on GraphQuestions, with competi-tive results on WebQuestions.

1 Introduction

The Universal Dependencies (UD) initiative seeksto develop cross-linguistically consistent annota-tion guidelines as well as a large number of uni-formly annotated treebanks for many languages.1

Such resources could advance multilingual applica-tions of parsing, improve comparability of evalua-tion results, enable cross-lingual learning, and moregenerally support natural language understanding.

Seeking to exploit the benefits of UD for natu-ral language understanding, we introduce UDEP-LAMBDA, a semantic interface for UD that maps

1http://www.universaldependencies.org/.

natural language to logical forms, representing un-derlying predicate-argument structures, in an al-most language-independent manner. Our frame-work is based on DEPLAMBDA (Reddy et al., 2016)a recently developed method that converts EnglishStanford Dependencies (SD) to logical forms. Theconversion process is illustrated in Figure 1 and dis-cussed in more detail in Section 2. Whereas DEP-LAMBDA works only for English, UDEPLAMBDAapplies to any language for which UD annotationsare available.2 In this paper, we describe the ra-tionale behind UDEPLAMBDA and highlight im-portant differences from DEPLAMBDA, some ofwhich stem from the different treatment of variouslinguistic constructions in UD.

Our experiments focus on semantic parsing as atestbed for evaluating the framework’s multilingualappeal. We address the task of learning to mapnatural language to machine interpretable formalmeaning representations, specifically retrieving an-swers to questions from Freebase. To facilitatemultilingual evaluation, we provide translationsof the English WebQuestions (Berant et al., 2013)and GraphQuestions (Su et al., 2016) datasets toGerman and Spanish. We demonstrate that U-DEPLAMBDA can be used to derive logical formsfor these languages using a minimal amount oflanguage-specific knowledge. Aside from devel-oping the first multilingual semantic parsing toolfor Freebase, we also experimentally show that U-DEPLAMBDA outperforms strong baselines acrosslanguages and datasets. For English, it achievesthe strongest result to date on GraphQuestions,with competitive results on WebQuestions. Be-yond semantic parsing, we believe that the log-ical forms produced by our framework will beof use in various natural understanding tasks in-cluding entailment (Beltagy et al., 2016), text-

2As of v1.3, UD annotations are available for 47 languages.

arX

iv:1

702.

0319

6v2

[cs

.CL

] 1

5 Fe

b 20

17

mailto:siva.reddy@ed.ac.ukmailto:oscart@google.commailto:slav@google.commailto:steedman@inf.ed.ac.ukmailto:mlap@inf.ed.ac.ukhttp://www.universaldependencies.org/

based question answering (Lewis and Steedman,2013), sentence simplification (Narayan and Gar-dent, 2014), summarization (Liu et al., 2015), para-phrasing (Pavlick et al., 2015), and relation extrac-tion (Rocktäschel et al., 2015). Our implementa-tion and translated datasets are publicly availableat https://github.com/sivareddyg/udeplambda.

2 DEPLAMBDA

Before describing UDEPLAMBDA, we provide anoverview of DEPLAMBDA (Reddy et al., 2016) onwhich our approach is based. DEPLAMBDA con-verts a dependency tree to its logical form in threesteps: binarization, substitution, and composition,each of which is briefly outlined below.

Binarization A dependency tree is first mappedto a Lisp-style s-expression indicating the orderof semantic composition. Figure 1(b) shows thes-expression for the sentence Disney won an Os-car for the movie Frozen, derived from the depen-dency tree in Figure 1(a). Here, the sub-expression(dobj won (det Oscar an)) indicates that the logi-cal form of the phrase won an Oscar is derived bycomposing the logical form of the label dobj withthe logical form of the word won and the logicalform of the phrase an Oscar, derived analogously.

An obliqueness hierarchy is employed to imposea strict ordering on the modifiers to each head inthe dependency tree. As an example, won hasthree modifiers in Figure 1(a), which accordingto the obliqueness hierarchy are composed in theorder dobj> nmod> nsubj. In constructions likecoordination, this ordering is crucial to arrive at thecorrect semantics (see Section 3.3).

Substitution Each symbol in the s-expressionsis substituted for a lambda expression encodingits semantics. Words and dependency labels areassigned different types of expressions. In general,words have expressions of the following kind:ENTITY ⇒ λx.word(xa); e.g. Oscar⇒ λx.Oscar(xa)EVENT ⇒ λx.word(xe); e.g. won⇒ λx.won(xe)FUNCTIONAL⇒ λx.TRUE; e.g. an⇒ λx.TRUE

Here, the subscripts ·a and ·e denote the types ofindividuals (Ind) and events (Event), respectively,whereas x denotes a paired variable (xa,xe) of typeInd×Event. Roughly speaking, proper nouns andadjectives invoke ENTITY expressions, verbs andadverbs invoke EVENT expressions, and commonnouns invoke both ENTITY and EVENT expressions(see Section 3.3), while remaining words invoke

Disney won an Oscar for the movie Frozenpropn verb det propn adp det noun propn

nsubj

dobj

nmod

det

case

det

compound

root

(a) The dependency tree for Disney won an Oscar for themovie Frozen in the Universal Dependencies formalism.

(nsubj (nmod (dobj won (det Oscar an))(case (det (comp. Frozen movie) the) for)) Disney)

(b) The binarized s-expression for the dependency tree.

λx.∃yzw.won(xe)∧Disney(ya)∧Oscar(za)∧Frozen(wa)∧ movie(wa)∧arg1(xe,ya)∧ arg2(xe,za)∧ nmod.for(xe,wa)

(c) The composed lambda-calculus expression.

Figure 1: The mapping of a dependency tree to itslogical form with the intermediate s-expression.

FUNCTIONAL expressions. As in DEPLAMBDA,we enforce the constraint that every s-expressionis of the type η = Ind×Event → Bool, whichsimplifies the type system considerably.

Expressions for dependency labels glue thesemantics of heads and modifiers to articulatepredicate-argument structure. These expressions ingeneral take one of the following forms:COPY ⇒ λ f gx.∃y. f (x)∧g(y)∧ rel(x,y)e.g. nsubj, dobj, nmod, advmodINVERT ⇒ λ f gx.∃y. f (x)∧g(y)∧ reli(y,x)e.g. amod, aclMERGE ⇒ λ f gx. f (x)∧g(x)e.g. compound, appos, amod, aclHEAD ⇒ λ f gx. f (x)e.g. case, punct, aux, mark .

As an example of COPY, consider the lambdaexpression for dobj in (dobj won (det Oscar an)):λ f gx.∃y. f (x)∧ g(y)∧ arg2(xe,ya). This expres-sion takes two functions f and g as input, wheref represents the logical form of won and g repre-sents the logical form of an Oscar. The predicate-argument structure arg2(xe,ya) indicates that thearg2 of the event xe, i.e. won, is the individual ya,i.e. the entity Oscar. Since arg2(xe,ya) mimics thedependency structure dobj(won, Oscar), we referto the expression kind evoked by dobj as COPY.

Expressions that invert the dependency direc-tion are referred to as INVERT (e.g. amod in run-ning horse); expressions that merge two subexpres-sions without introducing any relation predicatesare referred to as MERGE (e.g. compound in movieFrozen); and expressions that simply return the par-

https://github.com/sivareddyg/udeplambda

ent expression semantics are referred to as HEAD(e.g. case in for Frozen). While this generalizationapplies to most dependency labels, several labelstake a different logical form not listed here, someof which are discussed in Section 3.3. 3 Sometimesthe mapping of dependency label to lambda expres-sion may depend on surrounding part-of-speechtags or dependency labels. For example, amod actsas INVERT when the modifier is a verb (e.g. in run-ning horse), and as MERGE when the modifier isan adjective (e.g. in beautiful horse).4

Composition The final logical form is computedby beta-reduction, treating expressions of the form(f x y) as the function f applied to the argumentsx and y. For example, (dobj won (det Oscar an))results in λx.∃z.won(xe)∧Oscar(za)∧ arg2(xe,za)when the expression for dobj is applied to thosefor won and (det Oscar an). Figure 1(c) shows thelogical form for the s-expression in Figure 1(b).

3 UDEPLAMBDA

We now introduce UDEPLAMBDA, a semantic in-terface for Universal Dependencies.5 WhereasDEPLAMBDA only applies to English Stanford De-pendencies, UDEPLAMBDA takes advantage of thecross-lingual nature of UD to facilitate an (almost)language independent semantic interface. This isaccomplished by restricting the binarization, sub-stitution, and composition steps described aboveto rely solely on information encoded in the UDrepresentation. Importantly, UDEPLAMBDA is de-signed to not rely on lexical forms in a language toassign lambda expressions, but only on informationcontained in dependency labels and postags.

However, some linguistic phenomena are lan-guage specific (e.g. pronoun-dropping) or mean-ing specific (e.g. every and the in English havevery different semantics, despite being both deter-miners) and are not encoded in the UD schema.Furthermore, some cross-linguistic phenomena,such as long-distance dependencies, are not part ofthe core UD representation. To circumvent thislimitation, a simple enhancement step enrichesthe original UD representation before binariza-

3Mappings are available at https://github.com/sivareddyg/udeplambda.

4We use Tregex (Levy and Andrew, 2006) for substitu-tion mappings and Cornell SPF (Artzi, 2013) as the lambda-calculus implementation. For example, in running horse, thetregex /label:amod/=target < /postag:verb/ matches amod toits INVERT expression λ f gx.∃y. f (x)∧g(y)∧ amodi(ye,xa).

5In what follows, all references to UD are to UD v1.3.

Anna wants to marry Kristoff

nsubj

xcomp

mark dobj

nsubj

(a) With long-distance dependency.

Anna wants to marry Kristoff

Ω Ω

nsubj

xcomp

mark dobj

bind nsubj

(b) With variable binding.

Figure 2: The original and enhanced dependencytrees for Anna wants to marry Kristoff.

tion takes place (Section 3.1). This step adds tothe dependency tree missing syntactic informationand long-distance dependencies, thereby creating agraph. Whereas DEPLAMBDA is not able to han-dle graph-structured input, UDEPLAMBDA is de-signed to work directly with dependency graphs(Section 3.2). Finally, the representation of severallinguistic constructions differ between UD and SD,which requires different handling in the semanticinterface (Section 3.3).

3.1 Enhancement

Both Schuster and Manning (2016) and Nivre et al.(2016) note the necessity of an enhanced UD rep-resentation to enable semantic applications. How-ever, such enhancements are currently only avail-able for a subset of languages in UD. Instead, werely on a small number of enhancements for ourmain application—semantic parsing for question-answering—with the hope that this step can be re-placed by an enhanced UD representation in the fu-ture. Specifically, we define three kinds of enhance-ments: (1) long-distance dependencies; (2) typesof coordination; and (3) refined question word tags.

First, we identify long-distance dependencies inrelative clauses and control constructions. We fol-low Schuster and Manning (2016) and find theseusing the labels acl (relative) and xcomp (control).Figure 2(a) shows the long-distance dependency inthe sentence Anna wants to marry Kristoff. Here,marry is provided with its missing nsubj (dashedarc). Second, UD conflates all coordinating con-structions to a single dependency label, conj. Toobtain the correct coordination scope, we refineconj to conj:verb, conj:vp, conj:sentence,

https://github.com/sivareddyg/udeplambdahttps://github.com/sivareddyg/udeplambda

conj:np, and conj:adj, similar to Reddy et al.(2016). Finally, unlike the PTB tags (Marcus et al.,1993) used by SD, the UD part-of-speech tags donot distinguish question words. Since these are cru-cial to question-answering, we use a small lexiconto refine the tags for determiners (DET), adverbs(ADV) and pronouns (PRON) to DET:WH, ADV:WHand PRON:WH, respectively. Specifically, we usea list of 12 (English), 14 (Spanish) and 35 (Ger-man) words, respectively. This is the only partof UDEPLAMBDA that relies on language-specificinformation. We hope that, as the coverage of mor-phological features in UD improves, this refine-ment can be replaced by relying on morphologicalfeatures, such as the interrogative feature (INT).

3.2 Graph Structures and BIND

To handle graph structures that may result fromthe enhancement step, such as those in Figure 2(a),we propose a variable-binding mechanism that dif-fers from that of DEPLAMBDA. First, each long-distance dependency is split into independent arcsas shown in Figure 2(b). Here, Ω is a placeholderfor the subject of marry, which in turn correspondsto Anna as indicated by the binding of Ω via thepseudo-label BIND. We treat BIND like an ordinarydependency label with semantics MERGE and pro-cess the resulting tree as usual, via the s-expression:(nsubj (xcomp wants (nsubj (mark

(dobj marry Kristoff) to) Ω) (BIND Anna Ω)) ,

with the lambda-expression substitutions:wants, marry ∈ EVENT; to ∈ FUNCTIONAL;Anna, Kristoff ∈ ENTITY;mark ∈ HEAD; BIND ∈ MERGE;xcomp = λ f gx.∃y. f (x)∧g(y)∧xcomp(xe,ye) .

These substitutions are based solely on unlexi-calized context. For example, the part-of-speechtag PROPN of Anna invokes an ENTITY expression.

The placeholder Ω has semantics λx.EQ(x,ω),where EQ(u,ω) is true iff u and ω are equal (havethe same denotation), which unifies the subject vari-able of wants with the subject variable of marry.

After substitution and composition, we get:λz.∃xywv.wants(ze)∧Anna(xa)∧ arg1(ze,xa)∧ EQ(x,ω)∧marry(ye)∧xcomp(ze,ye)∧ arg1(ye,va)∧ EQ(v,ω)∧ Kristoff(wa)∧ arg2(ye,wa) ,

This expression may be simplified further byreplacing all occurrences of v with x and removingthe unification predicates EQ, which results in:λz.∃xyw.wants(ze)∧Anna(xa)∧ arg1(ze,xa)∧marry(ye)∧xcomp(ze,ye)∧ arg1(ye,xa)∧ Kristoff(wa)∧ arg2(ye,wa) .

This expression encodes the fact that Anna is thearg1 of the marry event, as desired. DEPLAMBDA,in contrast, cannot handle graph-structured input,since it lacks a principled way of generating s-expressions from graphs. Even given the aboves-expression, BIND in DEPLAMBDA is defined ina way such that the composition fails to unify vand x, which is crucial for the correct semantics.Moreover, the definition of BIND in DEPLAMBDAdoes not have a formal interpretation within thelambda calculus, unlike ours.

3.3 Linguistic ConstructionsBelow, we highlight the most pertinent differencesbetween UDEPLAMBDA and DEPLAMBDA, stem-ming from the different treatment of various lin-guistic constructions in UD versus SD.

Prepositional Phrases UD uses a content-headanalysis, in contrast to SD, which treats functionwords as heads of prepositional phrases, Accord-ingly, the s-expression for the phrase presidentin 2009 is (nmod president (case 2009 in)) in U-DEPLAMBDA and (prep president (pobj in 2009))in DEPLAMBDA. To achieve the desired semantics,λx.∃y.president(xa)∧president event(xe)∧

arg1(xe,xa)∧2009(ya)∧prep.in(xe,ya) ,

DEPLAMBDA relies on an intermediate logicalform that requires some post-processing, whereasUDEPLAMBDA obtains the desired logical formdirectly through the following entries:in ∈ FUNCTIONAL; 2009 ∈ ENTITY; case ∈ HEAD;president = λx.president(xa)∧president event(xe)

∧arg1(xe,xa) ;nmod = λ f gx.∃y. f (x)∧g(y)∧nmod.in(xe,ya) .

Other nmod constructions, such as possessives(nmod:poss), temporal modifiers (nmod:tmod)and adverbial modifiers (nmod:npmod), are han-dled similarly. Note how the common noun presi-dent, evokes both entity and event predicates above.

Passives DEPLAMBDA gives special treatmentto passive verbs, identified by the fine-grained part-of-speech tags in the PTB tag together with de-pendency context. For example, An Oscar waswon is analyzed as λx.won.pass(xe)∧Oscar(ya)∧arg1(xe,ya), where won.pass represents a passiveevent. However, UD does not distinguish be-tween active and passive forms.6 While the labelsnsubjpass or auxpass indicate passive construc-tions, such clues are sometimes missing, such as in

6UD encodes voice as a morphological feature, but mostsyntactic analyzers do not produce this information yet.

language target people

x e1 y e2 Ghanaspeak.arg2

speak.arg1

people.arg1

people.nmod.in

type

type

(a) English

sprache target

x e1 Ghanagesprochen

.arg2gesprochen.nmod.in

type

(b) German

lengua target

x e1 Ghanalengua.arg1

lengua.nmod.de

type

(c) Spanish

language.human language

target

x m Ghanalocation.country

.official language.2location.country

.official language.1

type

(d) Freebase

Figure 3: The ungrounded graphs for What language do the people in Ghana speak?, Welche Sprachewird in Ghana gesprochen? and Cuál es la lengua de Ghana?, and the corresponding grounded graph.

reduced relatives. We therefore opt to not have sep-arate entries for passives, but aim to produce identi-cal logical forms for active and passive forms whenpossible (for example, by treating nsubjpass asdirect object). With the following entries,

won ∈ EVENT; an, was ∈ FUNCTIONAL; auxpass ∈ HEAD;nsubjpass = λ f gx.∃y. f (x)∧g(y)∧ arg2(xe,ya) ,

the lambda expression for An Oscar was won be-comes λx.won(xe)∧Oscar(ya)∧arg2(xe,ya), iden-tical to that of its active form. However, not havinga special entry for passive verbs may have unde-sirable side-effects. For example, in the reduced-relative construction Pixar claimed the Oscar wonfor Frozen, the phrase the Oscar won ... willreceive the semantics λx.Oscar(ya)∧won(xe)∧arg1(xe,ya), which differs from that of an Oscarwas won. We leave it to the target application todisambiguate the interpretation in such cases.

Long-Distance Dependencies As discussed inSection 3.2, we handle long-distance dependen-cies evoked by clausal modifiers (acl) and con-trol verbs (xcomp) with the BIND mechanism,whereas DEPLAMBDA cannot handle control con-structions. For xcomp, as seen earlier, we use themapping λ f gx.∃y. f (x)∧g(y)∧xcomp(xe,ye). Foracl we use λ f gx.∃y. f (x)∧ g(y), to conjoin themain clause and the modifier clause. However, notall acl clauses evoke long-distance dependencies,e.g. in the news that Disney won an Oscar, theclause that Disney won an Oscar is a subordinatingconjunction of news. In such cases, we insteadassign acl the INVERT semantics.

Questions Question words are marked with theenhanced part-of-speech tags DET:WH, ADV:WHand PRON:WH, which are all assigned the seman-tics λx.${word}(xa)∧ TARGET(xa). The predicateTARGET indicates that xa represents the variable ofinterest, that is the answer to the question.

3.4 Limitations

In order to achieve language independence, UDEP-LAMBDA has to sacrifice semantic specificity, sincein many cases the semantics is carried by lexicalinformation. Consider the sentences John broke thewindow and The window broke. Although it is thewindow that broke in both cases, our inferred logi-cal forms do not canonicalize the relation betweenbroke and window. To achieve this, we would haveto make the substitution of nsubj depend on lexicalcontext, such that when window occurs as nsubjwith broke, the predicate arg2 is invoked rather thanarg1. We do not address this problem, and leave itto the target application to infer that arg2 and arg1have the same semantic function in these cases. Weanticipate that the ability to make such lexicalizedsemantic inferences in a task-agnostic cross-lingualframework would be highly useful and a crucialavenue for future work on universal semantics.

Other constructions that require lexical infor-mation are quantifiers like every, some and most,negation markers like no and not, and intentionalverbs like believe and said. UD does not have spe-cial labels to indicate these. Although not currentlyimplemented, we discuss how to handle quantifiersin this framework in the supplementary material.Fancellu et al. (2017) is a first step in this direction.

4 Cross-lingual Semantic Parsing

To study the multilingual nature of UDEPLAMBDA,we conduct an empirical evaluation on questionanswering against Freebase in three different lan-guages: English, Spanish, and German. Beforediscussing the details of this experiment, we brieflyoutline the semantic parsing framework employed.

4.1 Semantic Parsing as Graph Matching

UDEPLAMBDA generates ungrounded logicalforms that are independent of any knowledge base,such as Freebase. We use GRAPHPARSER (Reddyet al., 2016) to map these logical forms to their

WebQuestions

en What language do the people in Ghana speak?de Welche Sprache wird in Ghana gesprochen?es ¿Cuál es la lengua de Ghana?

en Who was Vincent van Gogh inspired by?de Von wem wurde Vincent van Gogh inspiriert?es ¿Qué inspiró a Van Gogh?

GraphQuestions

en NASA has how many launch sites?de Wie viele Abschussbasen besitzt NASA?es ¿Cuántos sitios de despegue tiene NASA?

en Which loudspeakers are heavier than 82.0 kg?de Welche Lautsprecher sind schwerer als 82.0 kg?es ¿Qué altavoces pesan más de 82.0 kg?

Table 1: Example questions and their translations.

grounded Freebase graphs, via corresponding un-grounded graphs. Figures 3(a) to 3(c) show theungrounded graphs corresponding to logical formsfrom UDEPLAMBDA, each grounded to the sameFreebase graph in Figure 3(d). Here, rectangles de-note entities, circles denote events, rounded rectan-gles denote entity types, and edges between eventsand entities denote predicates or Freebase relations.Finally, the TARGET node represents the set of val-ues of x that are consistent with the Freebase graph,that is the answer to the question.

GRAPHPARSER treats semantic parsing as agraph-matching problem with the goal of findingthe Freebase graphs that are structurally isomorphicto an ungrounded graph and rank them accordingto a model. To account for structural mismatches,GRAPHPARSER uses two graph transformations:CONTRACT and EXPAND. In Figure 3(a) there aretwo edges between x and Ghana. CONTRACT col-lapses one of these edges to create a graph isomor-phic to Freebase. EXPAND, in contrast, adds edgesto connect the graph in the case of disconnectedcomponents. The search space is explored by beamsearch and model parameters are estimated withthe averaged structured perceptron (Collins, 2002)from training data consisting of question-answerpairs, using answer F1-score as the objective.

4.2 Datasets

We evaluate our approach on two public bench-marks of question answering against Freebase:WebQuestions (Berant et al., 2013), a widely usedbenchmark consisting of English questions andtheir answers, and GraphQuestions (Su et al., 2016),a recently released dataset of English questions

with both their answers and grounded logical forms.While WebQuestions is dominated by simple entity-attribute questions, GraphQuestions contains alarge number of compositional questions involvingaggregation (e.g. How many children of EddardStark were born in Winterfell? ) and comparison(e.g. In which month does the average rainfall ofNew York City exceed 86 mm? ). The number oftraining, development and test questions is 2644,1134, and 2032, respectively, for WebQuestionsand 1794, 764, and 2608 for GraphQuestions.

To support multilingual evaluation, we createdtranslations of WebQuestions and GraphQuestionsto German and Spanish.7 For WebQuestions twoprofessional annotators were hired per language,while for GraphQuestions we used a trusted pool of20 annotators per language (with a single annotatorper question). Examples of the original questionsand their translations are provided in Table 1.

4.3 Implementation Details

Here we provide details on the syntactic analyzersemployed, our entity resolution algorithm, and thefeatures used by the grounding model.

Dependency Parsing The English, Spanish, andGerman Universal Dependencies (UD) treebanks(v1.3; Nivre et al 2016) were used to train part ofspeech taggers and dependency parsers. We used abidirectional LSTM tagger (Plank et al., 2016) anda bidirectional LSTM shift-reduce parser (Kiper-wasser and Goldberg, 2016). Both the tagger andparser require word embeddings. For English, weused GloVe embeddings (Pennington et al., 2014)trained on Wikipedia and the Gigaword corpus.8

For German and Spanish, we used SENNA em-beddings (Collobert et al., 2011; Al-Rfou et al.,2013) trained on Wikipedia corpora (589M wordsGerman; 397M words Spanish).9 Measured on theUD test sets, the tagger accuracies are 94.5 (En-glish), 92.2 (German), and 95.7 (Spanish), withcorresponding labeled attachment parser scores of81.8, 74.7, and 82.2.

Entity Resolution We follow Reddy et al. (2016)and resolve entities in three steps: (1) potential en-tity spans are identified using seven handcraftedpart-of-speech patterns; (2) each span is associatedwith potential Freebase entities according to the

7Translations will be publicly released upon publication.8http://nlp.stanford.edu/projects/glove/.9https://sites.google.com/site/rmyeid/projects/polyglot.

http://nlp.stanford.edu/projects/glove/https://sites.google.com/site/rmyeid/projects/polyglot

k WebQuestions GraphQuestionsen de es en de es

1 89.6 82.8 86.7 47.2 39.9 39.510 95.7 91.2 94.0 56.9 48.4 51.6

Table 2: Structured perceptron k-best entity linkingaccuracies on the development sets.

WebQuestions GraphQuestionsMethod en de es en de es

SINGLEEVENT 47.6 43.9 45.0 15.9 8.3 11.2DEPTREE 47.8 43.9 44.5 15.8 7.9 11.0CCGGRAPH 48.4 – – 15.9 – –UDEPLAMBDA 48.3 44.2 45.7 17.6 9.0 12.4

Table 3: F1-scores on the test for models trainedon the training set (excluding the development set).

Freebase/KG API;10 and (3) the 10-best entity link-ing lattices, scored by a structured perceptron, areinput to GRAPHPARSER, leaving the final disam-biguation to the semantic parsing problem. Table 2shows the 1-best and 10-best entity disambiguationF1-scores for each language and dataset.11

Features We use features similar to Reddy et al.(2016): basic features of words and Freebase re-lations, and graph features crossing ungroundedevents with grounded relations, ungrounded typeswith grounded relations, and ungrounded answertype crossed with a binary feature indicating if theanswer is a number. In addition, we add featuresencoding the semantic similarity of ungroundedevents and Freebase relations. Specifically, we usedthe cosine similarity of the translation-invariant em-beddings of Huang et al. (2015).12

4.4 Comparison Systems

We compared UDEPLAMBDA to prior work andthree versions of GRAPHPARSER that operate ondifferent representations: entity cliques, depen-dency trees, and CCG-based semantic derivations.

SINGLEEVENT This model resembles thelearning-to-rank model of Bast and Haussmann(2015). An ungrounded graph is generated by con-necting all entities in the question with the TARGETnode, representing a single event. Note that this

10http://developers.google.com/freebase/.11Due to the recent Freebase API shutdown, we used the

KG API for GraphQuestions. We observed that this leads toinferior entity linking results compared to those of Freebase.

12http://128.2.220.95/multilingual/data/.

Method GraphQ. WebQ.

SEMPRE (Berant et al., 2013) 10.8 35.7JACANA (Yao and Van Durme, 2014) 5.1 33.0PARASEMPRE (Berant and Liang, 2014) 12.8 39.9QA (Yao, 2015) – 44.3AQQU (Bast and Haussmann, 2015) – 49.4AGENDAIL (Berant and Liang, 2015) – 49.7DEPLAMBDA (Reddy et al., 2016) – 50.3

STAGG (Yih et al., 2015) – 48.4 (52.5)BILSTM (Türe and Jojic, 2016) – 24.9 (52.2)MCNN (Xu et al., 2016) – 47.0 (53.3)AGENDAIL-RANK (Yavuz et al., 2016) – 51.6 (52.6)

UDEPLAMBDA 17.6 49.5

Table 4: F1-scores on the English GraphQuestionsand WebQuestions test sets (results with additionaltask-specific resources in parentheses). Followingprior work, for WebQuestions the union of the train-ing and development sets were used for training.

baseline cannot handle compositional questions, orthose with aggregation or comparison.

DEPTREE An ungrounded graph is obtained di-rectly from the original dependency tree. An eventis created for each parent and its dependents in thetree. Each dependent is linked to this event with anedge labeled with its dependency relation, while theparent is linked to the event with an edge labeledarg0. If a word is a question word, an additionalTARGET predicate is attached to its entity node.

CCGGRAPH This is the CCG-based semanticrepresentation of Reddy et al. (2014). Note thatthis baseline exists only for English.

4.5 ResultsTable 3 shows the performance of GRAPHPARSERwith these different representations. Here and inwhat follows, we use average F1-score of predictedanswers (Berant et al., 2013) as the evaluation met-ric. We first observe that UDEPLAMBDA consis-tently outperforms the SINGLEEVENT and DEP-TREE representations in all languages.13

For English, performance is almost on par withCCGGRAPH, which suggests that UDEPLAMBDAdoes not sacrifice too much specificity for univer-sality. With both datasets, results are lower forGerman compared to Spanish. This agrees withthe lower performance of the syntactic parser on

13For the DEPTREE model, we CONTRACT each multi-hop path between the question word and an entity to a singleedge. Without this constraint, DEPTREE F1 results are 45.5(en), 42.9 (de), and 44.2 (es) on WebQuestions, and 11.0 (en),6.6 (de), and 2.6 (es) on GraphQuestions.

http://developers.google.com/freebase/http://128.2.220.95/multilingual/data/

the German portion of the UD treebank. Finally,while these results confirm that GraphQuestions ismuch harder compared to WebQuestions, we notethat both datasets predominantly contain single-hopquestions, as indicated by the competitive perfor-mance of SINGLEEVENT on both datasets.

Table 4 compares UDEPLAMBDA with previ-ously published models which exist only for En-glish and have been mainly evaluated on Web-Questions. These are either symbolic like ours (firstblock) or employ neural networks (second block).Results for models using additional task-specifictraining resources, such as ClueWeb09, Wikipedia,or SimpleQuestions (Bordes et al., 2015) are shownin parentheses. On GraphQuestions, we achievea new state-of-the-art result with a gain of 4.8 F1-points over the previously reported best result. OnWebQuestions we are 2.1 points below the bestmodel using comparable resources, and 3.8 pointsbelow the state of the art. Most related to ourwork is the English-specific system of Reddy et al.(2016). We attribute the 0.8 point difference in F1-score to their use of the more fine-grained PTB tagset and Stanford Dependencies.

5 Related Work

Our work continues the long tradition of buildinglogical forms from syntactic representations initi-ated by Montague (1973). The literature is rife withattempts to develop semantic interfaces for HPSG(Copestake et al., 2005), LFG (Kaplan and Bresnan,1982; Dalrymple et al., 1995; Crouch and King,2006), TAG (Kallmeyer and Joshi, 2003; Gardentand Kallmeyer, 2003; Nesson and Shieber, 2006),and CCG (Steedman, 2000; Baldridge and Kruijff,2002; Bos et al., 2004; Artzi et al., 2015). Unlikeexisting semantic interfaces, UDEPLAMBDA (likeDEPLAMBDA) uses dependency syntax, taking ad-vantage of recent advances in multilingual parsing(McDonald et al., 2013; Nivre et al, 2016).

A common trend in previous work on semanticinterfaces is the reliance on rich typed feature struc-tures or semantic types coupled with strong typeconstraints, which can be very informative but un-avoidably language specific. Creating rich seman-tic types from dependency trees which lack a typingsystem would be labor intensive and brittle in theface of parsing errors. Instead, UDEPLAMBDArelies on generic unlexicalized information presentin dependency treebanks and uses a simple typesystem (one type for dependency labels, and one

for words) along with a combinatory mechanism,which avoids type collisions. Earlier attempts atextracting semantic representations from depen-dencies have mainly focused on language-specificdependency representations (Spreyer and Frank,2005; Simov and Osenova, 2011; Hahn and Meur-ers, 2011; Reddy et al., 2016; Falke et al., 2016;Beltagy, 2016), and multi-layered dependency an-notations (Jakob et al., 2010; Bédaride and Gar-dent, 2011). In contrast, UDEPLAMBDA derivessemantic representations for multiple languages ina common schema directly from Universal Depen-dencies. This work parallels a growing interest increating other forms of multilingual semantic repre-sentations (Akbik et al., 2015; Vanderwende et al.,2015; White et al., 2016; Evang and Bos, 2016).

We evaluate UDEPLAMBDA on semantic pars-ing for question answering against a knowledgebase. Here, the literature offers two main model-ing paradigms: (1) learning of task-specific gram-mars that directly parse language to a groundedrepresentation (Zelle and Mooney, 1996; Zettle-moyer and Collins, 2005; Wong and Mooney, 2007;Kwiatkowksi et al., 2010; Liang et al., 2011; Be-rant et al., 2013; Flanigan et al., 2014; Pasupat andLiang, 2015; Groschwitz et al., 2015); and (2) con-verting language to a linguistically motivated task-independent representation that is then mapped to agrounded representation (Kwiatkowski et al., 2013;Reddy et al., 2014; Krishnamurthy and Mitchell,2015; Gardner and Krishnamurthy, 2017). Ourwork belongs to the latter paradigm, as we mapnatural language to Freebase indirectly via logi-cal forms. Capitalizing on natural-language syn-tax affords interpretability, scalability, and reducedduplication of effort across applications (Benderet al., 2015). Our work also relates to literatureon parsing multiple languages to a common exe-cutable representation (Cimiano et al., 2013; Haasand Riezler, 2016). However, existing approaches(Kwiatkowksi et al., 2010; Jones et al., 2012; Jieand Lu, 2014) still map to the target meaning rep-resentations (more or less) directly.

6 Conclusions

We introduced UDEPLAMBDA, a semantic inter-face for Universal Dependencies, and showed thatthe resulting semantic representation can be usedfor question-answering against a knowledge basein multiple languages. We provided translations ofbenchmark datasets in German and Spanish, in the

hope to stimulate further multilingual research onsemantic parsing and question answering in general.We have only scratched the surface when it comesto applying UDEPLAMBDA to natural languageunderstanding tasks. In the future, we would like toexplore how this framework can benefit other taskssuch as summarization and machine translation.

Acknowledgements

This work greatly benefitted from discussionswith Michael Collins, Dipanjan Das, FedericoFancellu, Julia Hockenmaier, Tom Kwiatkowski,Adam Lopez, Valeria de Paiva, Martha Palmer,Fernando Pereira, Emily Pitler, Vijay Saraswat,Nathan Schneider, Bonnie Webber, and the mem-bers of ILCC Edinburgh University, the MicrosoftResearch Redmond NLP group, the Stanford NLPgroup, and the UW NLP and Linguistics group.The authors would also like to thank the Univer-sal Dependencies community for the treebanks anddocumentation. This research is supported by aGoogle PhD Fellowship to the first author.

ReferencesAlan Akbik, laura chiticariu, Marina Danilevsky, Yun-

yao Li, Shivakumar Vaithyanathan, and Huaiyu Zhu.2015. Generating High Quality Proposition Banksfor Multilingual Semantic Role Labeling. In Pro-ceedings of the 53rd Annual Meeting of the Associ-ation for Computational Linguistics and the 7th In-ternational Joint Conference on Natural LanguageProcessing. Association for Computational Linguis-tics, Beijing, China, pages 397–407.

Rami Al-Rfou, Bryan Perozzi, and Steven Skiena.2013. Polyglot: Distributed Word Representationsfor Multilingual NLP. In Proceedings of the Seven-teenth Conference on Computational Natural Lan-guage Learning. Sofia, Bulgaria, pages 183–192.

Yoav Artzi. 2013. Cornell SPF: Cornell Semantic Pars-ing Framework. arXiv:1311.3011 [cs.CL] .

Yoav Artzi, Kenton Lee, and Luke Zettlemoyer. 2015.Broad-coverage CCG Semantic Parsing with AMR.In Proceedings of Empirical Methods on NaturalLanguage Processing. pages 1699–1710.

Jason Baldridge and Geert-Jan Kruijff. 2002. CouplingCCG and Hybrid Logic Dependency Semantics. InProceedings of Association for Computational Lin-guistics. pages 319–326.

Hannah Bast and Elmar Haussmann. 2015. More Ac-curate Question Answering on Freebase. In Pro-ceedings of ACM International Conference on Infor-

mation and Knowledge Management. pages 1431–1440.

Paul Bédaride and Claire Gardent. 2011. Deep Seman-tics for Dependency Structures. In Proceedings ofConference on Intelligent Text Processing and Com-putational Linguistics. pages 277–288.

Islam Beltagy. 2016. Natural Language Semantics Us-ing Probabilistic Logic. Ph.D. thesis, Departmentof Computer Science, The University of Texas atAustin.

Islam Beltagy, Stephen Roller, Pengxiang Cheng, Ka-trin Erk, and Raymond J Mooney. 2016. Represent-ing meaning with a combination of logical and dis-tributional models. Computational Linguistics .

Emily M. Bender, Dan Flickinger, Stephan Oepen,Woodley Packard, and Ann Copestake. 2015. Lay-ers of Interpretation: On Grammar and Composition-ality. In Proceedings of the 11th International Con-ference on Computational Semantics. Associationfor Computational Linguistics, London, UK, pages239–249.

Jonathan Berant, Andrew Chou, Roy Frostig, and PercyLiang. 2013. Semantic Parsing on Freebase fromQuestion-Answer Pairs. In Proceedings of Empiri-cal Methods on Natural Language Processing. pages1533–1544.

Jonathan Berant and Percy Liang. 2014. Semantic Pars-ing via Paraphrasing. In Proceedings of Associationfor Computational Linguistics. pages 1415–1425.

Jonathan Berant and Percy Liang. 2015. ImitationLearning of Agenda-Based Semantic Parsers. Trans-actions of the Association for Computational Lin-guistics 3:545–558.

Antoine Bordes, Nicolas Usunier, Sumit Chopra, andJason Weston. 2015. Large-scale simple ques-tion answering with memory networks. CoRRabs/1506.02075.

Johan Bos, Stephen Clark, Mark Steedman, James R.Curran, and Julia Hockenmaier. 2004. Wide-Coverage Semantic Representations from a CCGParser. In Proceedings of International Conferenceon Computational Linguistics. pages 1240–1246.

Philipp Cimiano, Vanessa Lopez, Christina Unger,Elena Cabrio, Axel-Cyrille Ngonga Ngomo, andSebastian Walter. 2013. Multilingual question an-swering over linked data (QALD-3): Lab overview.In Information Access Evaluation. Multilinguality,Multimodality, and Visualization. Springer, Valencia,Spain, volume 8138.

Michael Collins. 2002. Discriminative Training Meth-ods for Hidden Markov Models: Theory and Exper-iments with Perceptron Algorithms. In Proceedingsof Empirical Methods on Natural Language Process-ing. pages 1–8.

Ronan Collobert, Jason Weston, Leon Bottou, MichaelKarlen, Koray Kavukcuoglu, and Pavel Kuks. 2011.Natural language processing (almost) from scratch.Journal of Machine Learning Research 12:2493–2537.

Ann Copestake, Dan Flickinger, Carl Pollard, andIvan A. Sag. 2005. Minimal Recursion Semantics:An Introduction. Research on Language and Com-putation 3(2-3):281–332.

Dick Crouch and Tracy Holloway King. 2006. Seman-tics via f-structure rewriting. In Proceedings of theLFG’06 Conference. CSLI Publications, page 145.

Mary Dalrymple, John Lamping, Fernando C. N.Pereira, and Vijay A. Saraswat. 1995. Linear Logicfor Meaning Assembly. In Proceedings of Computa-tional Logic for Natural Language Processing.

Kilian Evang and Johan Bos. 2016. Cross-lingualLearning of an Open-domain Semantic Parser. InProceedings of COLING 2016, the 26th Interna-tional Conference on Computational Linguistics.The COLING 2016 Organizing Committee, Osaka,Japan, pages 579–588.

Tobias Falke, Gabriel Stanovsky, Iryna Gurevych, andIdo Dagan. 2016. Porting an Open Information Ex-traction System from English to German. In Pro-ceedings of the 2016 Conference on Empirical Meth-ods in Natural Language Processing. Associationfor Computational Linguistics, Austin, Texas, pages892–898.

Federico Fancellu, Siva Reddy, Adam Lopez, and Bon-nie Webber. 2017. Universal Dependencies to Logi-cal Forms with Negation Scope. arXiv Preprint .

Jeffrey Flanigan, Sam Thomson, Jaime Carbonell,Chris Dyer, and Noah A. Smith. 2014. A Discrimi-native Graph-Based Parser for the Abstract MeaningRepresentation. In Proceedings of Association forComputational Linguistics. pages 1426–1436.

Claire Gardent and Laura Kallmeyer. 2003. SemanticConstruction in Feature-based TAG. In Proceedingsof European Chapter of the Association for Compu-tational Linguistics. pages 123–130.

Matt Gardner and Jayant Krishnamurthy. 2017. Open-Vocabulary Semantic Parsing with both Distribu-tional Statistics and Formal Knowledge. In Proceed-ings of Association for the Advancement of ArtificialIntelligence.

Jonas Groschwitz, Alexander Koller, and Christoph Te-ichmann. 2015. Graph parsing with s-graph gram-mars. In Proceedings of Association for Computa-tional Linguistics. pages 1481–1490.

Carolin Haas and Stefan Riezler. 2016. A Corpus andSemantic Parser for Multilingual Natural LanguageQuerying of OpenStreetMap. In Proceedings ofthe North American Chapter of the Association forComputational Linguistics: Human Language Tech-nologies. Association for Computational Linguistics,San Diego, California, pages 740–750.

Michael Hahn and Detmar Meurers. 2011. On deriv-ing semantic representations from dependencies: Apractical approach for evaluating meaning in learnercorpora. In Proceedings of the Int. Conference onDependency Linguistics (Depling 2011). Barcelona,pages 94–103.

Kejun Huang, Matt Gardner, Evangelos Papalex-akis, Christos Faloutsos, Nikos Sidiropoulos, TomMitchell, Partha P. Talukdar, and Xiao Fu. 2015.Translation Invariant Word Embeddings. In Pro-ceedings of the 2015 Conference on Empirical Meth-ods in Natural Language Processing. Lisbon, Portu-gal, pages 1084–1088.

Max Jakob, Markéta Lopatková, and Valia Kordoni.2010. Mapping between Dependency Structures andCompositional Semantic Representations. In Pro-ceedings of the Fifth International Conference onLanguage Resources and Evaluation.

Zhanming Jie and Wei Lu. 2014. Multilingual Seman-tic Parsing : Parsing Multiple Languages into Se-mantic Representations. In Proceedings of Inter-national Conference on Computational Linguistics.Dublin City University and Association for Com-putational Linguistics, Dublin, Ireland, pages 1291–1301.

Bevan Keeley Jones, Mark Johnson, and Sharon Gold-water. 2012. Semantic Parsing with Bayesian TreeTransducers. In Proceedings of the 50th AnnualMeeting of the Association for Computational Lin-guistics. Association for Computational Linguistics,Stroudsburg, PA, USA, pages 488–496.

Laura Kallmeyer and Aravind Joshi. 2003. Factor-ing predicate argument and scope semantics: Under-specified semantics with LTAG. Research on Lan-guage and Computation 1(1-2):3–58.

Ronald M Kaplan and Joan Bresnan. 1982. Lexical-functional grammar: A formal system for gram-matical representation. Formal Issues in Lexical-Functional Grammar pages 29–130.

Eliyahu Kiperwasser and Yoav Goldberg. 2016. Sim-ple and Accurate Dependency Parsing Using Bidi-rectional LSTM Feature Representations. Transac-tions of the Association for Computational Linguis-tics 4:313–327.

Jayant Krishnamurthy and Tom M. Mitchell. 2015.Learning a Compositional Semantics for Freebasewith an Open Predicate Vocabulary. Transactionsof the Association for Computational Linguistics3:257–270.

Tom Kwiatkowksi, Luke Zettlemoyer, Sharon Gold-water, and Mark Steedman. 2010. Inducing Prob-abilistic CCG Grammars from Logical Form withHigher-Order Unification. In Proceedings of Em-pirical Methods on Natural Language Processing.pages 1223–1233.

Tom Kwiatkowski, Eunsol Choi, Yoav Artzi, and LukeZettlemoyer. 2013. Scaling Semantic Parsers withOn-the-Fly Ontology Matching. In Proceedings ofEmpirical Methods on Natural Language Process-ing. pages 1545–1556.

Roger Levy and Galen Andrew. 2006. Tregex and tsur-geon: tools for querying and manipulating tree datastructures. In Proceedings of LREC. pages 2231–2234.

Mike Lewis and Mark Steedman. 2013. CombinedDistributional and Logical Semantics. Transactionsof the Association for Computational Linguistics1:179–192.

Percy Liang, Michael Jordan, and Dan Klein. 2011.Learning Dependency-Based Compositional Seman-tics. In Proceedings of Association for Computa-tional Linguistics. pages 590–599.

Fei Liu, Jeffrey Flanigan, Sam Thomson, NormanSadeh, and Noah A. Smith. 2015. Toward Ab-stractive Summarization Using Semantic Represen-tations. In Proceedings of North American Chap-ter of the Association for Computational Linguistics.pages 1077–1086.

Mitchell P. Marcus, Mary Ann Marcinkiewicz, andBeatrice Santorini. 1993. Building a large annotatedcorpus of English: The Penn Treebank. Computa-tional linguistics 19(2):313–330.

Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuz-man Ganchev, Keith Hall, Slav Petrov, HaoZhang, Oscar Täckström, Claudia Bedini, NúriaBertomeu Castelló, and Jungmee Lee. 2013. Univer-sal Dependency Annotation for Multilingual Pars-ing. In Proceedings of Association for Computa-tional Linguistics. pages 92–97.

Richard Montague. 1973. The Proper Treatment ofQuantification in Ordinary English. In K.J.J. Hin-tikka, J.M.E. Moravcsik, and P. Suppes, editors,Approaches to Natural Language, Springer Nether-lands, volume 49 of Synthese Library, pages 221–242.

Shashi Narayan and Claire Gardent. 2014. HybridSimplification using Deep Semantics and MachineTranslation. In Proceedings of Association for Com-putational Linguistics. pages 435–445.

Rebecca Nesson and Stuart M. Shieber. 2006. SimplerTAG Semantics Through Synchronization. In Pro-ceedings of the 11th Conference on Formal Gram-mar. Center for the Study of Language and Informa-tion, Malaga, Spain, pages 129–142.

Joakim Nivre, Marie-Catherine de Marneffe, Filip Gin-ter, Yoav Goldberg, Jan Hajic, Christopher D. Man-ning, Ryan McDonald, Slav Petrov, Sampo Pyysalo,Natalia Silveira, Reut Tsarfaty, and Daniel Zeman.2016. Universal Dependencies v1: A MultilingualTreebank Collection. In Proceedings of the Tenth In-ternational Conference on Language Resources and

Evaluation. European Language Resources Associa-tion (ELRA), Paris, France.

Joakim Nivre et al. 2016. Universal dependencies 1.3.LINDAT/CLARIN digital library at the Institute ofFormal and Applied Linguistics, Charles Universityin Prague.

Panupong Pasupat and Percy Liang. 2015. Composi-tional Semantic Parsing on Semi-Structured Tables.In Proceedings of Association for ComputationalLinguistics. pages 1470–1480.

Ellie Pavlick, Johan Bos, Malvina Nissim, CharleyBeller, Benjamin Van Durme, and Chris Callison-Burch. 2015. Adding Semantics to Data-DrivenParaphrasing. In Proceedings of Association forComputational Linguistics. pages 1512–1522.

Jeffrey Pennington, Richard Socher, and ChristopherManning. 2014. Glove: Global Vectors for WordRepresentation. In Proceedings of the 2014 Con-ference on Empirical Methods in Natural LanguageProcessing. Association for Computational Linguis-tics, Doha, Qatar, pages 1532–1543.

Barbara Plank, Anders Søgaard, and Yoav Goldberg.2016. Multilingual Part-of-Speech Tagging withBidirectional Long Short-Term Memory Models andAuxiliary Loss. In Proceedings of the 54th AnnualMeeting of the Association for Computational Lin-guistics. Berlin, Germany, pages 412–418.

Siva Reddy, Mirella Lapata, and Mark Steedman. 2014.Large-scale Semantic Parsing without Question-Answer Pairs. Transactions of the Association forComputational Linguistics 2:377–392.

Siva Reddy, Oscar Täckström, Michael Collins, TomKwiatkowski, Dipanjan Das, Mark Steedman, andMirella Lapata. 2016. Transforming DependencyStructures to Logical Forms for Semantic Parsing.Transactions of the Association for ComputationalLinguistics 4:127–140.

Tim Rocktäschel, Sameer Singh, and Sebastian Riedel.2015. Injecting Logical Background Knowledgeinto Embeddings for Relation Extraction. In Pro-ceedings of North American Chapter of the Asso-ciation for Computational Linguistics. pages 1119–1129.

Sebastian Schuster and Christopher D. Manning. 2016.Enhanced English Universal Dependencies: An Im-proved Representation for Natural Language Under-standing Tasks. In Proceedings of the Tenth Interna-tional Conference on Language Resources and Eval-uation. European Language Resources Association(ELRA), Paris, France.

Kiril Simov and Petya Osenova. 2011. Towards Min-imal Recursion Semantics over Bulgarian Depen-dency Parsing. In Proceedings of the InternationalConference Recent Advances in Natural LanguageProcessing 2011. RANLP 2011 Organising Commit-tee, Hissar, Bulgaria, pages 471–478.

Kathrin Spreyer and Anette Frank. 2005. ProjectingRMRS from TIGER Dependencies. In Proceedingsof the HPSG 2005 Conference. CSLI Publications.

Mark Steedman. 2000. The Syntactic Process. TheMIT Press.

Yu Su, Huan Sun, Brian Sadler, Mudhakar Srivatsa,Izzeddin Gur, Zenghui Yan, and Xifeng Yan. 2016.On Generating Characteristic-rich Question Sets forQA Evaluation. In Proceedings of the 2016 Con-ference on Empirical Methods in Natural LanguageProcessing. Austin, Texas, pages 562–572.

Ferhan Türe and Oliver Jojic. 2016. Simple and Ef-fective Question Answering with Recurrent NeuralNetworks. CoRR abs/1606.05029.

Lucy Vanderwende, Arul Menezes, and Chris Quirk.2015. An AMR parser for English, French, German,Spanish and Japanese and a new AMR-annotatedcorpus. In Proceedings of the 2015 Conference ofthe North American Chapter of the Association forComputational Linguistics: Demonstrations. Asso-ciation for Computational Linguistics, Denver, Col-orado, pages 26–30.

Aaron Steven White, Drew Reisinger, Keisuke Sak-aguchi, Tim Vieira, Sheng Zhang, Rachel Rudinger,Kyle Rawlins, and Benjamin Van Durme. 2016. Uni-versal Decompositional Semantics on Universal De-pendencies. In Proceedings of the 2016 Conferenceon Empirical Methods in Natural Language Pro-cessing. Association for Computational Linguistics,Austin, Texas, pages 1713–1723.

Yuk Wah Wong and Raymond Mooney. 2007. Learn-ing Synchronous Grammars for Semantic Parsingwith Lambda Calculus. In Proceedings of Associa-tion for Computational Linguistics. pages 960–967.

Kun Xu, Siva Reddy, Yansong Feng, Songfang Huang,and Dongyan Zhao. 2016. Question Answering onFreebase via Relation Extraction and Textual Evi-dence. In Proceedings of the Association for Compu-tational Linguistics. Association for ComputationalLinguistics, Berlin, Germany.

Xuchen Yao. 2015. Lean Question Answering overFreebase from Scratch. In Proceedings of NorthAmerican Chapter of the Association for Computa-tional Linguistics. pages 66–70.

Xuchen Yao and Benjamin Van Durme. 2014. Informa-tion Extraction over Structured Data: Question An-swering with Freebase. In Proceedings of Associa-tion for Computational Linguistics. pages 956–966.

Semih Yavuz, Izzeddin Gur, Yu Su, Mudhakar Srivatsa,and Xifeng Yan. 2016. Improving Semantic Pars-ing via Answer Type Inference. In Proceedings ofthe 2016 Conference on Empirical Methods in Natu-ral Language Processing. Association for Computa-tional Linguistics, Austin, Texas, pages 149–159.

Wen-tau Yih, Ming-Wei Chang, Xiaodong He, andJianfeng Gao. 2015. Semantic Parsing via StagedQuery Graph Generation: Question Answering withKnowledge Base. In Proceedings of Association forComputational Linguistics. pages 1321–1331.

John M. Zelle and Raymond J. Mooney. 1996. Learn-ing to Parse Database Queries Using Inductive LogicProgramming. In Proceedings of Association for theAdvancement of Artificial Intelligence. pages 1050–1055.

Luke S. Zettlemoyer and Michael Collins. 2005. Learn-ing to Map Sentences to Logical Form: StructuredClassification with Probabilistic Categorial Gram-mars. In Proceedings of Uncertainty in Artificial In-telligence. pages 658–666.

Universal Semantic Parsing: Supplementary Material

Siva Reddy† Oscar Täckström‡ Slav Petrov‡ Mark Steedman† Mirella Lapata††ILCC, School of Informatics, University of Edinburgh

‡ Google, Inc.siva.reddy@ed.ac.uk, {oscart, slav}@google.com, {steedman, mlap}@inf.ed.ac.uk

Abstract

This supplementary material to UniversalSemantic Parsing, provides and outline ofhow quantification can be incorporated inthe UDEPLAMBDA framework.

1 Universal Quantification

Consider the sentence Everybody wants to buy ahouse,1 whose dependency tree in the UniversalDependencies (UD) formalism is shown in Fig-ure 1(a). This sentence has two possible readings:either (1) every person wants to buy a differenthouse; or (2) every person wants to buy the samehouse. The two interpretations correspond to thefollowing logical forms:(1) ∀x.person(xa)→

[∃zyw.wants(ze)∧ arg1(ze,xa)∧buy(ye)∧xcomp(ze,ye)∧house(wa)∧ arg1(ze,xa)∧ arg2(ze,wa)] ;

(2) ∃w.house(wa)∧ (∀x.person(xa)→[∃zy.wants(ze)∧ arg1(ze,xa)∧buy(ye)∧xcomp(ze,ye)∧

arg1(ze,xa)∧ arg2(ze,wa)]) .

In (1), the existential variable w is in the scope ofthe universal variable x (i.e. the house is dependenton the person). This reading is commonly referredto as the surface reading. Conversely, in (2) theuniversal variable x is in the scope of the existentialvariable w (i.e. the house is independent of theperson). This reading is also called inverse reading.Our goal is to obtain the surface reading logicalform in (1) with UDEPLAMBDA. We do not aim toobtain the inverse reading, although this is possiblewith the use of Skolemization (Steedman, 2012).

In UDEPLAMBDA, lambda expressions forwords, phrases and sentences are all of theform λx. . . .. But from (1), it is clear that we needto express variables bound by quantifiers, e.g. ∀x,while still providing access to x for composition.This demands a change in the type system since the

1Example borrowed from Schuster and Manning (2016).

Everybody wants to buy a house

nsubj

xcomp dobj

detmark

root

(a) Original dependency tree.

Everybody wants to buy a house

nsubj

xcomp dobj

detmark

root

Ω Ωbind nsub

j

(b) Enhanced dependency tree.

Everybody wants to buy a house

nsubj:univ

xcomp dobj

detmark

root

Ω Ωbind nsub

j

(c) Enhanced dependency tree with universal quantification.

Figure 1: The dependency tree for Everybodywants to buy a house and its enhanced variants.

same variable cannot be lambda bound and quanti-fier bound—that is we cannot have formulas of theform λx . . .∀x . . .. In this material, we first derivethe logical form for the example sentence usingthe type system from our main paper (Section 1.1)and show that it fails to handle universal quantifi-cation. We then modify the type system slightlyto allow derivation of the desired surface readinglogical form (Section 1.2). This modified type sys-tem is a strict generalization of the original typesystem.2 Fancellu et al. (2017) present an elaboratediscussion on the modified type system, and how itcan handle negation scope and its interaction withuniversal quantifiers.

2Note that this treatment has yet to be added to ourimplementation, which can be found at https://github.com/sivareddyg/udeplambda.

mailto:siva.reddy@ed.ac.ukmailto:oscart@google.commailto:slav@google.commailto:steedman@inf.ed.ac.ukmailto:mlap@inf.ed.ac.ukhttps://github.com/sivareddyg/udeplambdahttps://github.com/sivareddyg/udeplambda

1.1 With Original Type System

We will first attempt to derive the logical form in (1)using the default type system of UDEPLAMBDA.Figure 1(b) shows the enhanced dependency treefor the sentence, where BIND has been introducedto connect the implied nsubj of buy (BIND is ex-plained in the main paper in Section 3.2). Thes-expression corresponding to the enhanced tree is:(nsubj (xcomp wants (mark

(nsubj (dobj buy (det house a)) Ω) to))(BIND everybody Ω)) .

With the following substitution entries,wants, buy ∈ EVENT;everybody, house ∈ ENTITY;a, to ∈ FUNCTIONAL;Ω = λx.EQ(x,ω);nsubj= λ f gx.∃y. f (x)∧g(y)∧ arg1(xe,ya);dobj= λ f gx.∃y. f (x)∧g(y)∧ arg2(xe,ya);xcomp= λ f gx.∃y. f (x)∧g(y)∧xcomp(xe,ya);mark ∈ HEAD;BIND ∈ MERGE,

the lambda expression after composition becomes:λz. ∃xywv.wants(ze)∧ everybody(xa)∧ arg1(ze,xa)∧ EQ(x,ω)∧buy(ye)∧xcomp(ze,ye)∧ arg1(ye,va)∧ EQ(v,ω)∧ arg1(xe,ya)∧house(wa)∧ arg2(ye,wa) .

This expression encodes the fact that x and v arein unification, and can thus be further simplified to:(3) λz.∃xyw.wants(ze)∧ everybody(xa)∧ arg1(ze,xa)

∧ buy(ye)∧xcomp(ze,ye)∧ arg1(ye,xa)∧ arg1(xe,ya)∧house(wa)∧ arg2(ye,wa) .

However, the logical form (3) differs from thedesired form (1). As noted above, UDEPLAMBDAwith its default type, where each s-expression musthave the type η = Ind×Event→ Bool, cannothandle quantifier scoping.

1.2 With Higher-order Type System

Following Champollion (2010), we make a slightmodification to the type system. Instead of usingexpressions of the form λx. . . . for words, we useeither λ f .∃x. . . . or λ f .∀x. . . ., where f has type η.As argued by Champollion, this higher-order formmakes quantification and negation handling soundand simpler in Neo-Davidsonian event semantics.Following this change, we assign the followinglambda expressions to the words in our examplesentence:everybody = λ f .∀x.person(x)→ f (x) ;wants = λ f .∃x.wants(xe)∧ f (x) ;to = λ f .TRUE ;buy = λ f .∃x.buy(xe)∧ f (x) ;a = λ f .TRUE ;house = λ f .∃x.house(xa)∧ f (x) ;Ω = λ f . f (ω) .

Here everybody is assigned universal quantifiersemantics. Since the UD representation does notdistinguish quantifiers, we need to rely on a small(language-specific) lexicon to identify these. Toencode quantification scope, we enhance the la-bel nsubj to nsubj:univ, which indicates thatthe subject argument of wants contains a universalquantifier, as shown in Figure 1(c).

This change of semantic type for words and s-expressions forces us to also modify the seman-tic type of dependency labels, in order to obeythe single-type constraint of DEPLAMBDA (Reddyet al., 2016). Thus, dependency labels will nowtake the form λPQ f . . . ., where P is the parent ex-pression, Q is the child expression, and the returnexpression is of the form λ f . . . .. Following thischange, we assign the following lambda expres-sions to dependency labels:nsubj:univ= λPQ f .Q(λy.P(λx. f (x)∧ arg1(xe,ya))) ;nsubj= λPQ f .P(λx. f (x)∧Q(λy.arg1(xe,ya))) ;dobj= λPQ f .P(λx. f (x)∧Q(λy.arg2(xe,ya))) ;xcomp= λPQ f .P(λx. f (x)∧Q(λy.xcomp(xe,ya))) ;det, mark= λPQ f .P( f ) ;BIND = λPQ f .P(λx. f (x)∧Q(λy.EQ(y,x))) .

Notice that the lambda expression ofnsubj:univ differs from nsubj. In the for-mer, the lambda variables inside Q have widerscope over the variables in P (i.e. the universalquantifier variable of everybody has scope over theevent variable of wants) contrary to the latter.

The new s-expression for Figure 1(c) is(nsubj:univ (xcomp wants (mark

(nsubj (dobj buy (det house a)) Ω) to))(BIND everybody Ω)) .

Substituting with the modified expressions, andperforming composition and simplification leads tothe expression:(6) λ f .∀x .person(xa)→

[∃zyw. f (z)∧wants(ze)∧ arg1(ze,xa)∧buy(ye)∧ xcomp(ze,ye)∧ house(wa)∧ arg1(ze,xa)∧ arg2(ze,wa)] .

This expression is identical to (1) except for theoutermost term λ f . By applying (6) to λx.TRUE,we obtain (1), which completes the treatment ofuniversal quantification in UDEPLAMBDA.

ReferencesLucas Champollion. 2010. Quantification and negation

in event semantics. Baltic International Yearbook ofCognition, Logic and Communication 6(1):3.

Federico Fancellu, Siva Reddy, Adam Lopez, and Bon-nie Webber. 2017. Universal Dependencies to Logi-cal Forms with Negation Scope. arXiv Preprint .

Siva Reddy, Oscar Täckström, Michael Collins, TomKwiatkowski, Dipanjan Das, Mark Steedman, andMirella Lapata. 2016. Transforming DependencyStructures to Logical Forms for Semantic Parsing.Transactions of the Association for ComputationalLinguistics 4:127–140.

Sebastian Schuster and Christopher D. Manning. 2016.Enhanced English Universal Dependencies: An Im-

proved Representation for Natural Language Under-standing Tasks. In Proceedings of the Tenth Interna-tional Conference on Language Resources and Eval-uation. European Language Resources Association(ELRA), Paris, France.

Mark Steedman. 2012. Taking Scope - The Natural Se-mantics of Quantifiers. MIT Press.