Claus Brabrand (ITU Copenhagen) & Jakob G. Thomsen (Aarhus University)DANSAS 2010 (In proc. of PPDP
2010)
Typed and Unambiguous Pattern Matching on Strings
using Regular Expressions
[http://xkcd.com/208/]
2
Main MessageFor regular expressions:Pattern matching
Precise syntax-directed ambiguity analysis
Typed mapping into a target language
3
Introduction & Motivation
Parsing dynamic input is an ubiquitous problem
URLs:
Log Files:
The solution is pattern matching
http://www.cs.au.dk/index.php?id=141&view=details
13/02/2010 66.249.65.107 get /support.html20/02/2010 42.116.32.64 post /search.html
protocol host path query-string
(list of key-value pairs)
4
Example Example (date):
Matching against string: yields:
"26/06/1992"day = 26 month = 06 year = 1992
[0-9]{1,2} "/" [0-9]{1,2} "/" [0-9]{4}
<day = [0-9]{1,2} > "/" <month = [0-9]{1,2} > "/" <year = [0-9]{4} >
5
Example (date):
String 2082010: day = 2 and month = 08 (ie. 2nd of
August) day = 20 and month = 8 (ie. 20th of
August)
Example
<day = [0-9]{1,2} > "/" <month = [0-9]{1,2} > "/" <year = [0-9]{4} >
<day = [0-9]{1,2} > <month = [0-9]{1,2} > <year = [0-9]{4} >
6
Expressive (enough)DeclarativeDecidable propertiesWell known
Why regular expressions?
7
.
Outline Our setup Regular Expressions:
The Recording Construction
Ambiguity: Disambiguation
Type Mapping Conclusion
8
Our setup<URL = [a-z]*>;...
url.rex
URL.java...
Compile (our tool)
Compile (javac)
URL.classFoo.class...
import URL;class Foo { ...}
Foo.java URL.javaFoo.java...
9
Outline Our setup Regular Expressions:
The Recording Construction
Ambiguity: Disambiguation
Type Mapping Conclusion
10
Regular Expressions Syntax:
Semantics:
where: L1 L2 is concatenation (i.e., { 1 2 | 1L1,
2L2 }) L* = i0 Li where L0 = { } and Li = L
Li-1
Usual extensions : Any character ”.” as c1|c2|...|cn,
ci Character ranges ”[a-z]” as
a|b|...|z Repetitions ”R{2,3}” as RR|
RRR
Recording Syntax:
” ” is a recording identifier (it "remembers" the substring it matches)
Semantics:
Example (simplified emails):
Matching against string:yields:
[a-z]+ "@" [a-z]+ ("." [a-z]+)*
user = "obama" domain = "whitehouse.gov"&
<user = > <domain = >
11
Related: "x as R" in XDuce; "x::R" in CDuce; and "x@R" in Scala and HaRP
12
Outline Our setup Regular Expressions:
The Recording Construction
Ambiguity: Disambiguation
Type Mapping Conclusion
13
Ambiguity Example from before
matched on the string “208” gives rise to: day = 2 and month = 08 (ie. 2nd of August) day = 20 and month = 8 (ie. 20th of August)
Multiple ways of matching => ambiguous
Problem: Concatenation
<day = [0-9]{1,2} > <month = [0-9]{1,2} >
2 0 8
day month
14
Ambiguity analysis Theorem:
R unambiguous iff NB: sound & complete !
Related work: [Brabrand+Giegerich+Møller’09]: Similar approach for context free grammars.[Book+Even+Greibach+Ott'71] and [Hosoya'03] for XDuce but indirectly via NFA, not directly (syntax-directed).
15
Outline Our setup Regular Expressions:
The Recording Construction
Ambiguity: Disambiguation
Type mapping Conclusion
16
2) Restriction: R1 - R2
L(R1 - R2) = L(R1) \ L(R2)
4) Default disambiguation: concat, choice, and star
are all left-biased (by default) !
(Our tool does this)
1) Manual rewriting: Always possible :-) Tedious :-( Error-prone :-( Not structure-preserving :-(
3) Disambiguators: Three basic operators choice:'|L', '|R' concat: 'L', 'R' star: '*L', '*R'
Disambiguation
<foo = a > | <bar = a* >is rewritten to <foo = a > | <bar = |aaa* >
<foo = a > | <bar = a* >using restriction <foo = a > | <bar = a*-a >
<foo = a > | <bar = a* >using restriction we get <foo = a > |L <bar = a* >
<foo = a > | <bar = a* >no need to rewrite
Related work: [Vansummeren'06] but with global, not local disambiguation
17
Outline Our setup Regular Expressions:
The Recording Construction
Ambiguity: Disambiguation
Type Mapping Conclusion
Type Mapping Our date example
Type of the recordings day, month, and year? Strings (=> many type casts) Infer the type
<day = [0-9]{2} > "/" <month = [0-9]{2} > "/”<year = [0-9]{4} >
18
Type Mapping A recording has three type components:
a linguistic type (language of the recording - maps to String, int, float, etc).
a structural type (nested recordings – maps to (nested) classes).
a type modifier (maps to lists).
19
Related work: Exact type inference in XDuce & CDuce(soundness+completeness proof in [Vansummeren'06])but not for stand-alone and non-intrusive usage (Java)
20
Type Mapping ExamplePerson = <name = > " (" <age = > ")"[a-z]+ [0-9]+
class Person { // auto-generated String name; int age; static Person match(String s) { ... } public String toString() { ... }}
compile(our tool)
String s = "obama (48)";
Person p = Person.match(s);print(p.name + " is " + p.age + "y old");
Usage
21
ConclusionRegular expressions are alive and well. This paper: Used for pattern matching Precise ambiguity analysis Type mappingFuture work: improve performance, subtype of
recordings "trade (excess) expressivity for
safety+simplicity” Thank you. Questions?
22
Abstract Syntax Trees (ASTs)
23
Ambiguity Definition:
R ambiguous iffT,T'ASTR: T T' ||T|| = ||T'||
where ||||: AST * (the flattening) is:
TR
T'R'
=
24
Characterization of Ambiguity
Theorem: R unambiguous iff
NB: sound & complete !
R* = | RR*
25
Type Inference Type Inference:
R : (L,S)