A Tool for Constructing A Tool for Constructing Syntax-Directed EditorsSyntax-Directed Editors

Yung-Shen Chang and Nai-Wei LinYung-Shen Chang and Nai-Wei LinDepartment of Computer Science and Information EngineerDepartment of Computer Science and Information Engineer

ingingNational Chung-Cheng UniversityNational Chung-Cheng University

Taiwan, R.O.C.Taiwan, R.O.C.

22

OutlineOutline

IntroductionIntroduction A generator for incremental parsersA generator for incremental parsers A simple interface to incremental A simple interface to incremental

parsersparsers Conclusions and future workConclusions and future work

33

Typical Program Development Typical Program Development Cycle Cycle

Program editing and program Program editing and program compilation are separated. compilation are separated.

It is more efficient if we can do program It is more efficient if we can do program editing and program compilation at the editing and program compilation at the same time.same time.

Edit Compile

Found Bugs

Edit Compile

…

Found Bugs

44

Syntax-Directed EditorsSyntax-Directed Editors

Syntax-directed editors have the Syntax-directed editors have the ability to perform program editing ability to perform program editing and program compilation at the and program compilation at the same time.same time.

Syntax-directed editors have become Syntax-directed editors have become an integral feature for modern an integral feature for modern integrated development integrated development environment.environment.

55

Incremental ParsingIncremental Parsing

The main capability of syntax-directed The main capability of syntax-directed editors is incremental parsing.editors is incremental parsing.

Incremental parsing has the ability to Incremental parsing has the ability to only parse the only parse the modifiedmodified portion of a portion of a program.program.

This ability allows us to interleave This ability allows us to interleave program editing and program program editing and program compilation.compilation.

66

ContributionsContributions

A generator for incremental parsers A generator for incremental parsers that is based on Bison.that is based on Bison.

A simple interface to incremental A simple interface to incremental parsers to facilitate the integration of parsers to facilitate the integration of editors and incremental parsers to editors and incremental parsers to form syntax-directed editors.form syntax-directed editors.

77

BisonBison

A popular generator for batch parsers A popular generator for batch parsers that converts an LALR(1) grammar that converts an LALR(1) grammar definition into a parser written in C.definition into a parser written in C.

Grammar definition

(sample.y)

Parser

(sample.tab.c)

Bison

88

OutlineOutline

IntroductionIntroduction A generator for incremental parsersA generator for incremental parsers A simple interface to incremental A simple interface to incremental

parsersparsers ConclusionsConclusions

99

Incremental Parsing Incremental Parsing AlgorithmsAlgorithms

The state matching incremental parsing The state matching incremental parsing algorithm (Larchêveque)algorithm (Larchêveque)– no need to change Bison parsing tableno need to change Bison parsing table

The sentential-form incremental parsing The sentential-form incremental parsing algorithm (Wagner)algorithm (Wagner)– need to change Bison parsing tableneed to change Bison parsing table

1010

Threaded Tree ParsingThreaded Tree Parsing

Use threaded tree data structure to Use threaded tree data structure to represent the parse treerepresent the parse tree

Threaded tree is a combination of Threaded tree is a combination of parse tree and parse stackparse tree and parse stack

All shift/reduce actions are operated All shift/reduce actions are operated on the threaded treeon the threaded tree

1111

Threaded Parse TreeThreaded Parse Tree

Each node in the threaded tree has Each node in the threaded tree has the following fieldsthe following fields– symbolsymbol– childrenchildren– statestate– threadingthreading

1212

Shift ActionShift Action

Shift symbol Shift symbol M M at at state state SS1.1. construct a node construct a node

N (M, S)N (M, S)

2.2. set set NN’s threading ’s threading to TOSto TOS

3.3. set TOS (Top of set TOS (Top of Stack) as Stack) as NN

N1

N (M, S)

TOS

1313

Reduce ActionReduce Action

Reduce symbol Reduce symbol MM at at State State SS with with kk children children1.1. construct a node construct a node N (M, S)N (M, S)

2.2. collect collect kk nodes from TOS nodes from TOS

3.3. construct connections construct connections between between NN and its and its childrenchildren

4.4. set set NN’s threading to ’s threading to leftmost node of its leftmost node of its childrenchildren

5.5. set TOS as set TOS as NN

N1

N(M,S)

s1 r2

TOS

M →→ sr

1414

An ExampleAn Example

S1

f2 f6F3

G4

g6

H7

h9

BOS

Input= fghf

Action:

Shift f2

Shift g6

Reduce G4

Shift h9

Reduce H7

Reduce F3

Shift f6

Reduce S1

TOS

1515

Incremental Threaded Tree Incremental Threaded Tree ParsingParsing

Split previous Split previous parse tree into parse tree into three parts three parts x y zx y z

xx and and z z are are unmodified partsunmodified parts

yy is the modified is the modified partpart

Let Let y’y’ be the be the modifying contentmodifying content

x y z

x, z =unmodified

y = modified

1616

Incremental Threaded Tree Incremental Threaded Tree ParsingParsing

1.1. Finding initial configuration Finding initial configuration ((xx))

2.2. Parse exhaustively Parse exhaustively ((y’y’))

3.3. Finding optimal grafting point from Finding optimal grafting point from candidates candidates ((zz))

4.4. Perform node graftingPerform node grafting

1717

Finding CandidatesFinding Candidates

A grafting point must be a common A grafting point must be a common ancestor of ancestor of y’y’ and and yy

We start searching from nearest We start searching from nearest common ancestor (NCA)common ancestor (NCA)

1818

Proper Grafting PointsProper Grafting Points

A candidate for a grafting point is an NCA candidate for a grafting point is an NCA A nn such that such that– terminal successor of terminal successor of nn = current lookahead = current lookahead– symbol of symbol of nn = symbol of the TOS = symbol of the TOS– predecessor of predecessor of nn = predecessor of the TOS = predecessor of the TOS

1919

An ExampleAn Example

S1

BOS0

f2

g5

G4

h9

H7

F3 f6

g8

G4

TOS NCA

lookahead

Grafting Point

F3

= deleted node

x: g5

y: g8

z: h9, f6

2020

Incremental Parser ModulesIncremental Parser Modules

node

Parsing table reader

tree

incparser

parser

manage

read

Determine what to do

Perform action (shift/reduce)

read

…

node

node

= module

= file

1

2

3

4 Perform grafting if find suitable grafting point

input

Bison Parsing Table

2121

OutlineOutline

IntroductionIntroduction A generator for incremental parsersA generator for incremental parsers A simple interface to incremental A simple interface to incremental

parsersparsers ConclusionsConclusions

2222

Interface to Incremental Parser Interface to Incremental Parser

Editor calls Editor calls incparserincparser to get service from to get service from incremental parserincremental parser

Editor passes parameters Editor passes parameters beginbegin, , endend, and , and deltadelta

incparserincparser will store the parsing result in will store the parsing result in variable variable errerr

voidvoid

incparserincparser

((POSPOS begin, begin, POSPOS end, end, constconst stringstring& delta, & delta, ERRERR& err);& err);

2323

Flow of an Incremental Parsing Flow of an Incremental Parsing SessionSession

incparserinterfaceeditor

Write delta

Return parse result

Return parse result

Read input from temporary file

2

1 3

4

56

Call interface

Wrapper call incparser

Temporary file

2424

Proper Timing to Trigger Proper Timing to Trigger Incremental ParsingIncremental Parsing

By timerBy timer By each key pressBy each key press By editing modelBy editing model

2525

Editing ModelEditing Model Editing is a sequence of key pressesEditing is a sequence of key presses Classify key presses into two groupsClassify key presses into two groups

– modikeysmodikeys – – which will cause content changewhich will cause content change– non-modikeysnon-modikeys – – which which won’twon’t cause content cause content

changechange Editing causes a state change between Editing causes a state change between

pressing pressing modikeysmodikeys and and non-modikeysnon-modikeys Editor should remember BEGIN and END Editor should remember BEGIN and END

position when a state change occursposition when a state change occurs

2626

Special KeysSpecial Keys BS (Backspace)/DEL are keys in BS (Backspace)/DEL are keys in

modikeys modikeys that need special treatmentsthat need special treatments BS might change the BEGIN positionBS might change the BEGIN position DEL might change the END positionDEL might change the END position Editor should perform appropriate Editor should perform appropriate

action when user pressing BS/DELaction when user pressing BS/DEL We use a counter to maintain how We use a counter to maintain how

many many modikeysmodikeys are pressed (except are pressed (except BS/DEL)BS/DEL)

2727

An ExampleAn Example

Assume that the cursor is at the Assume that the cursor is at the position between “ma”, “in”, the position between “ma”, “in”, the editor is at editor is at non-modikeynon-modikey state state

User performs the following User performs the following modificationsmodifications– press BS two timespress BS two times– press DEL two timespress DEL two times– key in “foo_function”key in “foo_function”– key in any key in any non-modikeysnon-modikeys

2828

An Example (cont.)An Example (cont.)

Initial situationInitial situation– cursor at (1, 7)cursor at (1, 7)– BEGIN and END is N/ABEGIN and END is N/A

Pressing BS two Pressing BS two timestimes– cursor at (1, 5)cursor at (1, 5)– BEGIN = (1, 5)BEGIN = (1, 5)– END = (1, 8)END = (1, 8)

}04

return 0;03

printf(“Hello, World\n”);02

int main (void) {01

}}04

return 0;03

printf(“Hello, World\n”);02

int in (void) {01

2929

An Example (cont.)An Example (cont.) Press DEL two timesPress DEL two times

– cursor at (1, 5)cursor at (1, 5)– BEGIN = (1, 5)BEGIN = (1, 5)– END = (1, 10)END = (1, 10)

}04

return 0;03

printf(“Hello, World\n”);02

int in (void) {01

}04

return 0;03

printf(“Hello, World\n”);02

int (void) {01

3030

An Example (cont.)An Example (cont.) Key in “Key in “foo_functionfoo_function””

– cursor at (1, 17)cursor at (1, 17)– BEGIN = (1, 5)BEGIN = (1, 5)– END = (1, 10)END = (1, 10)– DELTA = DELTA =

““foo_functionfoo_function”” Press any non-Press any non-

modikeys Call modikeys Call incparserincparser by by– BEGIN = (1, 5)BEGIN = (1, 5)– END = (1, 10)END = (1, 10)– DELTA = DELTA =

““foo_functionfoo_function””

}04

return 0;03

printf(“Hello, World\n”);02

int (void) {01

}04

return 0;03

printf(“Hello, World\n”);02

int foo_function (void) {01

3131

OutlineOutline

IntroductionIntroduction A generator for incremental parsersA generator for incremental parsers A simple interface to incremental A simple interface to incremental

parsersparsers ConclusionsConclusions

3232

ConclusionsConclusions We developed a generator for We developed a generator for

incremental parsers based on Bisonincremental parsers based on Bison We introduced a simple interface to We introduced a simple interface to

incremental parsers that facilitate incremental parsers that facilitate integration of an incremental parser integration of an incremental parser and an editor based on editing modeland an editor based on editing model

3333

Future WorkFuture Work A generator for incremental lexersA generator for incremental lexers A generator for incremental A generator for incremental

semantics analyzerssemantics analyzers A generator for syntax-directed A generator for syntax-directed

editorseditors