Enhancing the Web With End-User ProgrammingTak Yeon Lee, Ben Bederson

Outline Problem

Inefficiencies on the Web

Challenge

Building extension is very hard

Our goalExpressive and Easy Programming

Env.

- Improves a wide range of tasks

- No programming skills required

Snapshot

ANY WEBSITE

Inefficiencies on the Web

1. Missing relevant information

“I want to see the pictures.”

Detail page

“Is it a good value?”

Kelly’s Blue book

“How far is this place?”

Google Map

“Is the model reliable?”

Auto forums

2. Removing unwanted information

AD

AD

2. Removing unwanted information

3. Repetitive Operations

Howe-TheRiseOfCrowdsourcing-2006.pdf

Kittur-CrowdsourcingUserStudies-2008.pdf

Brabham-CrowdsourcingAsAModelFor-2008.pdf

Alonso-CrowdsourcingForRelevanceEvaluation-2008.pdf

Kittur-08-001.pdf

60947(1).pdf

Alonso2008.pdf

10.1.1.154.444.pdf

“I want to download all PDFs named as [first author]-[title]-[year].pdf ”

1. Finding mistakes

2. Add comments

3. Calculate the total points

4. Submit to grade server

Grading Procedure

Takes 4-8 hoursto grade 70

students

4. Finding a needle in a haystack

MyDouble.zero

MyDouble.zero

MyDouble.zero

Grading with an extension

Summary of Potential Mistakes

1. Click to scroll to the code line of the mistake

2. Click to add predefined comment

3. Setting partial credit

4. Report total score to another server

“Previously it took several hours, but now I can finish in 40 minutes.”

- Lecturer

“Three times faster than before, at least!”

-Grading TA

Browser Extensions add new features and personalize experience. Over 50,000 on Chrome Web Store

AdBlock>15 million users

Photo Zoom>2 million users

Print AllE-mails in a folder

Building an extension requires• Programming knowledge (HTML, CSS, and Ajax)

• Time and effort of development• The grading example took >40 hours

OUR GOAL IS TO BUILD

Expressive and Easy programming env.

The toolkit should support tasks like...• Automating repetitive jobs• Integrating data from multiple sources• Custom filtering• Information Summary

We defined a domain language model having

Pick, Change, Add operations

as generic functionalities of the extensions.

Pick extract information from web pages or elements.

Pick Change Add model

The rise of crowdsourcing.pdfCrowdsourcing user studies...Crowdsourcing as a model ...

http://sistemas-humano.com...http://dl.acm.org/citation.cfm?id=1357127http://con.sagepub.com/...

<A> tags

URLs

Changemanipulates dataset.

Pick Change Add model

• Arithmetic operations• +, -, /, %, ...• sum, avg, count, ...

• String manipulation• concatenate, split, find,

replace, ...

• Filter• Sort

1

2

3

3

3

6

9

[a] [b]

[a] * [b]

Changemanipulates dataset.

Pick Change Add model

• Arithmetic operations• +, -, /, %, ...• sum, avg, count, ...

• String manipulation• join, split, find, replace, ...

• Filter• Sort

I

am a

boy

-

I-am a-boy

[a] [b]

Join [a] with [b]

Changemanipulates dataset.

Pick Change Add model

• Arithmetic operations• +, -, /, %, ...• sum, avg, count, ...

• String manipulation• join, split, find, replace, ...

• Filter• Sort

a.jpg

b.gif

c.png

png

c.png

[a] [b]

Filter [a] having [b]

Changemanipulates dataset.

Pick Change Add model

• Arithmetic operations• +, -, /, %, ...• sum, avg, count, ...

• String manipulation• join, split, find, replace, ...

• Filter• Sort

Toronto

Seoul

Paris

[a]

Sort [a]

Paris

Seoul

Toronto

Addmodify the original web pages.

Pick Change Add model

• Create new elements• image, text, button, ...

• Modify element’s style• Show / Hide, changing color, fonts, ...

• Triggering events• click, ...

• Type text

[1]. Pick ad banner elements

[2]. Hide [1]

AD

AD

Pick Change Add model

e.g. Removing AD banners

e.g. Downloading files with custom name pattern

The rise of crowdsourcing.pdf

Crowdsourcing user studies with Mechanical Turk.pdf

Crowdsourcing as a model for.pdf

Crowdsourcing for relevance evaluation.pdf

[3]. Pick download links

[1]. Pick title text

[2]. Merge [1] with name pattern “.pdf”

[4]. Add [2] to ‘download’ attribute of [3]

Pick Change Add model

OUR GOAL IS TO BUILD

Expressive and Easy programming env.

• Requires no programming knowledge

• Programming-by-Example techniques• Telling computer what I want, without

worrying how to do.

?

?

PROGRAM

OUTPUT

INPUT

Traditional Programming Programming-by-Example

int sum=0;for(int v in input) {

sum=sum+v;} return sum;

1, 2, 3

6

!

Programming-by-Example technique

Paris

NewYork

Seoul

Tokyo

.jpg

Paris.jpg

NewYork.jpg

Seoul.jpg

Tokyo.jpg

[1] [2]

Merge [1] with [2]

Programming-by-Example technique

Programs with multiple inputse.g. Adding suffix to strings

Input

Output

Paris, France

Istanbul, Turkey

Seoul, Korea

Osaka, Japan

Korea

Japan

[1] [2]

Seoul, Korea

Tokyo, Japan

2

Filter [1] exist in [2]

[3]

Count [3]

Programming-by-Example technique

Multi-step programming e.g. Counting items of [1] that exist in [2]

London, UK

LA, US

Beizing, China

Toronto, Canada

UK

Canada

China

[1] [2]

London, UK

Beizing, China

Toronto, Canada

3

[3]

Count [3]

Programming-by-Example technique

Multi-step programming e.g. Counting items of [1] that exist in [2]

Filter [1] exist in [2]

Demo time

ConclusionA new opportunity for end-users to personalise the web by creating their own extensions.

Programming-by-Example techniques with Pick, Change, Add model

Future work includes• Usability Tune up • Evaluation of the system’s learnability

Please email us if you have an idea for extensions

Tak Yeon Lee <tylee@umd.edu>