understanding software - centrum wiskunde & informatica · repl metrics libraries math,...

62
Understanding Software Paul Klint #UnderstandingSoftware Wednesday, June 20, 2012

Upload: others

Post on 01-Aug-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Understanding SoftwarePaul Klint

#UnderstandingSoftwareWednesday, June 20, 2012

Page 2: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Where we are coming from(circa 1800)

Wednesday, June 20, 2012

Page 3: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

http://youtu.be/NSjmFD6Q7hw

Jacquard Looms atFamily HeirloomWeavers in Red

Lion, PA.

Wednesday, June 20, 2012

Page 4: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Using this development environment

Wednesday, June 20, 2012

Page 5: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Applications

Preparing input for Jacquard’s loom

Wednesday, June 20, 2012

Page 6: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

For the rest of this story ...

Wednesday, June 20, 2012

Page 7: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

For the rest of this story ...

... visit the Turing’s Legacy

exhibition during the breaks or reception

Wednesday, June 20, 2012

Page 8: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Where you can also see the Lego Turing Machine

Credits:Construction: Jeroen van den Bos, Davy Landman

Film: Andre Theelen

http://www.legoturingmachine.org/

Wednesday, June 20, 2012

Page 9: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Fast forward to today ...

Wednesday, June 20, 2012

Page 10: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Integrated Development Environments for All

IDE lovers IDE haters

Emacshaters

Vimhaters

Wednesday, June 20, 2012

Page 11: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

We have created a lot of software

Wednesday, June 20, 2012

Page 12: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

~900.000.000.000 LOC

Source:Booch, 2005

Also: Capers Jones

Wednesday, June 20, 2012

Page 13: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

~900.000.000.000 LOC

Source:Booch, 2005

Also: Capers Jones

Cumulative lines of code

Wednesday, June 20, 2012

Page 14: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

~900.000.000.000 LOC

Source:Booch, 2005

Also: Capers Jones

Nearly One (European) Billion Lines!

Cumulative lines of code

Wednesday, June 20, 2012

Page 15: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Howmuch

isthat?

Wednesday, June 20, 2012

Page 16: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Source: US Geological Survey

Howmuch

isthat?

Wednesday, June 20, 2012

Page 17: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Source: US Geological Survey

All water

Howmuch

isthat?

Wednesday, June 20, 2012

Page 18: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Source: US Geological Survey

All water

All drinkable water

Howmuch

isthat?

Wednesday, June 20, 2012

Page 19: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Source: US Geological Survey

All water

All drinkable water

All drinkable and accessible water

Howmuch

isthat?

Wednesday, June 20, 2012

Page 20: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Source: US Geological Survey

All water

All drinkable water

All drinkable and accessible water

Requires 100L per LOC to fill the smallest

sphere

Howmuch

isthat?

Wednesday, June 20, 2012

Page 21: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Source: US Geological Survey

All water

All drinkable water

All drinkable and accessible water

Requires 100L per LOC to fill the smallest

sphere

Howmuch

isthat?

Nice picture, but ...

Wednesday, June 20, 2012

Page 22: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

How much is that?

Wednesday, June 20, 2012

Page 23: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

How much is that?Print 1 billion lines in point size 10

Wednesday, June 20, 2012

Page 24: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

How much is that?Print 1 billion lines in point size 10

Wednesday, June 20, 2012

Page 25: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

How much is that?

0.8 times distance to the moon

Print 1 billion lines in point size 10

Wednesday, June 20, 2012

Page 26: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

How much is that?

0.8 times distance to the moon

Wrap Planet Earth 8 times

Print 1 billion lines in point size 10

Wednesday, June 20, 2012

Page 27: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

How can we understand and manage all that

software?

Wednesday, June 20, 2012

Page 28: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

How can we understand and manage all that

software?

Software Engineering

Wednesday, June 20, 2012

Page 29: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Credit: Adrian Kuhn

Word cloud of allpapers at International

Conference on Software Engineering

(ICSE) 2012

Wednesday, June 20, 2012

Page 30: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Software EngineeringDifferent Perspectives

Foundation

Application

Wednesday, June 20, 2012

Page 31: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Software EngineeringTraditional Academic View

Foundation

Application

How can we build new software?

Wednesday, June 20, 2012

Page 32: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Software EngineeringRealistic View

Construction EvolutionWednesday, June 20, 2012

Page 33: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Software EngineeringRealistic View

Construction Evolution

Construction costs are just 1-10% of costs of total lifecycle

Wednesday, June 20, 2012

Page 34: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Software EngineeringRealistic View

Construction Evolution

Construction costs are just 1-10% of costs of total lifecycle

How can we understand and improveexisting software?

Wednesday, June 20, 2012

Page 35: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Software EngineeringEvolution of Research at CWI/SWAT

Foundation

Application

Construction EvolutionWednesday, June 20, 2012

Page 36: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

ASF+SDF

Wednesday, June 20, 2012

Page 37: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

ASF+SDF

DSL

Wednesday, June 20, 2012

Page 38: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

ASF+SDF

DSL COBOL

Wednesday, June 20, 2012

Page 39: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

ASF+SDF

DSL COBOL

Algebraic SpecificationTerm RewritingGeneralized ParsingASF+SDF Meta-Environment

Wednesday, June 20, 2012

Page 40: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

ASF+SDF

DSL COBOL

Algebraic SpecificationTerm RewritingGeneralized ParsingASF+SDF Meta-Environment

Principles of DSL designRISLA: Interest-based products

Wednesday, June 20, 2012

Page 41: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

ASF+SDF

DSL COBOL

Algebraic SpecificationTerm RewritingGeneralized ParsingASF+SDF Meta-Environment

Principles of DSL designRISLA: Interest-based products

DocGenCluster analysisCOBOL transformationSoftware Improvement Group

Wednesday, June 20, 2012

Page 42: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Triggers for ChangeTechnology Usability Application

domains

New ideas did not fit in old approach

Students without formal background

Analysis & transformation of

real languagesEnable many

experiments and easy extension

Usable for real programmers Repository mining

Connect to external libraries

IDE features for DSls

New DSLs

Efficient built-in datatypes

Error reporting & recovery

Software Visualization

Wednesday, June 20, 2012

Page 43: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Current Goals

• “One-stop-shop” for

• Meta-programming

• Meta-data analysis

• Visualization

• Lab infrastructure

• Transfer medium http://www.rascal-mpl.org

Wednesday, June 20, 2012

Page 44: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

Rascal

Wednesday, June 20, 2012

Page 45: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

Rascal

DSL

Wednesday, June 20, 2012

Page 46: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

Rascal

DSLJava,PHP,Lua, ...

Wednesday, June 20, 2012

Page 47: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

Rascal

DSLJava,PHP,Lua, ...

Generalized ParsingPattern MatchingTree traversalRelational CalculusJava BridgeEclipse Plugin

Wednesday, June 20, 2012

Page 48: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

Rascal

DSLJava,PHP,Lua, ...

Generalized ParsingPattern MatchingTree traversalRelational CalculusJava BridgeEclipse Plugin

Forensics (NFI)Financial Auditing (PwC)Gaming (IC3D)GPUs (VU)HPC (Bergen)

Wednesday, June 20, 2012

Page 49: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Software EngineeringEvolution of Research at CWI/SWAT

Rascal

AnalysisRefactoringRepository Mining

DSLJava,PHP,Lua, ...

Generalized ParsingPattern MatchingTree traversalRelational CalculusJava BridgeEclipse Plugin

Forensics (NFI)Financial Auditing (PwC)Gaming (IC3D)GPUs (VU)HPC (Bergen)

Wednesday, June 20, 2012

Page 50: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Rascal Results so far

Technology Usability Application domains

Types, locations, resources, visit, pattern-directed invocation, ...

Layered design: learn on demand

Java, PHP, Lua analysis & refactoring

New GLL parsing Eclipse IDE features

Repository mining

Native support for visualization REPL Metrics

Libraries Math, Statistics, JDBC, SVN, ...

Interactive Tutor

Various DSLs

Wednesday, June 20, 2012

Page 51: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

These considerations motivate today’s topic:

Understanding Software

Wednesday, June 20, 2012

Page 52: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Understanding Software

Wednesday, June 20, 2012

Page 53: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Understanding Software

Evolution

Wednesday, June 20, 2012

Page 54: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Understanding Software

Ecosystems Evolution

Wednesday, June 20, 2012

Page 55: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Understanding Software

Ecosystems Evolution

Business

Wednesday, June 20, 2012

Page 56: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Understanding Software

Ecosystems

Mark v.d. Brand: Meta-ModelingArie van Deursen: TestingMike Godfrey: Artifact ProvenanceJurgen Vinju: Meta-Programming Stéphane Ducasse: Visualization

Evolution

Business

Wednesday, June 20, 2012

Page 57: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Understanding Software

Krzysztof Czarnecki: VariabilityRalf Lämmel: Technological Spaces

Ecosystems

Mark v.d. Brand: Meta-ModelingArie van Deursen: TestingMike Godfrey: Artifact ProvenanceJurgen Vinju: Meta-Programming Stéphane Ducasse: Visualization

Evolution

Business

Wednesday, June 20, 2012

Page 58: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Foundation

Application

Construction Evolution

Understanding Software

Krzysztof Czarnecki: VariabilityRalf Lämmel: Technological Spaces

Ecosystems

Mark v.d. Brand: Meta-ModelingArie van Deursen: TestingMike Godfrey: Artifact ProvenanceJurgen Vinju: Meta-Programming Stéphane Ducasse: Visualization

Evolution

BusinessOege de Moor: Business IntelligenceJoost Visser: Benchmarking Quality

Wednesday, June 20, 2012

Page 59: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Word cloud of today’s topicsWednesday, June 20, 2012

Page 60: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

Incomplete list of former SWAT and UvA colleagues

Wednesday, June 20, 2012

Page 61: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

The Team

Wednesday, June 20, 2012

Page 62: Understanding Software - Centrum Wiskunde & Informatica · REPL Metrics Libraries Math, Statistics, JDBC, SVN, ... Interactive Tutor Various DSLs Wednesday, June 20, 2012. These considerations

EnjoyUnderstanding Software

Wednesday, June 20, 2012