virtual separation of concerns
DESCRIPTION
An overview of my work on virtual separation of concerns (improvement of preprocessors/conditional compilation for product line engineering)TRANSCRIPT
![Page 1: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/1.jpg)
Virtual Separation of Concerns
Toward preprocessors 2.0
Christian Kästner
Photo (cc) Horia Varlan
![Page 2: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/2.jpg)
Christian Kästner: Virtual Separation of Concerns 22
Software Product Lines & Features
• Set of related software systems (variants) in a domain
• Generated from common code base
• Variants distinguished in terms of features
![Page 3: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/3.jpg)
Christian Kästner: Virtual Separation of Concerns 33
Conditional Compilation with Preprocessors
● Annotated codefragments
● Variants with andwithout features
● Common for product-line implementationin industry
Preprocessors in C, C++, Java ME, Pascal, Erlang, C#, Visual Basic, …GPP, GNU M4, SPLET, Frames/XVCL, Gears, pure::variants
static int _rep_queue_filedone(...DB_ENV *dbenv;REP *rep;__rep_fileinfo_args *rfp; {
#ifndef HAVE_QUEUECOMPQUIET(rep, NULL);COMPQUIET(rfp, NULL);return (__db_no_queue_am(dbenv
#elsedb_pgno_t first, last;u_int32_t flags;int empty, ret, t_ret;
#ifdef DIAGNOSTICDB_MSGBUF mb;
#endif...
Excerpt from Oracle’s Berkeley DB
![Page 4: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/4.jpg)
Christian Kästner: Virtual Separation of Concerns 44
Objections / Criticism
“#ifdef considered harmful”“#ifdef hell”
Designed in the 70th and hardly evolved since
“preprocessor diagnostics are poor”
“is difficult to determine if the code being viewed is actually compiled into the system”
“programming errors are easy to make and difficult to detect”
“incomprehensible source texts”
“maintenance becomes a ‘hit or miss’ process”
“CPP makes maintenance difficult”
“source code rapidly becomes a maze”
![Page 5: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/5.jpg)
Christian Kästner: Virtual Separation of Concerns 55
Academia: Compositional Approaches
class Stack {void push(Object o) {
elementData[size++] = o;}...
}
class Stack {void push(Object o) {
elementData[size++] = o;}...
}
refines class Stack {void push(Object o) {
Lock l = lock(o);Super.push(o);l.unlock();
}...
}
refines class Stack {void push(Object o) {
Lock l = lock(o);Super.push(o);l.unlock();
}...
}
aspect Diagnostics {...
}
Base / Platform
Feature: Queue
Feature: Diagnosticaspect Diagnostics {
...}
class Stack {void push(Object o) {
Lock l = lock(o);elementData[size++] = o;l.unlock();
}...
}
class Stack {void push(Object o) {
Lock l = lock(o);elementData[size++] = o;l.unlock();
}...
}
Composition
ModuleComponentsFrameworks, Plug-insFeature-Modules / Mixin Layers / …Aspects / Subjects, Hyper/J
![Page 6: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/6.jpg)
Christian Kästner: Virtual Separation of Concerns 66
Agenda
• Problems and Advantages of Preprocessors• 4 Improvements
• Views• Visual Representation• Disciplined Annotations• Product-Line-Aware Type System
• Summary and Perspective
![Page 7: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/7.jpg)
Christian Kästner: Virtual Separation of Concerns 77
Problem 1: Lack of Separation of Concerns
• Scattered and tangled source code• Global search to understand a feature• Deleting obsolete feature as challenge• Difficult distributed development• Difficult reuse
Beispiel Berkeley DB: Feature Sperren mit 1835 LOC in 28 von 300 Dateienan 155 Stellen
![Page 8: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/8.jpg)
Christian Kästner: Virtual Separation of Concerns 88
Problem 2: Obfuscation
• Mixture of twolanguages(C und #ifdef, …)
• Control flow hard to follow
• Additional line breaks destroy code layout
• Long annotations difficult to recognize
class Stack {void push(Object o
#ifdef SYNC, Transaction txn
#endif) {
if (o==null#ifdef SYNC
|| txn==null#endif
) return;#ifdef SYNC
Lock l=txn.lock(o);#endif
elementData[size++] = o;#ifdef SYNC
l.unlock();#endif
fireStackChanged();}
}
class Stack {void push(Object o
#ifdef SYNC, Transaction txn
#endif) {
if (o==null#ifdef SYNC
|| txn==null#endif
) return;#ifdef SYNC
Lock l=txn.lock(o);#endif
elementData[size++] = o;#ifdef SYNC
l.unlock();#endif
fireStackChanged();}
}
![Page 9: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/9.jpg)
Christian Kästner: Virtual Separation of Concerns 99
Preprocessor in Femto OS
![Page 10: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/10.jpg)
Christian Kästner: Virtual Separation of Concerns 1010
Problem 3: Error-Prone
• Syntax Errors • Type Errors
static int _rep_queue_filedone(...)DB_ENV *dbenv;REP *rep;__rep_fileinfo_args *rfp; {
#ifndef HAVE_QUEUECOMPQUIET(rep, NULL);COMPQUIET(rfp, NULL);return (__db_no_queue_am(dbenv));
#elsedb_pgno_t first, last;u_int32_t flags;int empty, ret, t_ret;
#ifdef DIAGNOSTICDB_MSGBUF mb;
#endif// over 100 lines of additional code
}#endif
#ifdef TABLESclass Table {void insert(Object data,
Txn txn) { storage.set(data,
txn.getLock()); }
}#endifclass Storage {#ifdef WRITE
boolean set(…) { ... }#endif}
![Page 11: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/11.jpg)
Christian Kästner: Virtual Separation of Concerns 1111
Problem 3: Error-Prone
• Syntax Errors • Type Errors
static int _rep_queue_filedone(...)DB_ENV *dbenv;REP *rep;__rep_fileinfo_args *rfp; {
#ifndef HAVE_QUEUECOMPQUIET(rep, NULL);COMPQUIET(rfp, NULL);return (__db_no_queue_am(dbenv));
#elsedb_pgno_t first, last;u_int32_t flags;int empty, ret, t_ret;
#ifdef DIAGNOSTICDB_MSGBUF mb;
#endif// over 100 lines of additional code
}#endif
#ifdef TABLESclass Table {void insert(Object data,
Txn txn) { storage.set(data,
txn.getLock()); }
}#endifclass Storage {#ifdef WRITE
boolean set(…) { ... }#endif}
![Page 12: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/12.jpg)
Christian Kästner: Virtual Separation of Concerns 1212
Problem 3: Error-Prone
• Syntax Errors • Type Errors
static int _rep_queue_filedone(...)DB_ENV *dbenv;REP *rep;__rep_fileinfo_args *rfp; {
#ifndef HAVE_QUEUECOMPQUIET(rep, NULL);COMPQUIET(rfp, NULL);return (__db_no_queue_am(dbenv));
#elsedb_pgno_t first, last;u_int32_t flags;int empty, ret, t_ret;
#ifdef DIAGNOSTICDB_MSGBUF mb;
#endif// over 100 lines of additional code
}#endif
#ifdef TABLESclass Table {void insert(Object data,
Txn txn) { storage.set(data,
txn.getLock()); }
}#endifclass Storage {#ifdef WRITE
boolean set(…) { ... }#endif}
![Page 13: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/13.jpg)
Christian Kästner: Virtual Separation of Concerns 1313
Advantages
• Easy to use• “annotate and remove”• Already part of many languages / simple tools• Most developers already familiar
• Flexible / Expressive• Language-independent / Uniform
• Low Adoption Barrier• esp. with legacy code
![Page 14: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/14.jpg)
Christian Kästner: Virtual Separation of Concerns 1414
Agenda
• Problems and Advantages of Preprocessors• 4 Improvements
• Views• Visual Representation• Disciplined Annotations• Product-Line-Aware Type System
• Summary and Perspective
![Page 15: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/15.jpg)
Christian Kästner: Virtual Separation of Concerns 1515
Views
• Emulate modularity• Hide irrelevant source code
• Hides entire files• Hides code inside files
Related work:• C-CLR• Version Editor • FeatureMapper • pure::variants• Effective Views• on-demand remodularization…
[ICSE’08, ViSPLE’08]
![Page 16: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/16.jpg)
Christian Kästner: Virtual Separation of Concerns 1616
Expression Problem
![Page 17: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/17.jpg)
Christian Kästner: Virtual Separation of Concerns 1717
View on Feature Eval
• Shows all files containing Eval code• Shows context information (kursiv) • Hides code without annotations
![Page 18: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/18.jpg)
Christian Kästner: Virtual Separation of Concerns 1818
View on the Variant “Add and Eval”
• Entirely hides file Power• Hides method print in Add ([…])• Code without annotation remains visible
![Page 19: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/19.jpg)
Christian Kästner: Virtual Separation of Concerns 1919
Agenda
• Problems and Advantages of Preprocessors• 4 Improvements
• Views• Visual Representation• Disciplined Annotations• Product-Line-Aware Type System
• Summary and Perspective
![Page 20: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/20.jpg)
Christian Kästner: Virtual Separation of Concerns 2020
Visual Representation: Background Colors
class Stack {void push(Object o
#ifdef SYNC, Transaction txn
#endif) {
if (o==null#ifdef SYNC
|| txn==null#endif
) return;#ifdef SYNC
Lock l=txn.lock(o);#endif
elementData[size++] = o;#ifdef SYNC
l.unlock();#endif
fireStackChanged();}
}
class Stack {void push(Object o
#ifdef SYNC, Transaction txn
#endif) {
if (o==null#ifdef SYNC
|| txn==null#endif
) return;#ifdef SYNC
Lock l=txn.lock(o);#endif
elementData[size++] = o;#ifdef SYNC
l.unlock();#endif
fireStackChanged();}
}
Related Work:• Spotlight• NetBeans• fmp2rsm• FeatureMapper• AspectBrowser…
[ICSE’08, ViSPLE’08]
![Page 21: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/21.jpg)
Christian Kästner: Virtual Separation of Concerns 2121
Alternative Visual Representations
● NetBeans ● Spotlight
![Page 22: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/22.jpg)
Christian Kästner: Virtual Separation of Concerns 2222
Scaling Visual Representations
• Focus on few features at a time• Repeating colors / manual assignment
sufficient• Analysis of 4 Java ME and 40 C programs:
• 96 % pages of source code with ≤ 3 colors• 99 % pages of source code with ≤ 7 colors
![Page 23: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/23.jpg)
Christian Kästner: Virtual Separation of Concerns 2323
Program Comprehension: An Experiment
• #ifdef vs. colors• 43 subjects in 2
groups• S1-2: search tasks
faster with colors(43% & 23%)
• M1-3: maintenancetasks same perform.
• M4: maintenancetask with red back-ground color -37%
• No influence on correctness• Subjects prefer colors
![Page 24: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/24.jpg)
Christian Kästner: Virtual Separation of Concerns 2424
Agenda
• Problems and Advantages of Preprocessors• 4 Improvements
• Views• Visual Representation• Disciplined Annotations• Product-Line-Aware Type System
• Summary and Perspective
![Page 25: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/25.jpg)
Christian Kästner: Virtual Separation of Concerns 2525
Disciplined Annotations
• Syntax errors caused by annotations on plain text
• Solution: Use underlying artifact structure• Enforce disciplined annotations on entire classes,
methods, or statements• Annotations of isolated brackets or
keywords regarded as undisciplined
class Foo { }
class Foo { }
class C {void m(int p) {
s1();s2(p, true);
}}
ClassDeclarationName=C
MethodDeclarationName=m
ReturnTypeType-void
ParameterName=p
Block
MethodInvkName=s1
MethodInvkName=s2
ParameterName=p
ParameterName=true
[TOOLS‘09]
![Page 26: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/26.jpg)
Christian Kästner: Virtual Separation of Concerns 2626
Variant Generation by AST TransformationClassDeclaration
Name=C
MethodDeclarationName=m
ReturnTypeType-void
ParameterName=p
Block
MethodInvkName=s1
class C {void m(int p) {
s1();s2(p, true);
}}
ClassDeclarationName=C
MethodDeclarationName=m
ReturnTypeType-void
ParameterName=p
Block
MethodInvkName=s1
MethodInvkName=s2
ParameterName=p
ParameterName=true
class C {void m(int p) {
s1();}
}
parsingprint /unparse
variantgeneration
Variant generation cannot cause syntax errors.
![Page 27: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/27.jpg)
Christian Kästner: Virtual Separation of Concerns 2727
Expressiveness of Disciplined Annotations
• Not all annotations are possible• Sometimes workarounds required• Experience: not a significant restriction in
practice• ~90% of all annotations in 40 C programs is
disciplined already
unrestrictedannotationson any character
disciplined annotations(underlying structure)
noannotations
high flexibility no flexibility
classesonly
![Page 28: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/28.jpg)
Christian Kästner: Virtual Separation of Concerns 2828
Agenda
• Problems and Advantages of Preprocessors• 4 Improvements
• Views• Visual Representation• Disciplined Annotations• Product-Line-Aware Type System
• Summary and Perspective
![Page 29: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/29.jpg)
Christian Kästner: Virtual Separation of Concerns 2929
Product-Line-Aware Type System
• Base: Type correct without annotations (for tool support)
• Detecting all pairs:• Method invocation and
method declaration• Reference and declaration
of class, variable, etc.• …
• Comparison of annotations foreach pair
class Database {Storage storage;void insert(Object data
Txn txn) { storage.set(data,
txn.getLock())}
}class Storage {#ifdef WRITE
boolean set(…) { ... }#endif}
Declaration annotated +Reference not annotated
Error in somevariants
Related Work:• SafeGen• fmp2rsm• Safe Composition• FFJPL
• Conditional Methods
[ASE’08]
![Page 30: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/30.jpg)
Christian Kästner: Virtual Separation of Concerns 3030
class Database {void insert(…) {
storage.set(…); }
}class Storage {
boolean set(…) { ... }}
• Handling dependenciesbetween features
• Can all variants with reference reach a corresp. declaration?
• Implementation: Propositionallogic and SAT solvers:
Type Checking in the Context of a Feature Model
Write
¬Read
))((writereadAPIAPIFM
∨⇒∧=
))()(( bATaATFM ⇒⇒
In all variants in which code fragment A is included also code fragment B is included:
![Page 31: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/31.jpg)
Christian Kästner: Virtual Separation of Concerns 3131
Formalization: CFJ
● On top of Featherweight Java● Theorem: Generation preserves typing
● Formal proof with Coq
[ASE’08]
![Page 32: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/32.jpg)
Christian Kästner: Virtual Separation of Concerns 3232
Implementation: CIDE
http://fosd.de/cide
![Page 33: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/33.jpg)
Christian Kästner: Virtual Separation of Concerns 3333
Automated Refactorings
• Feature Module ● Annotationenclass Stack {
int[] data;#ifdef UNDO
int backup;void undo() { /*...*/ }
#endif#ifdef TOP
int top() { /*...*/ }#endif
void push(int o) {#ifdef UNDO
backup = top();#endif
/*...*/}int pop() { /*...*/ }
}
class Stack {int[] data;
#ifdef UNDOint backup;void undo() { /*...*/ }
#endif#ifdef TOP
int top() { /*...*/ }#endif
void push(int o) {#ifdef UNDO
backup = top();#endif
/*...*/}int pop() { /*...*/ }
}
class Stack {int[] data;void push(int o) { … }int pop() { /*...*/ }
}
class Stack {int[] data;void push(int o) { … }int pop() { /*...*/ }
}
refines class Stack {int top() { /*...*/ }
}
refines class Stack {int top() { /*...*/ }
}
refines class Stack {int backup;void undo() { /*...*/ }void push(int o) {
backup = top();original(v);
}}
refines class Stack {int backup;void undo() { /*...*/ }void push(int o) {
backup = top();original(v);
}}
Base
Top
Und
o
Automated Transformation
[GPCE’09]
![Page 34: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/34.jpg)
Christian Kästner: Virtual Separation of Concerns 3434
Agenda
• Problems and Advantages of Preprocessors• 4 Improvements
• Views• Visual Representation• Disciplined Annotations• Product-Line-Aware Type System
• Summary and Perspective
![Page 35: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/35.jpg)
Christian Kästner: Virtual Separation of Concerns 3535
• Problems:• Separation of Concerns• Obfuscation• Error-proneness
• Retained advantages:• Easy to use• Flexible• Language-independent• Low adoption barrier
• Improvements:• Views• Visual representation• Disciplined annotations• Product-line-aware type
system
Summary
“Virtual Separation of Concerns”
![Page 36: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/36.jpg)
Christian Kästner: Virtual Separation of Concerns 3636
• Problems:• Separation of Concerns• Obfuscation• Error-proneness
• Retained advantages:• Easy to use• Flexible• Language-independent• Low adoption barrier
• Improvements:• Views• Visual representation• Disciplined annotations• Product-line-aware type
system
“Virtual Separation of Concerns”
Summary
![Page 37: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/37.jpg)
Christian Kästner: Virtual Separation of Concerns 3737
• Problems:• Separation of Concerns• Obfuscation• Error-proneness
• Retained advantages:• Easy to use• Flexible• Language-independent• Low adoption barrier
• Improvements:• Views• Visual representation• Disciplined annotations• Product-line-aware type
system
• Remaining Problems:• Separate Compilation• Black-box Reuse
“Virtual Separation of Concerns”
Summary
![Page 38: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/38.jpg)
Christian Kästner: Virtual Separation of Concerns 3838
Perspective
• Preprocessors common in practice, despite strong criticism
• Neglected by researchers• Tool support can address most problems
• Interim solution (with migration path) orserious alternative?
• Give preprocessors a second chance!
![Page 39: Virtual Separation of Concerns](https://reader034.vdocuments.site/reader034/viewer/2022042816/55905cce1a28ab4e2e8b46c7/html5/thumbnails/39.jpg)
Christian Kästner: Virtual Separation of Concerns 3939
Questions?
http://fosd.de/cide