the scala refactoring library: problems and perspectives

IntroductionArchitecture and Problems

Perspectives

The Scala Refactoring Library:Problems and Perspectives

Matthias [email protected]

10. Februar 2016

Matthias Langer [email protected] The Scala Refactoring Library


Perspectives

Thanks



Perspectives

Contents

1 Introduction

2 Architecture and Problems

3 Perspectives



Perspectives

Section 1

Introduction



Perspectives

Overview

Scala Refactoring LibraryIDE independent refactoring libraryCreated by Mirco Stocker around 2009

See http://scala-refactoring.org/

Used by ENSIME and ScalaIDESupports Scala-2.10 and Scala-2.11Lots of features in theoryQuite buggy in practice



Perspectives

My Role

Started contributing to Scala-IDE in 2014Use Scala-IDE at workGot annoyed by refactoring bugsFixed lot’s of bugs in the last yearMany bugs leftWorking in free time



Perspectives

Disclaimer

Discussion ContributionThis is my personal opinion.

Bad NewsI’m not going to tell you how great things areHad hard time preparing this presentation

Good NewsLearned a lotForced me to see things in perspectiveShare conclusions with you



Perspectives

Section 2

Architecture and Problems



Perspectives

ASTs

Original Source

def f(x: Int) = x

Related AST



Perspectives

Tip

Use bracket-expression-beautifier

DefDef(Modifiers(), TermName("f"), List(), List(List(ValDef(Modifiers(PARAM), TermName("x"),

TypeTree().setOriginal(Select(Ident(scala),scala.Int)), EmptyTree

))

), TypeTree(), Ident(TermName("x")))



Perspectives

Overview

Library uses ASTs from the presentation compilerRefactorings are performed as transformations on ASTsCode is generated from transformed ASTsSee Mirco Stockers Master Theses for detailsOverall design looks good



Perspectives

Analysis Phase

Symbol lookup

trait IndexLookup {def declaration(s: Symbol): Option[DefTree]def references(s: Symbol): List[Tree]

}

Name validation

trait NameValidation {def isValidIdentifier(name: String): Booleandef doesNameCollide(name: String, s: Symbol):

List[Symbol]}



Perspectives

Transformations

Think of Tree => Option[Tree]

Combinable via andThen(t) and orElse(t)

Can be created from partial functions and predicatesSupports different traversal strategies

val reverseTemplateMembers = transformation {case t: Template => t.copy(body = t.body.reverse)

}val rename = topdown((tiedToRenamedSymbol.andThen(renameSingleTree)).orElse(identity))



Perspectives

Changeset Generation

Changed Tree Detection

def print(tree: Tree): Fragment = {if (hasBeenModifed(tree)) regenerateSourceCode(tree)else keepSourceCode(tree)

}

End Product

case class TextChange(sourceFile: SourceFile, from: Int,to: Int, text: String)



Perspectives

No Layout Information

Same AST

def f1(xs: Seq[Int]) =xs.map(i => i * 2)

def f2(xs: Seq[Int]) =xs map (i => i * 2)

def f3(xs: Seq[Int]) = {xs.map { i =>i * 2 //...

}}

OK for compilerProblematic for refactoring



Perspectives

ASTs are desugared

Same AST

def f1(xs: Seq[Int]) =for(x <- xs) yield x*x

def f2(xs: Seq[Int]) =xs.map(x => x*x)

def f1(xs: Seq[Int]) =xs.map(_ * 2)

def f2(xs: Seq[Int]) =xs.map(x$1 => x$1 * 2)

def g(a: Int = 0, b: Int = 0) = a + bdef f1 = g(b = 2)def f2 = {val x$1 = 2; val x$2 = 0; g(x$2, x$1)

}



Perspectives

Tree Printing Very Hard

How Trees Are PrintedUses positions to look into original sourceFalls back to regular expressions and/or handwritten parsersRelies on implementation details of the compilerMost complex and buggy part of the librarySee ReusingPrinter, LayoutHelper, PimpedTrees

Why doesn’t the library use it’s own ASTs?This was attempted in the beginningWriting a Scala Parser and Typer is a major effort



Perspectives

Section 3

Perspectives



Perspectives

Conclusions

Main ConclusionRefactorings based on ASTs from the compiler very hard toget right

ImplicationsGetting rid of compiler ASTs means rewriteTeam effort neededscalameta looks like very much what we needA clean implementation will attract more contributors



Perspectives

Short Term Improvements

Only keepRenameMove ClassOrganize Imports

in the IDEOverwork README.md to attract more contributors



Perspectives

Questions and Discussion


the scala refactoring library: problems and perspectives

Technology