implementing higher-kinded types in dotty
TRANSCRIPT
![Page 1: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/1.jpg)
Implementing Higher-Kinded Types in Dotty
Martin Odersky
Scala Symposium, October 2016, Amsterdam
joint work with
Guillaume Martres, Dmitry Petrashko
![Page 2: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/2.jpg)
Dotty• The new Scala compiler developed at LAMP
• Based on DOT as a theoretical foundation
• Long-standing question: How to implement higher-kinded types on these foundations?
• The paper and talk investigate 4 (!) different solutions to do so.
![Page 3: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/3.jpg)
Timeline
Simple Encoding
ProjectionEncoding
RefinementEncoding
DirectRepr.
![Page 4: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/4.jpg)
Foundations: DOT
The DOT calculus is intended to be a minimal foundation of Scala.
Its type structure is a blueprint for the types used internally in the compiler.
![Page 5: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/5.jpg)
DOT Terms• Translated to Scala notation, the language
covered by DOT is:Value v = (x: T) => t Function
new { x: T => d } Object
Definition d = def a = t Method definitiontype A = T Type
Term t = v Valuex Variablet1(t2) Applicationt.a Selection{ val x = t1; t2 } Local definition.
![Page 6: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/6.jpg)
DOT TypesThe Types covered by DOT are:
Type T = Any Top typeNothing Bottom typex.A Selection(x: T1) => T2 Function{ def a: T } Method declaration{ type T >: T1 <: T2 } Type declarationT1 & T2 Intersection{ x => T } Recursion
![Page 7: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/7.jpg)
DOT TypesThe Types covered by DOT are:
Type T = Any Top typeNothing Bottom typex.A Selection(x: T1) => T2 Function{ def a: T } Method declaration{ type T >: T1 <: T2 } Type declarationT1 & T2 Intersection{ x => T } Recursion
Should Scala have these?
![Page 8: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/8.jpg)
DOT TypesThe Types covered by DOT are:
Type T = Any Top typeNothing Bottom typex.A Selection(x: T1) => T2 Function{ def a: T } Method declaration{ type T >: T1 <: T2 } Type declarationT1 & T2 Intersection{ x => T } RecursionScala has only refinements
T { d1 … dn}
with this as self reference.
![Page 9: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/9.jpg)
Encoding Type Parameters
![Page 10: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/10.jpg)
Lists, Encoded
![Page 11: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/11.jpg)
A Variant of Variance
![Page 12: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/12.jpg)
Variance, Encoded
=
![Page 13: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/13.jpg)
Why Encode Parameterization?
• One fundamental concept (type member selection) instead of two.
• Clarifies interaction between parameterization and membership.
E.g.List[_ <: Number] = List { type T <: Number }
![Page 14: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/14.jpg)
The Simple Encoding
Simple Encoding
![Page 15: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/15.jpg)
Higher-Kinded TypesProblem: How to encode
type C[X] = List[X]
type C[+X <: Number] <: List[X]
?
Idea: Generalize definition of type parameters:• of a class: designated member types
• of an alias type: type params of its alias
• of an abstract type: type params of upper bound
![Page 16: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/16.jpg)
The Simple EncodingProblem: How to encode
type C[X] <: List[X]
type D[X <: Number] = Cons[X]
?
This leads to:type C <: List
type D = List { type T <: Number }
![Page 17: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/17.jpg)
The Simple EncodingProblem: How to encode
type C[X] <: List[X]
type D[X <: Number] = Cons[X]
?
This leads to:type C <: List List[_]
type D = List { type T <: Number } List[_ <: Number]
Note that these encodings can also be seen as wildcard (existential) types!
![Page 18: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/18.jpg)
Limitations of the Simple EncodingUnfortunately, there are things that can’t be encoded that way:
type Rep[T] = T
type LL[T] = List[List[T]]
type RMap[V, K] = Map[K, V]
What To Do?
![Page 19: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/19.jpg)
The Projection Encoding
ProjectionEncoding
![Page 20: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/20.jpg)
The Projection EncodingIdea (originally due to Adriaan Moors):
Simulate type application using type selection.
E.g.type C[X] <: List[X]
becomestype C <: Lambda$P {
type $hk0type $Apply <: List { type T = $hk0 }
}
![Page 21: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/21.jpg)
The Projection EncodingIdea (originally due to Adriaan Moors):
Simulate type application using type selection.
ThenC[String]
becomesC { type $hk0 = String } # $Apply
![Page 22: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/22.jpg)
The Projection EncodingIdea (originally due to Adriaan Moors):
Simulate type application using type selection.
If we instantiate C to List, say, we get:type C = Lambda$P {
type $hk0type $Apply = List { type T = $hk0 }
}
![Page 23: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/23.jpg)
The Projection EncodingHence,
C[String]
= C { type $hk0 = String } # $Apply
= Lambda$P { type $hk0; type $Apply = List { type T = $hk0 }
} { type $hk0 = String } # $Apply
=:=
List { type $hk0 = String }
= List[String]
Where “=:=” is interpreted as “being mutual subtype of”.
![Page 24: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/24.jpg)
Problems with the Projection Enc.• There were several, but the killer was:
It relies on type projection T#U, which is unsound in general and will therefore be phased out.
• See the paper for a discussion in detail.
![Page 25: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/25.jpg)
The Refinement Encoding
RefinementEncoding
![Page 26: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/26.jpg)
The Refinement EncodingIdea:
Encodetype C[X] <: List[X] C[String]
astype C <: { z => List { type T = $hk0 }
type $hk0 }
C { type $hk0 = String }
This also had several problems which are described in the paper.
![Page 27: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/27.jpg)
In the end…… we represented hk types directly, using special type forms for
• Type lambda: [X] -> T TypeLambda
• Higher-kindedtype application: C[T] HKApply
• A beta reduction rule: ([X] -> T)[U] à [X := U]T
• Rules for subtyping, type inference …
![Page 28: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/28.jpg)
Implementation Data
Simple Encoding
ProjectionEncoding
RefinementEncoding
DirectRepr.
Total type-checker + core: 28’000 cloc
![Page 29: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/29.jpg)
Evaluation• The simple encoding is a modest extension for a
language with wildcards or variance that wants to support basic concepts associated with hk types.• Cheap way to get a lot of the advantages of hk.
• Difficulty: Explain what’s legal and what is not.
• Other encodings were less successful than they looked at first.
• A direct representation is needed for robust support of full higher kinded types.
![Page 30: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/30.jpg)
Still Missing• A theory that explains what the compiler did
in terms of DOT.
• Hopefully, we can find something that’s small and fits well.
![Page 31: Implementing Higher-Kinded Types in Dotty](https://reader033.vdocuments.site/reader033/viewer/2022052915/58b8a1e31a28abc06d8b5549/html5/thumbnails/31.jpg)
A Take Away• Compiler hacking is a possible way to validate
ideas
• But it may not be the most efficient one.
• If we would have a better theory how higher-kinded types fit into DOT, we could have avoided some costly experiments.