the next generation mop, jochen theodorou, gr8conf 2013
DESCRIPTION
http://gr8conf.eu/Presentations/The-next-generation-MOPTRANSCRIPT
The next generation MOP
Jochen "blackdrag" Theodorou
About blackdrag● Working on Groovy Core since about 2005 ● Almost as long as that, Tech Lead of Groovy
● Currently Employed at Pivotal
● Responsible for most of the technical side of Groovy
Email: [email protected]
About a new MOP
● discussions since 2005
● good-for-all solution always missing
● some ideas open to discussion
● and some cleanup duties
● we insist on you helping us
Why change
Why change● has many inconsistencies
● makes many optimizations impossible
● in theory powerful
● have to know implementation details
● Impossible to extend and spec resistent
● the API just plain sucks
Some Basics and History
The Meta Class● Each class has a meta class
● Saves all the dynamic and static properties/methods for Groovy
● Internal control flow involves exceptions
● Designed for invoking methods through the meta class
The Meta Class● early Groovy meta programming:
● MOP methods invokeMethod and get/setProperty● A custom meta class (foo.metaClass = x)
● later class based automated meta class lookup
● transformed into the meta class creation handle (used by ExpandoMetaClass#enableGlobally())
The Meta Class
● Basic principle: Each instance has a meta class
● More specified: Only every GroovyObject instance(later we changed that with a global map)
● Global registry specifying initial meta class on first use
The Meta Class // myMetaClass some custom metaclass
// meta class in registry differentdef x1 = new X()assert x1.metaClass != myMetaClassx1.metaClass = myMetaClassassert x1.metaClass == myMetaClassdef x2 = new X()assert x2.metaClass != x1.metaClassX.metaClass = myMetaClassdef phantom = new X()def x3 = new X()assert x3.metaClass == x1.metaClassassert x3.metaClass != x2.metaClass
The Meta Class
X.metaClass = x2.metaClassassert phantom.metaClass == ???
Adding Methods/Properties● Standard meta class: MetaClassImpl
● does not support modifications
● New meta class for this: ExpandoMetaClass
● enabled with ExpandoMetaClass.enableGlobally()
● not always equally behaving to MetaClassImpl
More MetaClasses● ProxyMetaClass (intercepting, decorating)● MixinMetaClass (mixins)● ClosureMetaClass (GeneratedClosure)● DelegatingMetaClass (base class)● OwnedMetaClass (related to mixins)● HandleMetaClass (related to mixins)
Plus your own custom meta class
DSL not consistentFoo.metaClass.bar = 1 //defines propertyFoo.metaClass.bar = {1} //defines method
to use a closure as property:
foo.metaClass.bar = nullfoo.bar = {1}
● only for the instance● get metaproperty and set initial value creator
Overriding Super Methodsclass A { def doIt(){two() + ' done.'}def two(){'two'}
} class B extends A {}B.metaClass.two = {'my new two!'}def b = new B() assert b.two() == 'my new two!'assert b.doIt() == 'two done.'
Overriding Super MethodsTo make it work:
class A implements GroovyInterceptable { def doIt(){two() + ' done.'}def two(){'two'}
} class B extends A {}B.metaClass.two = {'my new two!'}def b = new B() assert b.two() == 'my new two!'assert b.doIt() == 'my new two! done.'
Adding Super Methodsclass A { def doIt(){two() + ' done.'}def methodMissing(String name, args){'two'}
} class B extends A {}def b = new B()assert b.two() == 'two'assert b.doIt() == 'two done.'A.metaClass.two = {'my new two!'}assert b.two() == 'my new two!'assert b.doIt() == 'my new two! done.'
Super Methods OverloadDoes not:
class A { def doIt(x){two(x) + ' done.'}def two(x) {'two'}
} class B extends A {}def b = new B()assert b.two('1') == 'two'assert b.doIt('1') == 'two done.'A.metaClass.two = {String s->'my new two!'}assert b.two('1') == 'my new two!'assert b.doIt('1') == 'my new two! done.'
Private Multi Methodsclass A { def doIt(){two() + ' done.'}
} class B extends A {private two(){1}
}def b = new B()assert b.two() == 1assert b.doIt() == '1 done.'
Speaking of privateclass A { private foo=1def b={foo}
} class B extends A {}def b = new B()assert b.b() == 1 //fails
● Information loss thorugh Closure#getProperty
get/setMetaClass
● persistency framework needs to be aware
● transient works for Serialization
● what about other frameworks?
● seamless integration anyone?
Propertiesclass X extends
org.xml.sax.helpers.XMLFilterImpl {def foo
}
● XMLFilterImpl has a get/setProperty
● cannot do new X().foo = bar
● cannot do println new X().foo
invokeMethod
No such conflict known.... but!
● dynamic entry point from Java● as methodMissing ● with GroovyInterceptable (EMC too) as upfront
method
conflicting concepts
What to make better?
… besides fixing those problems
Optimization efforts Lesson:
Java7 with invokedynamic is much better suited for Groovy's dynamic method calls
Reaction:
make Java7 the default (backport); rewrite DefaultGroovyMethods to use indy; throw out a lot of old code
Optimization efforts Lesson:
Hotspot is not happy about invoking target methods in the meta class (mega morphic call sites)
Reaction:
The meta class only gives back something you can call and does not do the call itself.
Optimization efforts Lesson:
Synchronization, Locks, volatiles usages on each method call destroy multithread performance as well as hotspot optimizations. Most applications set up mc changes on startup.
Reaction:
metaclass goes immutable; adding methods creates new meta class; lazy thread update (user level synchronization required)
Hot Swapping Lesson:
Keeping internal state in the class is bad (see timestamp_xxx, timestamp, $callSiteArray)
Reaction:
Removal. CallSiteArray not needed anymore, the timestamps are kept track off by the loader, not in the class
Optimization efforts Lesson:
Garbage collecting and recreating meta classes is very bad.
Reaction:
Keep the base meta class around and reuse everything possible to make new meta classes as lean as possible
API Design Lesson:
Conventions are good, forcing them is bad (GroovyObject)
Reaction:
Don't implement GroovyObject by default anymore.
General Design Lesson:
Too many ways of doing the same thing is no good
Reaction:
Most probably only methodMissing/propertyMissing anymore but easy way to „register“ a method to do invokeMethod.
API Design Lesson:
Having multiple, not equivalent entry points is bad. (MetaClassImpl has 3 for methods, multiusage of invokeMethod, information loss through get/setProperty)
Reaction:
Clean up the API to have only one entry point (removal of MetaClass#invokeMethod)
Possibilities
Internal vs. ExternalInternal usage:class X { def methodMissing(String name, args) {1}}
External usage:class X {}X.metaClass.methodMissing = {1}
Combined:class X { static {this.metaClass.methodMissing = {1}}}
Dynamic Invoke from JavaBefore:
GroovyObject foo = ...;String s = (String)foo.invokeMethod(“bar“, new Object[]{});
After:
Object foo = ...;String s = MopInterface.invoke(foo, “bar“);
● helper class for dynamic interactions with Groovy● similiar for properties
Adding a method from JavaBefore:GroovyObject foo = ...;foo.getMetaClass().registerInstanceMethod(“foo“,
new Closure(null){public Object doCall(){1}};
After:Object foo = ...;MopInterface.addInstanceMethod(foo, “foo“,
new T() {public Object myFancyMethod(){1}});
● Doesn't have to be a Closure or MetaMethod● All declared methods used (lambdas too)
Limited Meta Class ChangesUse Case:
unrolling all changes a GroovyShell or Eval.me did
Before:
● tracking meta class mutations impossible● „unmutate“ the metaclass certainly is● can only to track newly set meta classes
Limited Meta Class ChangesUse Case:
I am a library developer and I don't want intrusive changes from user code to my library classes, changing the way my library is calling methods.
Before:
If the change is not effecting the library, then most probably because of a bug.
Realms
RealmsA realm here defines how the class in the realm
„sees“ the meta classes. Different classes can have different realms.
MyClass(y)
MyOtherClass(x)
x.foo()
using metaClass(x)
Realms
MyClass(y)
MyOtherClass(x)
x.foo()
using metaClass(x) from realm(MyClass)
<Realm defined by MyClass>
Realms
MyClass(y)
MyOtherClass(x)
y.bar()
using metaClass(y) from realm(MyOtherClass)
<Realm defined by MyOtherClass>
Realms
INIT
DEFAULT
realm(A_n)
realm(C_n)realm(A_1)
........
realm(B_1)
realm(C_1)........
realm(D_1)
RealmsINIT does not allow changing the meta class
INIT
DEFAULT
Non isolated
isolated
RealmsDefault gets all the unspecified changes
INIT
DEFAULT
Non isolated
isolated
RealmsIsolated realms don't get unspecific changes
INIT
DEFAULT
Non isolated
isolated
RealmsNon isolated realms can override meta classes in
default without impossing themINIT
DEFAULT
Non isolated
isolated
Isolated Realm@Realm(
marker=SomeMarkerClass, parentRealm=Realm.INIT)
class MyLibraryClass {def foo(){bar()}def bar(){1}
}MyLibraryClass.metaClass.bar = {2}assert new MyLibraryClass().foo() == 1
Non-Isolated Realm@Realm(marker=SomeMarkerClass)class MyLibraryClass {def foo(){bar()}def bar(){1}
}MyLibraryClass.metaClass.bar = {2}assert new MyLibraryClass().foo() == 2
Testing Realm
class X {private foo(){1}}class MyTest extends GroovyTestCase {
@Realm(marker=SomeMarkerClass, allowPrivate=true)
void testPrivateWorking() {def x = new X()assert x.foo() == 1
}void testPrivateNotWorking() {
def x = new X()shouldFail(MissingMethodException) {
x.foo()}
}}
Limited Meta Class ChangesBefore:
Change intrusive, visible to everyone
After:
def foo(){1}assert foo() == 1realm.withSubRealm {
this.metaClass.foo = {2}assert foo() == 2
}assert foo() == 1
Q/A?