what java developers (don’t) know about api compatibility

Download What java developers (don’t) know about api compatibility

Post on 06-Aug-2015

174 views

Category:

Technology

2 download

Embed Size (px)

TRANSCRIPT

  1. 1. What Java Developers (dont) know about API compatibility, and why this matters Jens Dietrich, May 2014 http://tinyurl.com/brokenapi
  2. 2. About the Author background (PhD): non-classical logic, very formal 1996-2003: no research, work in industry and development aid 2003-2012: models, algorithms and tools to understand and improve the design and architecture of systems xplrarc.massey.ac.nz exploring better ways to safely assemble systems from modules
  3. 3. Current Research Topics fast algorithms and effective data structures for the computation of transitive closure and CFL-reachability in sparse graphs (contract research for Oracle Labs) understanding circular dependencies between modules statistical oracles for performance regression testing API / component compatibility and evolution
  4. 4. Motivation relevant to industry under-researched topic trend towards more empirical studies in SE combination of software evolution and composition
  5. 5. Overview background puzzlers survey impact analysis and repository studies roadmap
  6. 6. Background applications are neither monolithic nor static they are composed using APIs provided by libraries, both libraries and the applications themselves change when is the evolution of libraries safe? this depends on: the compatibility of changes made how applications are built and deployed consider the Java platform (language and JVM), but findings can be generalised
  7. 7. Building and Deploying Java Applications 1. build (compile and test) programs with all libraries, deploy together mainstream, good tools support (Ant, Maven, Jenkins, ..) 2. build and upgrade libs only facilitates 24/7 applications, approach used in OSGi, app servers, JNLP
  8. 8. Types of Compatibility Program P uses API in library L L1 L2 P provider API contract consumer horizontal compatibility (h-comp(L, P)): L is compatible with P vertical compatibility (v-comp(L1,L2)): is L1 is h-compatible with P and L1 is v-compatible with L2, then L2 is h- compatible with P for all P
  9. 9. Compatibility in Java P is source compatible with L if P can be compiled with L P is binary compatible with L if P can be linked with L vertical only: L1 is behavioural compatible with L2 if the observable behaviour of P linked with either L1 or L2 is the same (contextual w.r.t. all or some P)
  10. 10. Binary Compatibility (v-Version) A change to a type is binary compatible with pre-existing binaries if pre-existing binaries that previously linked without error will continue to link without error. [JLS 7, sect 13.1]
  11. 11. Evolution Problem Given a program P and two versions of a library L: 1. if P is source/binary compatible with L-v1, is it still source/binary compatible with L-v2 ? 2. Is L-v1 behavioural compatible with L-v2?
  12. 12. Evolution Problems as Deployment Puzzlers inspired by Bloch/Gafters famous book 18 problems (May 2014) slides: http://www.slideshare.net/JensDietrich/presentation-30367644 code: https://bitbucket.org/jensdietrich/java-library-evolution-puzzlers discovery of bug in JLS-7
  13. 13. Structure of a Puzzler provide a program P consisting of an executable class .Main Main uses a class lib..Foo defined in a library lib-1.0.jar there is a modified class lib..Foo defined in a library lib-2.0.jar
  14. 14. Structure of a Puzzler (ctd) experiments: compile P with lib-1.0.jar and then run it with lib-1.0.jar compile P with lib-1.0.jar and then run it with lib-2.0.jar compile P with lib-2.0.jar and then run it with lib-2.0.jar automated with build script
  15. 15. Source Compatibility !=> Binary Compatibility public class Foo { public static java.util.Collection getColl() { return new java.util.ArrayList(); } } public class Foo { public static java.util.List getColl() { return new java.util.ArrayList(); } } public class Main { public static void main(String[] args) { java.util.Collection coll = Foo.getColl(); System.out.println(coll); } } libversion1.0libversion2.0 client program using lib specialising return type (strengthen postcondition!)
  16. 16. Binary Compatibility !=> Source Compatibility public class Foo { public static List getList() { List list = new ArrayList(); list.add(42); return list; }} public class Foo { public static List getList() { List list = new ArrayList(); list.add(42); return list; }} public class Main { public static void main(String[] args) { List list = Foo.getList(); System.out.println(list.size()); } } libversion1.0libversion2.0 client program using lib
  17. 17. Binary Compatibility !=> Behavioural Comp. public class Foo { public static List getList() { List list = new ArrayList(); list.add(42); return list; }} public class Foo { public static List getList() { List list = new ArrayList(); list.add(42); return list; }} public class Main { public static void main(String[] args) { List list = Foo.getList(); for (String s:list) { System.out.println(s); } } } libversion1.0libversion2.0 client program using lib
  18. 18. How Bizarre ... import java.io.Serializable; class Foo { public void foo(T t) { t.compareTo(); System.out.println(t); } } import java.io.Serializable; class Foo { public void foo(T t) { t.compareTo(); System.out.println(t); } } public class Main implements java.io.Serializable { public static void main(String[] args) { Main m = new Main(); new Foo().foo(m); } } libversion1.0libversion2.0 client program using lib
  19. 19. Constant Inlining public class Foo { public static final int MAGIC = 42; } public class Foo { public static final int MAGIC = 43; } public class Main { public static void main(String[] args) { System.out.println(Foo.MAGIC); } } libversion1.0libversion2.0 client program using lib
  20. 20. How Linking Works (for Methods) the linker uses the method descriptor method descriptors do not contain type parameters method descriptors do not contain throw clauses descriptors have to match exactly (no reasoning performed by the linker) types that are handled as compatible by the compiler (boxing/unboxing) are strictly different in byte code
  21. 21. Why ? ensuring binary backwards compatibility is a major objective for JDK evolution bytecode kept stable, JVM innovation focuses on maintainability, stability and scalability language evolution focuses on programmer productivity
  22. 22. Why ? cew118
  23. 23. How Java Evolves language innovation by adding syntactic sugar ensure byte code compatibility, perhaps add features (invokedynamic), but dont modify existing ones compiler magic: generic types => erasure inner classes => synthetic methods to bypass encapsulation covariant return types => return type overloading reference vs value types => auto-(un)-boxing
  24. 24. How do Developers Cope ? from https://www.youtube.com/watch?v=M7FIvfx5J10 , Standard YouTube Licence
  25. 25. Study Repositories 1. Study program evolution in the qualitas corpus data set, find incompatible upgrades, and the impact this has on other programs. 2. Study Maven dependencies whether incompatibilities are introduced by recursive dependency resolution. Dietrich J., Jezek K., Brada P.: Broken Promises - An Empirical Study into Evolution Problems in Java Programs Caused by Library Upgrades. CSMR-WCRE'14. Jezek K., Dietrich J.: On the Use of Static Analysis to Safeguard Recursive Dependency Resolution. Accepted for SEAA 2014.
  26. 26. CSMR14 Study studied 111 programs, 661 versions (Qualitas Corpus) incompatible API upgrades are common in real-world libraries: 344/455 (75%) of upgrades in data set have incompatibilities commodity libraries (antlr, ant, hibernate, weka, colt, junit, jung) are affected many constants that are inlined are then changed a lot of potential, but only a few actual problems
  27. 27. SEEA14 Study study on transitive dependencies of 1902 Maven modules in Qualitas Corpus 367 modules depend on multiple versions of other modules common issues include: incompatibilities between multiple versions redundant (unused) dependencies missing dependencies
  28. 28. What do Developers Know ? developers we asked were puzzled about our puzzlers idea: turn this into a survey ! problem: how to convince people to participate?
  29. 29. How not to do it Tan Yuen http://www.gaolat.com/2013/10/vivo-pizza-free-pizzas-for-students.html
  30. 30. Challenges we are asking developers for a lot of time (>1H) i.e., we are asking them for a lot of money but we can offer them some value they will learn something !
  31. 31. Recruiting Participants JavaWorld Melbourne JUH several Czech JUGs NZ Industry Partners (Orion, SolNet, Kiwiplan) ex-students old boys networks: Stuttgart Java User Group, German Telecom
  32. 32. Acknowledgements Kamil Jezek (Uni Western Bohemia), Jeff Friesen, Athen O Shea (Java World), Kon Soulianidis (Melbourne JUG), Gareth Cronin (Orion), Manfred Duchrow (MD Consulting), Jochen Hiller (German Telecom), ...
  33. 33. Survey Design hosted on SurveyMonkey 7 background questions (experience) full survey: 21 standard puzzlers with 2 questions each, and 4 constant inlining puzzlers with 1 question per puzzler - a total of 46 technical questions short survey: 9 standard and 4 constant inlining puzzlers, a total of 22 technical questions
  34. 34. Standard Puzzler Q1 Q1 Can the version of the client program compiled with lib- 1.0.jar be executed with lib-2.0.jar ? a) no, an error occurs b) yes, but the behaviour of the program is different from the program version compiled and executed with lib-1.0.jar c) yes, and the behaviour of the program is the same as the program version compiled and executed with lib-1.0.jar
  35. 35. Standard Puzzler Q2 Q2 Can the client program be compiled and then executed w

Recommended

View more >