p2, modular provisioning for osgi
DESCRIPTION
In its 3.4 release, Eclipse introduced a new installation / update mechanism called p2. Beyond its Eclipse specific appearance, p2 is a modular provisioning platform for OSGi based systems (currently focused on Equinox) addressing the wide spectrum of provisioning needs from small devices to servers through desktops. In this talk, p2 key concepts and architecture will be introduced, and p2's flexibility will also be demonstrated. As importantly as the actual technology, this talk will also discuss how p2 and the various tools help throughout the software lifecycle from development to serviceability.TRANSCRIPT
1
p2, modular provisioning for OSGi
Pascal Rapicault
IBM RationalTM, p2 lead
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 2
• OSGi proves the power of componentization
• Componentization naturally spreads
• More components → more management
• Management is hard
• It’s all about the Contract– Defining– Instantiating– Executing– Maintaining
OSGi is good for you!
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 3
How does p2 help?
• Manages the contract– Dependencies– Code– Settings (VM args, start level, etc)– Integrations– Non-OSGi parts (e.g. native launcher)
• Extensible
• GUI and Headless
One consistent model from installation to servicing
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 4
• A provisioning solution for OSGi™ systems– Managing non-running instance (only on Equinox for now)– Start level, framework extension– Fine-grained dependency management
• An extensible provisioning platform
p2 is about installing!
© 2009 IBM Corp. and EclipseSource; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 5
Concepts
Architecture
Tooling
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 6
One construct to rule them all
Decouple decision making from the actual content
Everything is an IUEverything is installable
It is also referred to as “metadata”
IU(id, ver)
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 7
Anatomy of an IU
Provided
CapabilitiesRequired
Capabilities
Properties
Artifact reference
Actions
IU(id, ver)
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 8
Anatomy of an IU, requirements / capabilities
• Capabilities and requirements are the mechanism by which an IU express what it provides and what needs.
• A capability is composed of a:– Namespace (string), name (string) and version
• A requirement is composed of a:– Namespace, name and version range
• Namespace, name and version are open ended.
• The requirements and capabilities expressed by IUs can be arbitrary and don’t have to all be in the same namespace.– Resources – RPMs, .exes, docs, …– Virtual – if you can define a capability, I can depend on it
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 9
Separation of concerns
• Installable Unit– Exists independently of the repository
• Metadata Repository– Store only Installable Units. – API, no specified serialization format
• Artifact Repository– Store only Artifacts– API, no specified serialization or layout format
Metadata Artifacts
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 10
Artifacts
• Bytes/content to be installed
• Any form– JARs (e.g., bundles, features, …)– Binary executables– RPM, MSI, …
• Defined, maintained, loaded and used separately from the metadata
© 2009 IBM Corp. and EclipseSource; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 11
Concepts
Architecture
Tooling
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 12
p2 Architecture
Metadata Artifacts
Runtimes
Profiles
OS
Eclipse
Engine
Eclipse
Classic
Eclipse
for C++
Other
Director Repositories
Touchpoints
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 13
Terminology
• p2 / Agent – The provisioning infrastructure on client machines
• Installable Unit (IU) – Metadata that describes things that can be installed/configured
• Artifact – The actual content being installed/configured(e.g., bundle JARs)
• Repository– A store of metadata or artifacts
• Profile – The target of install/management operations
• Planner– The decision-making entity in the provisioning system
• Engine– The mechanism for executing provisioning requests
• Touchpoint – The part of the engine responsible for integrating the provisioning
system to a particular runtime or management system
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 14
p2 in Action – internal management - milli
TransportsHttp/Https
File systemVolume
Repositoriesp2
Update Site
Engine
OSGiNative/OS
Director
ProfilesRuntimes
Provisioning operation requested
Metadata fetched and constraints
analyzed
IU install, uninstall, update
operations
Artifact availability and mirroring
Mirroring
Data transfer
IUs configured into runtimes
Profile updated
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 15
External management - nano
Managed applicationAgent
Engine
OSGiNative/OS
Director
Repositoriesp2
Update Site
TransportsHttp/Https
File systemVolume
OSGi fwk
The agent writes out file to control the application.The “provisioning” presence in the managed application is small.
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 16
External management - micro
Director
OSGi fwk
Engine
OSGiNative/OS
Repositoriesp2
Update Site
TransportsHttp/Https
File systemVolume
Repositoriesp2
Update Site
The decision making on what to install is done on the agent andcommunicated to the managed application (no remoting provided in the open source). The agent could also carry a copy of the profile registry.Demo - http://eclipsesource.com/blogs/2009/05/05/remote-provisioning-with-p2/
Managed applicationAgent
© 2009 IBM Corp. and EclipseSource; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 17
Concepts
Architecture
Tooling
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 18
Build, when the metadata comes to be
Metadata matters
• The p2.publisher is responsible for metadata / artifact generation– Produces p2 metadata from bundles, Eclipse features and products– Can be used in any build system
• PDE Build is the richest (from source to repo)• Maven / tycho integration
• Not all metadata can be inferred– Actions to be executed on a given phase– Some tweaking is necessary– Author metadata advice
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 19
Repository management
Everybody creates repos, they need to be managed
• Problems– You want to promote one build over another one
• Composite repositories– You build more than you want to make available
• Slicing– You want to replicate builds from one repository to another
• Mirroring applications
• Repository validation– Repository validation that everything is installable– For every IU, each artifact is available– IU comparison tools, to ensure that the metadata is not changed– Artifact comparison tool to ensure that one artifact has not changed [1]
– Repository diff’ing tool
[1] - Talk on versioning and provisioning - http://www.eclipsecon.org/2009/sessions?id=585
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 20
What is p2?
• A provisioning solution for OSGi™ systems
• An extensible provisioning platform
• A complete offering from Build time to Runtime
• A community and an ecosystem
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 21
References
• p2 landing page http://wiki.eclipse.org/Equinox_p2
• All p2 articles http://wiki.eclipse.org/Category:Equinox_p2
• Getting the code http://wiki.eclipse.org/Equinox_p2_Getting_Started_for_Developers
• Contacting us– Equinox newsgroup– Mailing list: [email protected]
© 2009 IBM Corp. and EclipseSource; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 22
Appendix
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 23
Profile
A profile is the complete description in terms of IU of what is installed
• A profile contains– Properties defining the “environment” such as os, ws, arch,
install location, bundle pool location– The list of IUs– Properties associated with IUs
Class: Org.eclipse.equinox.internal.provisional.p2.engine.IProfile
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 24
Planning a profile change
Because of inter IU dependencies, modification against a profile should be planned
• ProfileChangeRequest– Capture the changes you want to make to the profile (e.g. Install, Uninstall).– The request is processed by the Planner– An update is a removal and an addition
• Planner– The entity responsible for the evaluation of the change request, computing the
transitive closure and checking the dependencies– If a solution exists it will find it
• ProvisioningPlan– The planner returns a provisioning plan. The planning succeed pass or fail. Upon
failure explanations are provided. Upon success the plan returns a set of operands to go from the initial state of the profile to the desired state.
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 25
Engine
The mechanism by which the profile is actually changed
• The engine ensures consistency of the modification by performing a “transaction”.
• Runs over a set of operands (usually resulting from a planning operation) and executes a given set of phases on them. No profile consistency validation is done
Engine.perform(Operand[] ops, PhaseSet phases, IProgressMonitor pm)
• For each phase, the engine interprets from the touchpoint data of the IU, the action specific to the phase. The engine looks up the action to execute.
• Currently 8 phases are defined, and the typical set of phases (and their order) is defined in DefaultPhaseSet. The design allows for new phases could be added.
• The engine emit events on the IProvisoningEventBus to describe what is happening. Events on the bus are post events and can not be vetoed.
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 26
Installable Unit Fragments
• An installable unit fragment is an entity that attaches to an installable unit.
• Much like OSGi fragments, IU fragments are used to complement an existing installable unit and appear as one with the IU they attached to.
• They are typically used to deliver action to an installable unit (e.g. start level) because IUs should stay as context agnostic as possible
• IU fragments can be attached to several IU at the same time. See for example the tooling.osgi.bundle.default IU that applies to all bundles.
• Note that IU fragments are not how OSGi fragments are delivered. OSGi fragments are delivered as regular IU with requirements on their host.
© 2009 IBM Corp; All right reserved. Made available under Creative Commons Att. Nc Nd 2.5 license 27
Installable Unit Patches
• An installable unit patch has the ability to “modify” the requirements of any other installable unit.
• Deal with three concerns:– The IUs to which the patch apply
– The lifecycle of patch
– The changes applied
Note: In 3.5 the feature patch editor only exposes some of those capabilities.