third-party software management under bsd

Third-party softwaremanagement under BSD

Andrew Pantyukhin

<infofarmer@FreeBSD.org>

EuroBSDCon 2006

powered by the FreeBSD Foundation

First things first

●The need for 3d-party software

●Why do we require packages

●The need for package management

●What do we expect to manage

History -Packages in UNIX

●OS is an all-included bundle

●"Users" must figure out how to "use" software

themselves

●SysV/SunOS PKG format

History -Packages in BSD

●JKH committed pkg_install in 1993

●JKH committed ports in 1994

●{Net,Open}BSD snatched both of them

History -Packages in Linux

●Many PM systems appeared in 1993-95, including

RPM and dpkg

●Portage came out in 2002

●A multitude of PM systems were started over since

90s, some of them still maintained, some actively

developed

Current Status

●Dozens of distinctive PM systems

●Focus on binary and/or source packages

●Automation of compilation-related routines

●Automation of installation tasks

FreeBSD ports

●Make + shell

●Multiple scripts in multilevel directories

●Effective automation of common tasks and hacks

NetBSD ports

●Good documentation

●pkginstall framework

●pkg options framework

OpenBSD ports

●Fake build environment

●Immaculate documentation

●Options, flavors, multi-packages

●New pkg_install + a hard-headed maintainer

Looking out of BSD

●RPM

●Dpkg

●Portage

●Other

●Linux Core Consortium - Standard Base

Why bother?

●Isolation has its benefits, but

●No system alone proved to be able to reach even

modest qualitative/quantitative targets

●We have too much in common not to share ideas and

code

What's up?

●Scalability

●Resource management/customization

●Dependencies, capabilities, functionality

●User-side management, interface, Clicks!

●Versioning everywhere

●Saving porter a lifetime (as in more automation)

●Much more...

What packages needand what they provide

●Other packages

●Resources

●Functionality

OS Resourcesfor packages

●Disk space

●TCP/UDP ports at different addresses

●File name spaces

●User/group names/ids

●CPU, RAM, disk throughput, etc.

●Bandwidth

●You name it...

Appliances

●Package + dependencies bundled together

●Jail appliances

●OS appliances

●BSD-based hardware appliances

Package-Provided Resources

●(SQL) Databases

●Virtual Hosts

●Client connections to an (antivirus) engine

●On-screen pixel real-estate

●Technically specific items, like lines in configuration

files

Package-Provided Functionality

●Needed by users directly and/or required by other

packages

●API-oriented, e.g. language modules and extensions,

binary libraries

●Task-oriented, e.g. web servers, mail clients, DBMS

Install-/run-time customization

●Automation of installation tasks based on simple

user-made choices

●Single interface to configure the usage of some

resources, like TCP ports

Multiple package instancing

●Many versions installed at one time for testing and

compatibility

●Several instances of the same version with different

configurations

More flexibility

●Movable packages - at post-install time

●Non-privileged management

●Per-user/per-jail package masking

Versioning

●Easily accessible older versions

●Depending on older versions

●Keeping distfiles in a VCS repo

●Incremental distfile updates

Getting pristine sources

●RCS, VCS repositories

●P2P

●Metalinks

●Rsync

●Guided non-distributable downloads

User Interface

●Centralized management

●Orthogonal switches

●Advanced on-line help/hints system in addition to

manpages and other documentation

●Integrated updating capabilities

●Options handling

●Clicks

User Package Sets

●Storage of package sets, easy access/distribution

●Marking packages as requested or depended on

●Marking some big (sets of) packages as undesirable

to be installed

Technicalities

●distcc

●ccache

●Cross-compilation

Security

●CVE is a dictionary

●We need a database

●Different projects use CVE/database as an overlay

●Or, the database stores info about projects

Exchanging hacks

●Thousands of hacks spread out throughout the PM

systems

●Only a few are documented, most of the other ones

are abandoned and forgotten

●Developers have to reinvent solutions

Collaboration(ism)

●Education

●Spirit

●Communication

Education

●Other PM systems exist, and they do work

●Many of them have good documentation

●Behind projects there are people to talk to and to learn

from

Spirit

●No single PM system dominates the market

●Rich traditions of forks and new-from-scratch solutions

gave us diversity, but prevent us from trying to work

together

●Working with people/approaches we don't necessarily

like

Communication

●Actual developers making contacts

●Looking for solutions (in)

●Sharing our ideas (out)

●Taking part in outside discussions

●Inviting outsiders over to our forums

What's next

●More forays into the wild

●Mentioning the diversity of PM systems in docs

●Learning from others

●Bringing in foreign solutions

●Standardizing on decisions