Open Source Project Management

Semen Cirit

Agenda

• Introduction• Getting Started

– Choose good name– Choosing a license– Setting the tone– Announcing

• Technical infrastructure– Mailing lists, forums– Version Control– Bug tracker– Wikis– Translation infrastructure

Agenda

• Packaging, releasing and daily development– Release numbering– Release branches– Stablizing a release– Packaging– Testing and releasing– Managing multiple release lines

Introduction 90% - 95% of them fails. There are similar reasons with proprietary software project fails. (Un realistic requirements, vague

specifications, poor resource management etc) Opening up means:

Arranging the code to be comprehensible to complete strangers Setting up development documentation and email lists Often writing documentation for the first time. If any interested developers do show up, there is the added burden of answering their questions for a

while before seeing any benefit from their presence. Packaging is crucial that it is a priority from the very start of the project. Management is mostly to ensure that developers continue to believe same vision, by setting

standards for communications, by making sure useful developers don't get marginalized due to personal idiosyncracies, and in general by making the project a place developers want to keep coming back to.

Most Free Software Projects Fails

Introduction Richard Stallman: «We did not call our software "free software", because that term did not yet exist; but that

is what it was. Whenever people from another university or a company wanted to port and use a program, we gladly let them. If you saw someone using an unfamiliar and interesting program, you could always ask to see the source code, so that you could read it, change it, or cannibalize parts of it to make a new program.» (1970)

Hiring AI lab developers and devisin GNU General Public License (GPL) (1980) On first hearing the term "free software" many people mistakenly think it means just "zero-cost software.

«But the real reason for the re-labeling is a marketing one. We're trying to pitch our concept to the corporate world now. We have a winning product, but our positioning, in the past, has been awful. The term "free software" has been misunderstood by business persons, who mistake the desire to share with anti-commercialism, or worse, theft.»

Free vs. Open Source

Getting Started Choose a good name:

Gives some idea what the project does Easy to remember Get Domain name Own the name in the important namespaces

Have a clear vision statement:The description should be concrete, limiting, and above all, short. State that the project is free Choose a license Feature and Requirements list: The features and requirements list would give the details, clarifying the mission statement's

scope Development status: For new projects, they want to know the gap between the project's promise and current reality Source code download Version control, bug tracker, commnuications channels Developer guidelines: they explain how the developers interact with each other and with the users, and ultimately how things

get done. Demos, Screenshots, Videos, and Example Output

Getting Started Avoid private discussions:

The discussion will help train and educate new developers. The discussion will train you in the art of explaining technical issues to people who are not as familiar

with the software as you are. The discussion and its conclusions will be available in public archives forever.

Nip Rudeness in the Bud: People will say things in email that they would never say face-to-face. When you think you see it happening, make a post that stresses the importance of keeping the

discussion friendly, without accusing anyone of being deliberately poisonous. Codes of Conduct: Codes of conduct remind participants that the maintenance of a respectful and

welcoming environment is everyone's responsibility. Practice Conspicuous Code Review: Peer review in the open source world, and directly helps to maintain

software quality Be Open From Day One: the longer the project is run closed-source, the more difficult it will be to open up

later.

Technical Infrastructure Your project should have a prominently-placed description of all the available public forums, to give newcomers

guidance in deciding which ones to browse or post to first. A typical such description might say something like this:The mailing lists are the main day-to-day communication channels for the Scanley community. You don't have to be subscribed to post to a list, but if it's your first time posting (whether you're subscribed or not), your message may be held in a moderation queue until a human moderator has a chance to confirm that the message is not spam. We're sorry for this delay; blame the spammers who make it necessary.

Need to encourage people to reply to the list whenever possible Archiving:

Prompt updating: try to set the archiver to update itself every hour or so. Referential stability: Once a message is archived at a particular URL, it should remain accessible at that

exact same URL forever. Thread support: It should be possible to go from any individual message to the thread (group of related

messages) that the original message is part of Searchability:An archiver that doesn't support searching—on the bodies of messages, as well as on authors

and subjects—is close to useless. Mailing list / Message forum software: Discourse, Google groups, Sympa, Mailman, Dada

Mailing lists/ Message forums:

Technical Infrastructure The reason version control is so universal is that it helps with virtually every aspect of running a project:

inter-developer communications

release management

bug management

code stability and experimental development efforts

attribution and authorization of changes by particular developers

Using version control system:

Version everything: Keep not only your project's source code under version control, but also its web pages, documentation, FAQ, design notes, and anything else that people might want to edit

Browsability: The project's repository should be browsable on the Web. This means not only the ability to see the latest revisions of the project's files, but to go back in time and look at earlier revisions, view the differences between revisions, read log messages for selected changes, etc.

Use branches to avoid bottlenecks: It's easy to see how this ability helps collaborative development. People need the freedom to try new things without feeling like they're interfering with others' work.

Authorisation: don't spend too much time fiddling with the version control authorization system, unless you have a specific reason to. There are advantages to relying on human controls instead.

Receiving and reviewing contributors: Pull requests, Commit notifications / commit emails

Version control:

Technical Infrastructure Bug tracking systems are used to track not only bug reports, but new feature requests, one-time tasks, unsolicited patches—really anything that

has distinct beginning and end states, with optional transition states in between, and that accrues information over its lifetime.

For this reason, bug trackers are also called issue trackers, ticket trackers, defect trackers, artifact trackers, request trackers, etc. The reason version control is so universal is that it helps with virtually every aspect of running a project:

Lifecycle:

Someone files the ticket.

Others read the ticket, add comments to it, and perhaps ask the original filer for clarification on some points.

The bug gets reproduced.

The bug gets diagnosed: its cause is identified, and if possible, the effort required to fix it is estimated.

The ticket gets scheduled for resolution.

The bug gets fixed (or the task completed, or the patch applied, or whatever).

Bug Template:Reproducible steps:SeverityPriorityActual Result:Expected Result:User Impact: (All the app could not be used, a specific functionality does not work, A part of the functionality does not work, trivial)User Workaround: (Is there another way to achieve to the expected result. If it is not possible NA will be used)Environment:

Bug tracker:

Technical Infrastructure When open source software project wikis go bad, they usually go bad for the same reasons: lack of consistent organization and

editing, leading to a mess of outdated and redundant pages, and lack of clarity on who the target audience is for a given page or section.

Try to have a clear page organization strategy and even a pleasing visual layout, so that visitors (i.e., potential editors) will instinctively know how to fit their contributions in.

Wikis and Spam:

Never allow open, anonymous editing on your wiki

you may need to keep watch for spam edits from users who registered under false pretences for the purpose of spamming.

Choosing Wiki:

it's usually best to use the built-in wiki feature that most such sites offer

That way your wiki will be automatically integrated with your repository or other project permissions, and you can rely on the site's user account system instead of having a separate registration system for the wiki

Translation infrastructure:

"Translation work" here means not just the process of translating the software's documentation, but also its run-time user interface, error messages, etc into different languages, so that each user can interact with the software in their preferred language.

Tool Transfifex

Wikis and Translations

Agenda

• Packaging, releasing and daily development– Release numbering– Release branches– Stablizing a release– Packaging– Testing and releasing– Managing multiple release lines

Packaging, Releasing and Daily DevelopmentA release means that:

Some old bugs and new bugs have been fixed

New features may have been added

New configuration options may have been added, or the meanings of old options may have changed subtly.

The purpose of release numbering, therefore, is twofold: obviously the numbers should unambiguously communicate the ordering of releases (i.e., by looking at any two releases' numbers, one can know which came later), but also they should indicate as compactly as possible the degree and nature of the changes in the release.

Release numbers are groups of digits separated by dots: (Scanley 2.3, Singer 5.11.4)

In addition to the numeric components, projects sometimes tack on a descriptive label such as "Alpha" or "Beta (Scanley 2.3.0 (Alpha), Singer 5.11.4 (Beta))

In a typical three-component system, the first component is the major number, the second is the minor number, and the third is the micro number (sometimes also called the "patch" number)

Changes to the micro number only (that is, changes within the same minor line) must be both forward- and backward-compatible. The changes should be bug fixes only, or very small enhancements to existing features. New features should not be introduced in a micro release.

Changes to the minor number (that is, within the same major line) must be backward-compatible, but not necessarily forward-compatible. It's normal to introduce new features in a minor release, but usually not too many new features at once.

Changes to the major number mark compatibility boundaries. A new major release can be forward- and backward-incompatible. A major release is expected to have new features, and may even have entire new feature sets.

Release Numbering

Packaging, Releasing and Daily Development Frequent code merges are important Release branch is just a branch in the version control system (see branch), on which the code destined for

this release can be isolated from mainline development Releases without release branches has always resulted in some developers sitting idle while others—usually

a minority—work on getting the release out the door. overall development momentum is slowed the release is of poorer quality than it needed to be, because there were only a few people working

on it, and they were hurrying to finish so everyone else could get back to work it divides the development team psychologically, by setting up a situation in which different types of

work interfere with each other unnecessarily

Release Branches

Packaging, Releasing and Daily Development Stabilization is the process of getting a release branch into a releasable state; that is, of deciding which

changes will be in the release, which will not, and shaping the branch content accordingly fixes for severe bugs can go in, especially for bugs without workarounds Documentation updates are fine, as are fixes to error messages

Thus, the process of stabilizing a release is mostly about creating mechanisms for saying "no". Dictatorship by Release Owner

The group agrees to let one person be the release owner. It is necessary to choose someone with the technical competence to understand all the changes, and

the social standing and people skills to navigate the discussions leading up to the release without causing too many hurt feelings.

Voting on Changes the system should be arranged such that subgroups of developers must act in cooperation to get any

change into the release. A single "no" vote is enough to stop the change from being included. Naturally, any such vote must be

accompanied by a justification.

Stabilization

Packaging, Releasing and Daily Development Most users will probably not compile the sources themselves, but will instead install from pre-built binary

packages Name and layout: The name of the package should consist of the software's name plus the release number,

plus the format suffixes appropriate for the archive type (scanley-2.5.0.tar.gz, scanley-2.5.0.zip) README file explaining what the software does and what release this is, and giving pointers to other

resources, such as the project's web site, other files of interest, etc. INSTALL file giving instructions on how to build and install the software for all the operating systems it

supports COPYING or LICENSE file, giving the software's terms of distribution

Pre-releases: scanley-2.3.0-alpha1.tar.gz, scanley-2.3.0-beta3.tar.gz

Packaging

Packaging, Releasing and Daily Development before the distribution is made available to the world at large, it should be tested and approved by some minimum number

of developers, usually three or more. Approval is not simply a matter of inspecting the release for obvious flaws; ideally, the developers download the package,

build and install it onto a clean system, run the regression test suite Candidate releases:

The purpose of a candidate is to subject the code to wide testing before blessing it as an official release. (scanley-2.5.0-beta1)

If problems are found, they are fixed on the release branch and a new candidate release is rolled out (scanley-2.5.0-beta2)

The cycle continues until no unacceptable bugs are left, at which point the last candidate release becomes the official release

Announcing release: Whenever you write the URL to the downloadable release tarball, make sure to also write the MD5/SHA1 checksums In the announcement email, and on news pages that contain more than just a blurb about the release, make sure to

include the relevant portion of the CHANGES file don't forget to thank the development team, the testers, and all the people who took the time to file good bug

reports.

Testing and Releasing

Packaging, Releasing and Daily Development Most mature projects maintain multiple release lines in parallel. after 1.0.0 comes out, that line should continue with micro (bugfix) releases 1.0.1, 1.0.2, etc., until the

project explicitly decides to end the line. After the 1.1.x line is well under way, you can declare 1.0.x to be at end of life. This should be announced

officially. The announcement could stand alone, or it could be mentioned as part of a 1.1.x release announcement

Security bugs: A security release is a release made solely to close a security vulnerability. The code that fixes the bug cannot be made public until the release is available the developers can examine the fix among themselves, and test the release privately, but widespread

real-world testing is not possible. Have each commit be a single logical change, and don't mix unrelated changes in the same commit. If a change is too big or too disruptive to do in one commit, break it across N commits, where each commit

is a well-partitioned subset of the overall change, and includes nothing unrelated to the overall change.

Maintaining Multiple release lines

Gitflow Branches

Two historical (permanent) branchesDevelopment branchMaster branchAlfa, Beta, SDC, LA etc. Processes for each team

Temporary branchesFeature branchesRelease branchesHotfix branches

Gitflow Branches

Use development branch for integration branch of featuresUse master branch as a official release historyFeature branch use development branch as its parent Use release branch to polish the current release and let the other teams work on new features

Release branches are a fork of development branchAfter all work done release branch is merged into master and back to development and tagged with a version number

Use hotfix branch for quick fixes on productionHotfix branch is a fork of masterAfter all work done hotfix branch is merged into master and development and tagged an updated version number

Gitflow

Gitflow Feature Branch

All developers clone the develop branch to create a tracking branch on stashWhen a developer start a new feature create a new feature branch from develop on stashCommits his feature

git checkout feature/some-feature git statusgit add <some-file>git commitgit push origin feature/some-feature

Gitflow Development Branch

When feature is finished create pull request for develop branchAfter code review, reviewer can merge it to develop

git checkout develop git pull origin develop git merge origin/feature/some-feature git push origin develop git branch -d feature/some-feature (locally delete) git push origin :feature/some-feature

Gitflow Release Branch

New release is created from develop branchOnly bugfixes and documentation is added to this branch, branch is feature frozen

git checkout release/release-0.1 git status git add <some-file> git commit git push origin release/release-0.1

Gitflow Master Branch

When the release is ready for production create pull request for master After code review, reviewer can merge it to develop and master, and tag as a new release

git checkout develop git pull origin develop git merge origin/release/release-0.1 git push origin develop git checkout master git pull origin master git merge origin/release/release-0.1 git push origin master git branch -d release/release-0.1 (locally delete) git push origin :release/release-0.1 git tag -a 0.1 -m "Initial public release" master git push --tags

Gitflow Maintenance to Dev and Master Branch

When bug is finished create pull request for master and develop as the same for release pull requestAfter code review, reviewer can merge it to develop and master and tag as a new release

git checkout develop git pull origin develop git merge origin/bugfix/bug-001 git push origin develop git checkout master git pull origin master git merge origin/bugfix/bug-001 git push origin master git branch -d bugfix/bug001 (locally delete) git push origin :bugfix/bug-001 git tag -a 0.1.1 -m bugfix release 0.1.1" master git push --tags

Thank you