chris berarducci handspring, inc. - perforce

Merge as you go

Chris BerarducciHandspring, Inc.

Introduction - Handspring ØWho is Handspring?

• Palm OS licensee• 5 year old company• About 30 software developers• www.handspring.com

ØWhat are our products?• Treo 180, 270

• GSM based communicator

• Treo 300• CDMA based communicator

ProblemØTwo groups developing related code at the

same time• Developers working on features• Developers working on localization

ØBreaking each other• English US resource file changes• Corresponding changes need to be made in localization

resource file

ØRunning on different schedules• Localized builds tended to run 4 weeks behind English

US builds.

SolutionØTwo branches

• Domestic branch• Only domestic engineering changes• US English build built from this

• Localization branch• Most changes driven by Domestic change• Localized build built from this

• Everything goes back to the main tree through the localization branch

Main Branch

Domestic Branch

Localization Branch

New problemsØLocalization developers really want all

domestic changes ASAPØExtra merging work for the localization

engineerØBasic Perforce merge may collapse

multiple changes into one submission losing the detailed submission notes.

Merge as you go

ØReally two things:• Changed based merge

• The set of files submitted in one change in the Domestic branch are merged as one change into the localization branch.

• Do it automatically• Run as a daemon every N minutes• One daemon instance, data driven

• Merges can be added or halted by editing a data file.• Daemon does not stop and start for this.

Auto merge example

Main Branch

Branch A - Domestic Branch

Branch A - Localization Branch

152 155

158

154

157

153

156 159

The BenefitsØ Increased productivity

• Complete merges without conflicts automatically.• Engineers get involved when there is a conflict.

ØMerges are scheduled• Merge more – results in smaller easier to resolve

integrations.ØMore complete submission notes

• Change based merge, 1:1 relationship• Original submission notes carry forward

When to use?ØWhen does “Merge as you go” makes sense?

• Our general rule is when similar projects are being developed on different branches

• Handspring utilized this for project development that had both a English US and localized component.

• Provided isolation for English US development.• Provided localized branch with latest changes in real

time.• Win Win situation for us.

• Engineering sandbox• Developer wants to work in private branch.

• Keeps them up to date automatically

Just another toolØTools have appropriate usages

ØNot all tools fit all jobs

ØUse them where they are appropriate

Components

ØSupport scripts• Mergechanges• Mergereview

ØRoles and responsibilities• Development Engineering• Tools Engineering

ØProcess Flow

Mergechanges.plØPerl scriptØ Input

• Branch specification• Change number• Optional – additional perforce merge flags

ØResult• Specified change merged to destination branch• Conflict aborts the submission• Option to send mail based on configuration file

Branch Specification

Branch: Branch_A_Domestic

…

View:

//branches/BranchA_Localized/… //branches/BranchA_Domestic/…

Branch A - Localized Branch


Notification configuration file

ØDetermine who to notify…

ØFormat:• Branch Name[_reverse] – The name of the branch. • Merge OK – List of folks to notify if merge is successful• Merge Conflict – List of folks to notify if merge fails.

ØExample:Branch: Branch_A_Domestic_reverseMERGE OK: [email protected] CONFLICT: [email protected]

Sample mergeØMergechange command:

mergechanges.pl -s -c=151 -b=Branch_A_Domestic –m –r

ØFlags:• -s: submit the change, otherwise a change is created with notes but not

submitted.• -c: The change to merge• -b: The branch to use• -m: Send mail as documented in the notification configuration file.• -r: merge direction is reversed from syntax in the branch specification.

Successful mergeØMerge Occurs

• No conflict• Results in change 157• Success Mail sent to:

• “[email protected]”

Branch A - Localized Branch


150 151

154 156 157

Mergechanges resultØP4 describe –s 157

• Change 157 by reviewdaemon@_reviewdaemon_mycomputer_software on 2002/09/17 15:39:36"MergeChanges - Branch_A_Domestic_reverse_Automerge">> Change 151 by JoeEngineer@JoeEngineer_2kPC on 2002/09/17 15:37:37>> >> fixed bug 25 Device crashes after repeatedly changing fonts>> >> Affected files ...>> >> ... //branches/BranchA_Domestic/Libraries/TextEngine/TextEngine.c#21 edit>>

Affected files ...

... //branches/BranchA_Localized/Libraries/TextEngine/TextEngine.c#26 integrate

Components

üSupport scriptsüMergechanges• Mergereview


ØProcess Flow

Mergereview.plØ Perl daemon scriptØ Input read via configuration file

• Branch specification name• Direction of merge

• Branch specification as written • Branch specification mappings reversed

• Text Filter• Change Number to start merging at

• Only used the first time• Subsequent runs utilize a P4 counter to keep track.

Ø Result• Execute mergechanges.pl with appropriate parameters.• Perforce counter keeps track of last change merged.

Merge configuration file

ØDetermine which branches will be auto merged.

ØFormat:• Branch Name – The perforce branch specification that defines

which files are in the view.• Direction – Either as written in the branch specification or

reversed.• Text Filter – Pattern match that must evaluate to true for auto

merge to occur.• Change number – Perforce number to initially start with.

Sample configuration file

ØFormat details:• Lines that begin with ‘#’ are comments• Each line is a configured merge

ØExample:#This is Chris’s auto merge configuration#Branch_A_Domestic is the branch specification#I Want to push changes from branch to original tree, so reverse#No text filter#Start with change 150

Branch_A_Domestic,reverse,,150

Sample configuration file cont.

Branch_A_Domestic,reverse,,150

ØWould configure the following:• A Perforce counter

• “Branch_A_Domestic_Reverse”

• Set the counter to change 150• Execute mergechanges.pl when there are

changes that should be merged.

Text Filter (side track)ØConditional merge ability

• Mergechanges puts a known header based on branch name and direction in merged submission notes.

• If text is specified in “merge configuration” file then that text must exist in source submission notes. If not the merge is not attempted.

• Configuration looks like:May,,April_reverse_automerge,300

• Only changes 306 and 313 are auto merged into the May branch

Main Branch

Branchname “April”

307

306 313

305 312

199

Shorthand for a largenumber of submissions and

time passing

320

Branchname“May”

309

314

Components

üSupport scriptsüMergechangesüMergereview


ØProcess Flow

Roles and Responsibility

ØTools team• Maintain Mergereview and Mergechanges

• Fix bugs• Add enhancements

ØDevelopment Engineers• Maintain the configuration files

• Merge and Notify configuration files

• Resolve conflicts when they occur

Components


üRoles and responsibilitiesüDevelopment EngineeringüTools Engineering

ØProcess Flow

Process flow

Edit the mergeconfiguration file toeither configure a

branch to merge orstop an existing automerge configuration

Merge daemonreads the

configuration fileand looks formerges to be

attempted

Update thePerforce counter

value. Sendemail to the

branch owner

Change ismerged

Attempt the mergemanually using the

same Perforcecommands as the

merge daemon (theywere in the email)

DO NOT updatethe Perforce

counter value.Send email to the

branch owner

Sleep for Nminutes

Resolve theconflict and do a

Perforcesubmission

Merge Daemon Process

Project LeadProject Lead

Submitconfiguration file

to Perforce SuccessfulMerge

Merge conflictencountered

Done

Done

Components


üRoles and responsibilitiesüDevelopment EngineeringüTools Engineering

üProcess Flow

ExampleØ Project A separated into two branches

• Branch A domestic• Isolated for domestic US changes only. Mostly non-localized development

• Branch A localized• Localization changes

Ø All changes merge to the main branch.Ø Perforce branch specifications written

• Branch_A_Domestic• Branch_A_Localized

Main Branch



Configure Auto MergeØProject Engineer adds the following merge

configuration lines:• Branch_A_Domestic,reverse,,150• Branch_A_Localized,reverse,,150

ØProject Engineer add the following Notification configuration lines• Branch: Branch_A_Domestic_reverse• MERGE OK: [email protected]• MERGE CONFLICT: [email protected]

• Branch: Branch_A_Localized_reverse• MERGE OK: [email protected]• MERGE CONFLICT: [email protected]

ØSubmit files to Perforce

Daemon Executes

ØDaemon reads the merge configuration fileØCreate Perforce counters

• Branch_A_Domestic_Reverse• Branch_A_Localized_Reverse

ØBoth are set to change 150ØNo changes submitted since change 150

Main Branch



Changes submitted

ØChange 152 – Submitted main branchØChange 153 – Submitted Domestic branchØMergereview daemon executes

• Reads first merge configuration• Domestic to Localized branch

• Executes mergechanges to merge change 153 to localization branch. Change 154 submitted

• Updates counter Branch_A_Domestic_Reverse to 153

Main Branch



152

154

153

Changes submitted cont.

ØMergereview daemon executes• Reads second merge configuration

• Localized to Main branch

• Calls mergechanges to merge change 154 to the main branch. Change 155 submitted

• Updates counter Branch_A_Localized_Reverse to 154

Main Branch



152

154

153

155

More changes submitted

ØChange 156 – Submitted main branch• Edit foo.c line 300

ØChange 158 – Submitted Domestic branch• Edit foo.c line 299 through 305

v Both submissions changed the same file and section of code.

Main Branch



152 155

158

154

153

156

Conflict

Ø Mergereview daemon executes• Reads first merge configuration

• Domestic to Localized branch• Executes mergechanges to merge change 158 to the

main branch. Conflict detected• Merge aborted, Files reverted, P4 counter not updated• Mail sent to appropriate engineers

• Reads second merge configuration• Localized to main branch

• Change 156 merged to Main branch• Change 159 submitted

Main Branch



152 155

158

154

159

153

156

Conflict files reverted

Conflict still

Ø Mergereview daemon executes again• Reads first merge configuration

• Domestic to Localized branch

• Executes mergechanges to merge change 158 to the main branch. Conflict detected again

• Merge aborted, Files reverted, P4 counter not updated• No mail sent this time

• Reads second merge configuration• No changes to merge

Main Branch



152 155

158

154

159

153

156

Conflict files reverted

Conflict resolved

ØEngineer manually integrates change 158 to the localized branch• Change 163 submitted

Main Branch



152 155

158

154

159

153

156 163

Eng resolves conflict

Daemon executes

ØMergereview daemon executes• Reads first merge configuration

• Domestic to Localized branch

• No outstanding integrations exist• So no conflict!

• “Project_A_Domestic_Reverse” Counter updated to 163

Main Branch



152 155

158

154

159

153

156 163

Daemon executes cont.

ØMergereview daemon executes• Reads second merge configuration

• Localized to Main branch

• Change 163 merged to main branch• “Branch_A_Localized_Reverse” counter

updated to 163.

Main Branch



152 155

158

154

159

153

156 163

164

Summary

ØApproach StrengthsØApproach WeaknessesØClosingØQuestions?

Approach StrengthsØEasy to turn on and off

• Helps keep the tree stable when near milestone.ØProcess is defined and documented

• Participants do their part.• Engineers like it

• It is distributed so not a heavy burden on any one person.

ØUtilizes features in Perforce• Standard merge and resolve commands

• Perforce always knows what merge was completed.

• Relies on Perforce merge behavior when a conflict occurs.

• Relies on Perforce Branch specification

Approach WeaknessesØDevelopment overhead added

• Somebody to write and maintain scripts.ØAutomation can impact Perforce performance

• Must be used appropriatelyØExtra learning curve for engineersØMergechanges relies on Branch specification

• Not flexible enough to do a change based merge across branches that don’t have a branch specification

ClosingØIt works well for Handspring when used

appropriately.• It does not solve all branching issues.

• A good branch practice is still required.

ØEasy to support and makes good use of Perforce branching abilityØEngineering views “Merge as you go” as

a benefit and generally like it.

Questions

Questions?

chris berarducci handspring, inc. - perforce

Documents