Code Management Working Practices

The previous chapter described how to use the various parts of the FCM code management system. They also described aspects of working practises which are enforced by the system. This section discusses other recommended working practises. They are optional in the sense that you don't have to follow them to use FCM. It is a matter for individual projects to decide which working practises to adopt (although we expect most projects/systems at the Met Office to adopt similar practises).

Making Changes

This sub-section gives an overview of the recommended approach for preparing changes. Particular topics are discussed in more detail in later sub-sections where appropriate.

The recommended process for making a change is as follows:

1. Before work starts on any coding you should make sure that there is a Trac ticket open which explains the purpose of the change.
   • Make sure that you set the ticket milestone to indicate which release of the system you are aiming to include your change in.
   • Accept the ticket to indicate that you are working on the change.
   • For further advice on using tickets see Trac Tickets later in this section.
2. Create a branch
   • For very simple changes you may be happy to prepare your changes directly on the trunk. For further details see When to Branch later in this section.
   • Create your branch either from the latest version or from a stable release (see Where to Branch From later in this section).
3. Prepare your code changes on the branch
   • Commit interim versions to your branch on a regular basis as you develop your change. This makes it much easier to keep track of what you're changing and to revert changes if necessary.
   • You may wish to merge in changes from the trunk. For further details see Merging From the Trunk later in this section.
     • Make sure that you always commit any local changes to your branch before doing a merge. Otherwise it becomes impossible to distinguish your changes from those you have merged in. It is also impossible to revert the merge without losing your local changes.
     • Likewise, always commit the merge to your branch (after resolving any conflicts) before making any further changes.
   • Don't include unrelated changes. If you want to make some changes which aren't really associated with your other changes then use a separate ticket and branch for these changes.
4. Once your changes are ready for review, update the Trac ticket to record which version of the branch is to be reviewed and assign the ticket to your reviewer.
5. If the reviewer is happy with the change then he/she should update the ticket to record that the change is approved and assign the ticket back to you.
   • The reviewer can use the command fcm diff --branch <branch_name> to examine all of the changes on the branch.
   • If changes are necessary then these should be prepared and then the ticket updated to refer to the new version under review.
6. Once the change is approved it can be merged back to the trunk
   • If you have been merging the latest changes from the trunk onto your branch then the merge should be automatic. If not you may have conflicts to resolve.
   • Make sure that each merge is a separate commit to the trunk. i.e. Don't combine changes from several branches in one commit. This makes it easier to reverse changes if necessary. It also makes the changeset easier to understand.
   • Make sure that you use a good log message to describe your change. For further details see Commit Log Messages later in this section.
   • Once the changes are commited, update the ticket to refer to the changeset. Then the ticket can be closed.
7. Once you are finished with the branch it should be deleted.

Working Copies

1. In general we recommend that you keep your working copies in your home directory. This ensures that any local changes which you accidently delete can be recovered via the snapshot facility (on Met Office Exeter and Reading based systems).
2. If the size of your project is small then you will probably find it easiest to work with a complete copy of the project (either the trunk or your branch). This means that you always have immediate access to all the files and that you are always able to perform merges using your normal working copy.
3. If you have a large project then you may prefer to work on a sub-tree of your project.
   • Pros:
     • Subversion operations on your working copy are faster.
     • Your working copies use up less disk space. Remember that you may be working on several changes at once on separate branches so you may wish to have several working copies.
   • Cons:
     • You cannot always perform merge operations in sub-trees (if the changes which need to be merged include files outside of your sub-tree). To handle this we suggest that if you need to perform a merge using a complete copy of your project you check it out in your $LOCALDATA area (local disk space which is not backed up) to be used purely for doing the merge.
     • You may find that your change involves more files than you originally thought and that some of the files to be changed lie outside of your working copy. You then have to make sure that you have committed any changes before checking out a larger working copy.

Branching & Merging

When to Branch

If you are making a reasonably large change which will take more than a hour or two to prepare then there are clear advantages to doing this work on a branch.

• You can commit intermediate versions to the branch.
• If you need to merge in changes from the trunk then you have a record of your files prior to the merge.
• The version of the code which gets reviewed is recorded. If subsequent changes are required then only those changes will need reviewing.

However, if you are only making a small change (maybe only one line) should you create a branch for this? There are two possible approaches:

1. The Always Branch system
   • ALL coding changes are prepared on branches.
   • Pros: Same process is followed in all cases.
   • Cons: The extra work required to create the branch and merge it back to the trunk may seem unnecessary for a very small change.
2. The Branch When Needed system
   • Small changes can be committed directly to the trunk (after testing and code review).
   • Pros: Avoids the overhead of using branches.
   • Cons: Danger of underestimating the size of a change. What you thought was a small change may turn out to be larger than you thought (although you can always move it onto a branch if this happens).

This is a matter for project policy although, in general, we would recommend the Branch When Needed approach.

Where to Branch From

When you create a new branch you have two choices for which revision to create the branch from:

1. The latest version of the trunk.
   • This is the preferred choice where possible. It minimised the chances of conflicts when you need to incorporate your changes back onto the trunk.
2. An older version of the trunk. There are a number of reasons why you may need to do this. For example:
   • You are using a stable version to act as your "control" data.
   • You need to know that your baseline is well tested (e.g. scientific changes).
   • Your change may need to be merged with other changes relative to a stable version for testing purposes or for use in a package (see Creating Packages later in this section).

Merging From the Trunk

Once you've created your branch you need to decide whether you now work in isolation or whether you periodically merge in the latest changes from the trunk.

• Regularly merging from the trunk minimises the work involved when you are ready to merge back to the trunk. You deal with any merge issues as you go along rather than all at the end (by which time your branch and the trunk could have diverged significantly).
• One downside of merging from the trunk is that the baseline for your changes is a moving target. This may not be what you want if you have some "control" results that you have generated.
• Another downside of merging from the trunk is that it may introduce bugs. Although any code on the trunk should have been tested and reviewed it is unlikely to be as well tested as code from a stable release.
• Unless you originally created your branch from the latest version of the trunk it is unlikely that you are going to want to merge in changes from the trunk. The exception to this is once your change is complete when it may make sense to merge all the changes on the trunk into your branch as a final step. This is discussed in Merging Back to the Trunk below.

So, there are basically three methods of working:

1. Branch from a stable version and prepare all your changes in isolation.
   • Necessary if you need to make your change relative to a well tested release.
2. Branch from the latest code but then prepare all your changes in isolation.
   • Necessary if you need a stable baseline for your "control" data.
3. Branch from the latest code and then update your branch from the trunk on a regular basis.
   • This is considered "best practise" for parallel working and should be used where possible.

Merging Back to the Trunk

Before merging your change back to the trunk you will need to test your change and get it reviewed. There are two options for what code to test and review:

1. Test and review your changes in isolation. Then merge to the trunk and deal with any conflicts at this stage. This may be the best method if:
   • Your changes have already been tested against a stable baseline and re-testing after merging would be impracticable.
   • Your branch needs to be available for others to merge in its changes in isolation.
2. Merge in the latest code from the trunk before your final test and review. This has the advantage that you are testing and reviewing the actual code which will be committed to the trunk. However, it is possible that other changes could get committed to the trunk whilst you are completing your testing and review. There are several ways of dealing with this:
   • Use locking to prevent it happening. The danger with this is that you may prevent others from being able to get their change tested and reviewed, hence inhibiting parallel devlopment.
   • Insist that the change is re-tested and reviewed. The problem with this is that there is no guarantee that the same thing won't happen again.
   • Merge in the new changes but don't insist on further testing or review.
     • In most cases any changes won't clash so there is little to worry about.
     • Where there are clashes then, in most cases, they will be trivial with little danger of any side-effects.
     • Where the clashes are significant then, in most cases, this will be very obvious whilst you are resolving the conflicts. In this case you should repeat the testing and get the updates reviewed.
     This is the recommended approach since it doesn't inhibit parallel development and yet the chances of a bad change being committed to the trunk are still very small.
   You should also consider what can be done to minimise the time taken for testing and review.
   • Try to keep your changes small by breaking them down where possible. Smaller changes are easier and quicker to review. This also helps to minimise merge problems by getting changes back onto the trunk earlier.
   • Automate your testing as far as possible to speed up the process.

Most projects will require the developer who prepared the change to merge it back to the trunk once it is complete. However, larger projects may wish to consider restricting this to a number of experienced / trusted developers.

• This makes it easier to control and prioritise the merges.
• It applies an extra level of quality control.
• It minimises the risk of mistakes being merged back on to the trunk by less experienced developers
• Scientific developers can concentrate on the scientific work.
• One issue is that the person doing the merge to the trunk may need help from the original developer to prepare a suitable log message.

When to Delete Branches

Once you are finished with your branch it is best to delete it to avoid cluttering up the directory tree (remember that the branch and all its history will still be available). There are two obvious approaches to deleting branches.

1. Delete the branch as soon as it has been merged back to the trunk (prior to closing any associated Trac ticket).
   • This is the tidiest approach which minimises the chances of old branches being left around.
2. Delete the branch once a stable version of the system has been released which incorporates your change.
   • If a bug is found in your change during integration testing then you can prepare the fix on the original branch (without having to do any additional work to restore the branch).

Working with Binary Files

The fcm conflicts command and xxdiff can only help you resolve conflicts in text files. If you have binary files in your repository you need to consider whether conflicts in these files would cause a problem.

Resolving Conflicts in Binary Files

Conflicts in some types of binary files can be resolved manually. When you are satisfied that the conflicts are resolved, issue the fcm resolved command on the file to remove the conflict status. (You will be prevented from committing if you have a conflicting file in your working copy.)

If you have a conflicting MS Office 2003 document, you may be able to take advantage of the "Compare and Merge Documents" facility under the "Tools" menu in a MS Office application. Consider a working copy, which you have just updated from revision 100 to revision 101, and someone else has committed some changes to a file doument.doc you are editing, you will get:

(SHELL PROMPT)$ fcm conflicts
Conflicts in file: document.doc
document.doc: ignoring binary file, please resolve conflicts manually.
(SHELL PROMPT)$ fcm status
=> svn st
?      document.doc.r100
?      document.doc.r101
C      document.doc

Open document.doc.r101 with MS Word. In Tools > Compare and Merge Documents..., open document.doc. You will be in Track Changes mode automatically. Go through the document to accept, reject or merge any changes. Save the document and exit MS Word when you are ready. Finally, issue the fcm resolved command to remove the conflict status:

(SHELL PROMPT)$ fcm resolved document.doc
=> svn resolved document.doc
Resolved conflicted state of 'document.doc'
(SHELL PROMPT)$ fcm status
=> svn st
M      document.doc

Another type of conflict that you may be able to resolve manually is where the binary file is generated from another file which can be merged. For instance, some people who use LaTeX also store a PDF version of the document in the repository. In such cases it is easy to resolve the conflict by re-generating the PDF file from the merged LaTeX file and then issuing the fcm resolved command to remove the conflict status. Note that, in this particular case, a better approach might be to automate the generation of the PDF file outside of the repository.

Using Locking

For files with binary formats, such as artwork or sound, it is often impossible to merge conflicting changes. In these situations, it is necessary for users to take strict turns when changing the file in order to prevent time wasted on changes that are ultimately discarded.

Subversion supports locking to allow you to prevent other users from modifying a file while you are preparing changes. For details please refer to the chapter Locking from the Subversion book. Note that:

• FCM does not add any functionality to the locking commands provided by Subversion.
• If you need to lock a file you must do this in a working copy of the trunk. There is nothing to stop you preparing the changes in a branch (maybe you want to prepare the change in combination with a number of other changes which do not require locking). However, you must always remember to lock the file in the trunk first to prevent other users from preparing changes to the file in parallel.
• Locking isn't the only way of preventing conflicts with binary files. If you only have a small project team and a small number of binary files you may find it easier to use other methods of communication such as emails or just talking to each other. Alternatively, you may have a working practise that particular files are only modified by particular users.

Commit Log Messages

Certain guidelines should be adhered to when writing log messages for code changes when committing to the trunk:

• Try to start off the log message with one line indicating the general nature of the change. This helps developers to tell whether a change is important to them when viewing the Trac timeline view.
• If you want to use bullets in your message then make them compatible with Wiki Formatting. For example:

  No bullet * First level bullet (single space at beginning) * Second level bullet (three spaces at beginning) 1. Numbered item instead of a bullet

  This will ensure that the log message is displayed with proper bullets in the Trac changeset view. You can also include other types of wiki formatting but please be aware that the message still needs to be readable when simply viewed as text (e.g. via fcm log).
• If your changes close a Trac ticket, make sure that your log message refers to this using Trac Links, e.g. Closes issue #26.
• Don't leave blank lines at the end of your log message since they get included in the message and, therefore, get included in the ouptut from fcm log.
• Take care to avoid making mistakes in your log messages since correcting them involves additional work. However, if you realise that that you've made a mistake don't leave it - get it corrected.
  • A log message can be corrected using the propedit command,
    e.g. fcm propedit svn:log --revprop -r REV
    Take care since this is an unversioned property so you run the risk of losing information if you aren't careful with your edits.
  • By default, FCM repositories are configured such that all users can update log messages. If you are not the original author of the changeset then the original author will be sent an e-mail informing them of the change. Other users can also be informed of log message changes if they wish (see the section Watching changes in log messages for details).

There are two possible approaches to recording the changes to individual files:

1. Maintain history entries in file headers.
   • Pros:
     • You don't need access to the Subversion repository in order to be able to view a files change history (e.g. external collaborators).
   • Cons:
     • History entries will produce clashes whenever files are changed in parallel (although these conflicts are trivial to resolve).
     • Source files which are changed regularly can become cluttered with very long history entries.
     • It is not possible to include history entries in some types of file.
2. Record which files have changed in the commit log message.
   • The log message should name every modified file and explain why it was changed. For example:

     * working_practices.html: Added guidelines for writing log messages.

     Make sure that the log message includes some sort of description for every change. The value of the log becomes much less if developers cannot rely on its completeness. Even if you've only changed comments, note this in the message.
   • If you make exactly the same change in several files, list all the changed files in one entry. For example:

     * code_management.html, system_admin.html, index.html: Ran pages through tidy to fix HTML errors.

   • It shouldn't normally be necessary to include the full path in the file name - just make sure it is clear which of the changed files you are referring to. You can get a full list of the files changed using fcm log -v.

When you're committing to your own branch then you can be much more relaxed about log messages. Use whatever level of detail you find helpful. However, if you follow similar guidelines then this will help when it comes to preparing the log message when your change is merged back to the trunk.

Trac Tickets

Creating Tickets

There are two different approaches to using the issue tracker within Trac:

1. All problems should be reported using Trac tickets.
   • Pros: The issue tracker contains a full record of all the problems reported and enhancements requested.
   • Cons: The issue tracker gets cluttered up with lots of inappropriate tickets.
     • Duplicate tickets.
     • Issues already discussed in the documentation.
     • Problems which turn out to be unrelated to the system.
     • Problems which are poorly described.
     • Things which would be better solved by a quick conversation.
     This makes it much harder to search the issues and can slow down the response to simple issues.
2. A Trac ticket shouldn't be created until the issue has been agreed.
   • Problems and issues should first be discussed with the project team / system maintainers. Depending on the project, this could be via email, on the newsgroups or through a quick chat over coffee.
   • Nothing is lost this way. Issues which are appropriate for the issue tracker still get filed. It just happens slightly later, after initial discussion has helped to clarify the best description for the issue.

Using Tickets

This sub-section provides advice on the best way of using tickets:

1. In general, mature systems will require that there is a Trac ticket related to every changeset made to the trunk. However this doesn't mean that there should be a separate ticket for each change.
   • If a change is made to the trunk and then a bug is subsequently found then, if this happens before the next release of the system, the subsequent change can be recorded on the same ticket.
   • There can often be changes which don't really affect the system itself since they are just system administration details. One way of dealing with this is to open a ticket for each release in which to record all such miscellaneous changes. It will probably be acceptable to review these changes after they have been committed, prior to the system release.
2. Whenever you refer to source files/directories in tickets, make sure that you refer to particular versions of the files. This ensures that the links will work in the future, even if those files are no longer in the latest version. For example:
   Changes now ready for review: source:/OPS/branches/dev/frdm/r123_MyBranch@234
3. For some types of information, simply appending to the ticket may not be the best way of working. For example, design notes or test results may be best recorded elsewhere, preferably in a wiki page. If using wiki pages we recommend using a naming convention to identify the wiki page with the associated ticket, for example:
   Please refer to [wiki:ticket/123/Design design notes]
   See separate [wiki:ticket/123/TestResults test results]
   Note that the square brackets have to be used since a page name containing numbers is not recognised automatically.

Creating Packages

Sometimes you may need to combine the changes from several different branches. For example:

• Your branch is just part of a larger change which needs to be tested in its entirety before committing to the trunk.
• You have some diagnostic code stored on a branch which you want to combine with another branch for testing purposes.

To create a package you simply create a new branch as normal. The type should be a package or possibly a configuration branch to help you distinguish it from your other branches. You then simply merge in all of the branches that you want to combine using fcm merge.

• The chance of conflicts will be reduced if the branches you are combining have been created from the same point on the trunk. Your package branch should also be created from the same point on the trunk.
  • Currently, fcm merge will not work unless this is true.
• If further changes are made on a branch you are using in a package then you can incorporate these changes into your package using fcm merge. Note, however, that if you have a branch which is being used in a package then you should avoid merging changes from the trunk into your branch. If you do then it will be very difficult to get updates to your branch merged into the package.

The fcm branch --info command is very useful for maintaining packages. It tells you all of the branches which have been merged into your package and whether there are any more recent changes on those branches.

Preparing System Releases

1. A system release is simply a particular version of the trunk. In order to do this it will be necessary to restrict changes on the trunk whilst the release is being prepared.
   • Users can continue to develop changes not intended for inclusion in this release on branches.
   • This may be a problem if preparing the release takes too long.
2. Create a release branch where the release is finalised.
   • You then lose the ability to be able to branch from the release.
   • It may be harder to identify what changes have been made between releases (since you can't simply look at all the changesets made between two versions of the trunk).

Rapid vs Staged Development Practises

Most of this section on working practises has focussed on projects/systems which are quite mature. Such systems are likely to have regular releases and will, for example, insist that all changes to the trunk are reviewed and tested.

If your system is still undergoing rapid development and has not yet reached any sort of formal release then you will probably want to adopt a much more relaxed set of working practises. For example:

• Changes don't need to be reviewed.
• More changes will be committed to the trunk. Only very large changes will be prepared on branches.
• No requirement to have a Trac ticket associated with each change.

We have tried to avoid building too many assumptions about working practises into the FCM system. This gives projects the flexibility to decide which working practises are appropriate for their system. Hopefully this means that FCM can be used for large or small systems and for rapidly evolving or very stable systems.