New URL for NEMO forge!   http://forge.nemo-ocean.eu

Since March 2022 along with NEMO 4.2 release, the code development moved to a self-hosted GitLab.
This present forge is now archived and remained online for history.
index.html in vendors/fcm/current/doc/collaboration – NEMO

source: vendors/fcm/current/doc/collaboration/index.html @ 1980

Last change on this file since 1980 was 1980, checked in by flavoni, 14 years ago

importing fcm vendor
File size: 21.5 KB
Line 
1<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
2    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
3
4<html xmlns="http://www.w3.org/1999/xhtml">
5<head>
6  <meta name="generator" content=
7  "HTML Tidy for Linux/x86 (vers 1st December 2004), see www.w3.org" />
8
9  <title>External Distribution &amp; Collaboration for FCM Projects</title>
10  <meta name="author" content="FCM development team" />
11  <meta name="descriptions" content=
12  "External Distribution &amp; Collaboration" />
13  <meta name="keywords" content="FCM, distribution, collaboration" />
14  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
15  <link rel="stylesheet" type="text/css" href="style.css" />
16<script type="text/javascript" src="fcm.js">
17</script>
18</head>
19
20<body onload="javascript: FCM.load('doc/collaboration/', 'collaboration');">
21  <div id="ukmo-logo">
22    <img src="logo.png" alt="UK Met Office" />
23  </div>
24
25  <h1>External Distribution &amp; Collaboration for FCM Projects</h1>
26
27  <address>
28    Latest content update: 02 Nov 2007. <span id="fcm-js-maintenance"></span>
29  </address>
30
31  <address>
32    Questions regarding this document or permissions to quote from it should be
33    directed to:
34  </address>
35
36  <address>
37    <a href="mailto:iprmanager@metoffice.gov.uk">IPR Manager</a><br />
38    Met Office<br />
39    FitzRoy Road<br />
40    Exeter, Devon<br />
41    EX1 3PB<br />
42    United Kingdom
43  </address>
44
45  <address>
46    &copy; Crown Copyright 2006-7
47  </address>
48
49  <address id="fcm-js-pdf"></address>
50
51  <hr />
52
53  <div id="fcm-content"></div>
54
55  <h2 id="introduction">Introduction</h2>
56
57  <p>This document describes how projects configured under FCM can be
58  distributed externally. Particular attention is given to collaborative
59  distributions - where the external user regularly returns code for
60  consolidation into the central repositories which hold the master copies of
61  the code.</p>
62
63  <p><dfn>Note:</dfn> This document assumes that the repositories are
64  inaccessible to the external user, due to issues of security and
65  practicality.</p>
66
67  <h2 id="distribution">Creating a Distribution</h2>
68
69  <p>A system configured under FCM can be distributed by packaging a known
70  revision (usually corresponding to a stable release) into an archive (e.g. a
71  tarball) of directories and files. Various issues need to be considered:</p>
72
73  <ul>
74    <li>A distribution may contain a variety of different files including
75    source code, scripts, benchmark and validation tests, documentation,
76    etc.</li>
77
78    <li>A system may consist of several different <em>projects</em> which
79    should be put into separate directories in the distribution. Please refer
80    to the section <a href=
81    "../user_guide/system_admin.html#svn_design">Repository design</a> in the
82    FCM user guide for an explanation of what is meant by a project in this
83    context.</li>
84
85    <li>Some files in a project may not be included in the distribution. This
86    may be because they are of no interest to external users or because of
87    license restrictions. Such files will need to be filtered out when creating
88    the distribution.</li>
89
90    <li>The distribution may also contain some files which are not maintained
91    under FCM version control (test results for instance).</li>
92
93    <li>Some systems share code with other systems.
94
95      <ul>
96        <li>If a distribution is intended to be used standalone then the
97        necessary files from these other systems will need to be included. e.g.
98        The VAR system requires code from the OPS and GEN systems.</li>
99
100        <li>If the distribution is part of a wider collaboration then it is
101        likely that the files from the other systems will be distributed
102        separately. It is best if stable releases of the various systems can be
103        synchronised so that, for example, a VAR stable release uses code from
104        an OPS stable release which both use code from the same GEN
105        release.</li>
106      </ul>
107    </li>
108
109    <li>Release notes should be prepared to accompany a distribution which
110    explain, among other things, how the distribution is structured.</li>
111
112    <li>The distribution should contain a file which identifies the repository
113    revision(s) contained in the distribution.</li>
114
115    <li>System managers will probably wish to maintain a script which automates
116    the generation of these distributions.</li>
117  </ul>
118
119  <h2 id="feedback">Feeding Back Changes</h2>
120
121  <p>Although we would encourage all collaborators to make use of the FCM
122  system for version control, we recognise that they may already have their own
123  preferred systems in place. There is no particular problem with this. The
124  main requirement is that any proposed changes are provided as a modification
125  relative to the provided distribution. The changeset could be provided in the
126  form of a modified project tree or as a patchfile (refer to the later section
127  <a href="#patchfiles">Exchanging Changesets using Patchfiles</a> for further
128  discussion). If the change involves any renaming or removal of files or
129  directories then special instructions should be provided plus a script to
130  perform the changes.</p>
131
132  <p>At the central repository, the changeset should be applied to a branch
133  created from the repository revision which formed the basis of the changeset
134  (possibly making use of the Subversion utility <a href=
135  "http://svnbook.red-bean.com/en/1.4/svn.advanced.vendorbr.html#svn.advanced.vendorbr.svn_load_dirs">
136  svn_load_dirs.pl</a>). Note that extra care is needed with changesets
137  provided as modified project trees if there are any files in the project
138  which are excluded from the distribution. Once imported, the changeset should
139  then undergo any necessary testing or review before being merged into the
140  trunk.</p>
141
142  <h2 id="usingfcm">Collaborating Using FCM for Version Control</h2>
143
144  <p>There are a number of advantages if the FCM system is used for version
145  control by the collaborator. In particular it means that:</p>
146
147  <ul>
148    <li>Collaborators will be able to see all of the individual changesets
149    which went in to a new release rather than only being able to view each new
150    release as one big change.</li>
151
152    <li>The process of sending a proposed change to the central repository can
153    be standardised through the use of an <em>FCM patch</em> (explained
154    later).</li>
155
156    <li>The FCM Extract system can be fully utilised.</li>
157
158    <li>Common tools will help to ease communication. We will all use technical
159    terms to mean the same thing.</li>
160  </ul>
161
162  <p>This section explains the recommended way of using FCM in a
163  collaboration.</p>
164
165  <h3 id="initsvn">Initialising the Subversion Repositories</h3>
166
167  <p>The collaborator needs to set up a repository and import each of the
168  projects. Please see the section <a href=
169  "../user_guide/system_admin.html#svn_create">Creating a repository</a> in the
170  FCM user guide for advice. Collaborators may wish to use separate
171  repositories and Trac systems for each project or they may prefer to use a
172  single repository for all projects and use a single Trac system. Either
173  option should be fine so long as the same set of projects is retained.</p>
174
175  <p>After completing the initial import, the collaborator should have the
176  required set of projects available in Subversion where the initial version of
177  the trunk of each project corresponds with the initial stable release
178  provided in the distribution.</p>
179
180  <h3 id="prepchanges">Preparing Changes at the Collaborator's Site</h3>
181
182  <p>The recommended way of preparing changes is illustrated in Figure 1a:</p>
183
184  <p class="image"><img src="working-as-collaborator.png" alt=
185  "Figure 1a: working at the collaborator's site" /><br />
186  Figure 1a: working at the collaborator's site</p>
187
188  <p>The collaborator will create a shared package branch from the latest
189  stable release on the trunk. This branch will contain all the changes that
190  will eventually be fed back to the central repository. Developers will also
191  create their own development branches. These may be branched from the latest
192  stable release on the trunk. Alternatively, if the change needs to build on
193  other changes then a branch can be created from the shared package branch.
194  When the changes are ready (i.e. tested, documented, reviewed, etc) then they
195  are merged into the shared package branch. The trunk is not used for the
196  shared changes as it is reserved for changes received from the central
197  repository.</p>
198
199  <p>Should it be required, a second shared package branch can be created from
200  the same point to contain any local modifications that will not be fed back
201  to the central repository. A configuration branch can then be used to combine
202  the local changes with those destined to be fed back. This is illustrated in
203  Figure 1b:</p>
204
205  <p class="image"><img src="managing-local-changes.png" alt=
206  "Figure 1b: managing local changes" /><br />
207  Figure 1b: managing local changes</p>
208
209  <h3 id="feedbackfcm">Feeding Back Changes Using FCM</h3>
210
211  <p>Eventually, a series of changesets will exist on the first package branch.
212  These changes will be fed back to the central repository via an <em>FCM
213  patch</em>. This contains a series of differences associated with changesets
214  in a given branch of development, created by the <code>fcm mkpatch</code>
215  command. For further information about the command, please refer to its
216  <a href="../user_guide/command_ref.html#fcm_svn_mkpatch">command
217  reference</a> in the FCM user guide.</p>
218
219  <p>At the central repository, the changeset will be applied to a branch
220  created from the repository revision which formed the basis of the changeset.
221  This is illustrated in Figure 2:</p>
222
223  <p class="image"><img src="feeding-back-patch.png" alt=
224  "Figure 2: feeding back changes" /><br />
225  Figure 2: feeding back changes</p>
226
227  <p>Patches will usually be exchanged in the form of a tarball. To apply the
228  patch it must first be extracted to a directory. In this directory there
229  should be a shell script called <code>fcm-import-patch</code>. A TARGET needs
230  to be specified when invoking the script. The TARGET must either be a URL or
231  a working copy of a valid project tree that can accept the import of the
232  patches. It is essential that this target matches the version of the project
233  from which the patch was created (usually this means a particular stable
234  release). The script contains a series of <code>cp</code> and
235  <code>svn</code> commands to import the changesets one by one. Note that the
236  changesets are committed automatically with no user interaction. It is worth
237  ensuring that an up to date backup of the repository is available in case of
238  problems.</p>
239
240  <h3 id="changescentral">Incorporating Changes into the Trunk of the Central
241  Repository</h3>
242
243  <p>Once the changes have undergone any necessary testing or review they can
244  be merged into the trunk. There are three ways of approaching this:</p>
245
246  <ol>
247    <li>As one changeset: all changes in the branch will be merged into the
248    trunk as a single changeset. This approach is the easiest and has the
249    advantage that any conflicts only need to be resolved once. However, the
250    drawback of this approach is that the logical changesets as fed back by the
251    collaborator will be combined into a large single changeset on the trunk,
252    which may not be the most desirable (although the logical changesets will
253    still be available to examine on the import branch). This is illustrated in
254    Figure 3a:
255
256      <p class="image"><img src="merging-patch-one.png" alt=
257      "Figure 3a: merging a patch in a single changeset" /><br />
258      Figure 3a: merging a patch in a single changeset</p>
259    </li>
260
261    <li>As multiple changesets: each changeset in the branch will be merged
262    into the trunk in order. This can be quite complicated and time consuming,
263    especially if you have a large number of changesets and there are a lot of
264    clashes. The advantage is that each logical changeset will retain its
265    logical identity, which may be more desirable in the long run, when you
266    come to inspect the history. This is illustrated in Figure 3b:
267
268      <p class="image"><img src="merging-patch-multi.png" alt=
269      "Figure 3b: merging a patch in multiple changesets" /><br />
270      Figure 3b: merging a patch in multiple changesets</p>
271    </li>
272
273    <li>As a mixture of the above: you may want to combine the above two
274    approaches when it makes sense to do so. For example, there may be a series
275    of small changesets that can be combined logically, or there may be a
276    changeset that fixes a bug introduced in the previous one. The bottom line
277    is that the project/system manager should use his/her own judgement in the
278    matter for what is best for the future of the project.</li>
279  </ol>
280
281  <h3 id="changescollab">Incorporating Updates at the Collaborator's Site</h3>
282
283  <p>Once a new stable release is available it will be supplied in the form of
284  a distribution tarball as described earlier. However, collaborators will also
285  be supplied with an <em>FCM patch</em> (as described earlier) for each
286  project containing all the changes made since the previous stable release.
287  Note that this assumes that stable releases are prepared on the trunk and not
288  in branches.</p>
289
290  <p>Each patch should be applied to the trunk of the collaborator's
291  repository. This means that the collaborator's trunk will always be mirroring
292  that of the central repository. This is illustrated in Figure 4:</p>
293
294  <p class="image"><img src="mirroring-trunk.png" alt=
295  "Figure 4: mirroring the trunk at the collaborator's site" /><br />
296  Figure 4: mirroring the trunk at the collaborator's site</p>
297
298  <p>In order to be certain that the patch has worked correctly, we recommend
299  that a check is performed to ensure that the new stable release on the trunk
300  matches the files provided in the distribution (preferably using a copy of
301  the repository for testing purposes before applying the patch to the live
302  repository).</p>
303
304  <h3 id="updatebranches">Updating Existing Branches</h3>
305
306  <p>Old branches that are still active at the collaborators site should be
307  updated to the latest stable release when it becomes available. Developers
308  should create a new branch from the latest stable release and then merge the
309  changes from the old branch to the new branch. The old branch should be
310  deleted once it is no longer required. This is illustrated in Figure 5a:</p>
311
312  <p class="image"><img src="updating-branch.png" alt=
313  "Figure 5a: updating a branch to the latest stable release" /><br />
314  Figure 5a: updating a branch to the latest stable release</p>
315
316  <p>Note that the merge will be easiest if the old branch was created from the
317  previous stable release. If it was created from the shared package branch
318  then a custom merge will be required to achieve the desired result (a normal
319  FCM merge command would choose the wrong base for comparison). This is
320  illustrated in Figure 5b:</p>
321
322  <p class="image"><img src="updating-shared-branch.png" alt=
323  "Figure 5b: updating a branch of the shared package branch" /><br />
324  Figure 5b: updating a branch of the shared package branch</p>
325
326  <h3 id="other">Other Scenarios</h3>
327
328  <p>The previous sections have only considered how developments on the trunk
329  of a central repository can be shared with a single collaborator. However,
330  the same techniques can be applied to more complex situations.</p>
331
332  <ul>
333    <li>If there are multiple external collaborators each working with their
334    own repository then hopefully it is clear that this does not alter things
335    in any way. Inevitably there will be an increased workload on the
336    maintainers of the central repository. There will also be an increased need
337    for coordination of planned code changes. However, the method of code
338    exchange is unaltered.</li>
339
340    <li>Sometimes there may be the need to collaborate on development of a
341    branch (i.e. to exchange code which is not yet ready to be incorporated
342    onto the trunk). The collaborator would maintain the trunk of their
343    repository as before, importing patches to keep their trunk alligned with
344    the stable releases from the central repository. In addition, they would
345    receive an <em>FCM patch</em> from the central repository representing the
346    changes on the shared branch relative to the stable release. The
347    collaborator should create a branch from the stable release and the patch
348    should then be imported onto this branch. They should then create a branch
349    from this branch on which to prepare their changes. When ready the changes
350    would be returned in the form of an <em>FCM patch</em>, and so on.
351    Hopefully it can be seen that the same process can be applied to this
352    shared branch as we have previously described for trunk developments.</li>
353  </ul>
354
355  <h3 id="alternative">An Alternative Branching Strategy</h3>
356
357  <p>We have described the branching strategy we believe will work best for
358  collaborators. However, this is by no means the only branching strategy that
359  can be used. In particular, some collaborators may prefer to keep the latest
360  copies of the code they are using on the trunk. This effectively means
361  getting rid of the shared package branches for shared and local changes and
362  merging all changes on to the trunk. A separate branch would be used for
363  keeping a pristine copy of the main site and merging changes from new stable
364  builds on to the trunk.</p>
365
366  <p>This approach is certainly possible and has the advantage that developers
367  at the collaborator's site may find it easier to work with. However there are
368  two disadvantages that need to be considered:</p>
369
370  <ol>
371    <li>Merging in changes from a new stable release may be more difficult. If
372    the new stable release includes changes which were fed back by the
373    collaborator then these will already be present on the collaborators trunk.
374    If these changes were modified in any way or if they overlap with other
375    changes then this will result in a conflict which could be tricky to
376    resolve.</li>
377
378    <li>Any changes which need to be fed back by the collaborator need to be
379    made relative to a stable release. However, changes will have been prepared
380    relative to some version of the trunk. This means that a separate branch
381    will need to be taken (from the branch containing the pristine copy of the
382    main site) and a custom merge will be required in order to achieve the
383    desired result.</li>
384  </ol>
385
386  <h3 id="patchfiles">Exchanging Changesets using Patchfiles</h3>
387
388  <p>In some cases, an <em>FCM patch</em> may not be the best way of exchanging
389  changesets. For instance, when distributing code changes which have not yet
390  been finalised, you probably wouldn't want to send a patch containing all the
391  individual commits to the branch on which the change is being developed. What
392  you want is a summary of the changes in a single changeset. In this case you
393  will often be better to use a patchfile (which can be applied using the Unix
394  command <code>patch</code>). A patchfile is simply the output from an
395  <code>fcm diff</code> command. For example:</p>
396  <pre>
397fcm diff --branch fcm:myproj-br/dev/frdm/r2134_my_branch &gt; my_patchfile
398</pre>
399
400  <p>The patchfile must be applied to a working copy of the project which
401  corresponds to the same revision from which the patchfile was generated. The
402  option <code>-p0</code> must be used with the <code>patch</code> command. For
403  example:</p>
404  <pre>
405patch -p0 &lt; my_patchfile
406</pre>
407
408  <p>Patchfiles have the advantage that they are simple to generate and
409  exchange and that they can combine the changes from a number of changsets
410  into one. However, they have a number of limitations such as:</p>
411
412  <ul>
413    <li>Binary files are ignored.</li>
414
415    <li>Deleted directories are ignored.</li>
416
417    <li>Deleted files are left as empty files.</li>
418
419    <li>Copied files appear as new files.</li>
420
421    <li>A moved file is treated as a deleted file and a new file.</li>
422  </ul>
423
424  <p>Fortunately these limitations will not be an issue for the majority of
425  changes and, where they are a problem, there are various options such as
426  providing additional instructions with the patchfile, using an <em>FCM
427  patch</em>, or exchanging a modified project tree.</p>
428
429  <h2 id="further">Further Considerations</h2>
430
431  <p>The previous sections have only considered the version control aspects of
432  a collaboration. This section lists some other aspects of the collaboration
433  which will need to be considered.</p>
434
435  <ul>
436    <li>The FCM build system can be used regardless of what version control
437    system is used. This avoids effort being wasted trying to maintain
438    compatibility with an alternate build system. It also ensures that any code
439    changes prepared by the collaborator are compatible with the coding
440    standards which the FCM build system requires. Even if there are good
441    reasons for the collaborator not to use FCM for version control, it is
442    highly recommended that the FCM build system is used (assuming that is what
443    is used at the central repository).</li>
444
445    <li>Coding standards should be agreed by all collaborators.</li>
446
447    <li>Working practises should be agreed which should define, amongst other
448    things, what level of testing, review and documentation is expected to
449    accompany any proposed change.</li>
450
451    <li>All parties in the collaboration should note the advice given in the
452    <a href="../user_guide/code_management.html#svn_problems">FCM user
453    guide</a> to avoid renaming files or directories unless you can ensure that
454    no-one is working in parallel on the affected areas of the project.</li>
455
456    <li><acronym title="intellectual property rights">IPR</acronym>, copyright
457    and license issues should be agreed by all collaborators.</li>
458  </ul>
459</body>
460</html>
Note: See TracBrowser for help on using the repository browser.