source: LMDZ6/branches/Amaury_dev/tools/fcm/doc/collaboration/index.html

Last change on this file was 5129, checked in by abarral, 4 months ago

Re-add removed by mistake fcm
File size: 22.3 KB
Line 
1<!DOCTYPE html>
2<html>
3<head>
4  <title>FCM: External Distribution &amp; Collaboration for FCM Projects</title>
5  <meta name="author" content="FCM team" />
6  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7  <link rel="icon" href="../etc/fcm-icon.png" type="image/png" />
8  <link rel="shortcut icon" href="../etc/fcm-icon.png" type="image/png" />
9  <link href="../etc/bootstrap/css/bootstrap.min.css" rel="stylesheet" media="screen" />
10  <link href="../etc/fcm.css" rel="stylesheet" media="screen" />
11</head>
12<body>
13  <div class="navbar navbar-inverse">
14    <div class="container-fluid">
15      <div class="navbar-header">
16        <a class="navbar-brand" href=".."><span class="fcm-version">FCM</span></a>
17      </div>
18      <div class="collapse navbar-collapse">
19        <ul class="nav navbar-nav">
20          <li><a href="../installation/">Installation</a></li>
21
22          <li><a href="../user_guide/">User Guide</a></li>
23        </ul>
24      </div>
25    </div>
26  </div>
27
28  <div class="page-header">
29    <div class="fcm-page-content pull-right well well-sm"></div>
30    <h1>FCM: Distribution &amp; Collaboration for FCM Projects</h1>
31  </div>
32
33  <div class="container">
34  <div class="row">
35  <div class="col-md-12">
36
37  <h2 id="introduction">Introduction</h2>
38
39  <p>This document describes how projects configured under FCM can be
40  distributed externally. Particular attention is given to collaborative
41  distributions - where the external user regularly returns code for
42  consolidation into the central repositories which hold the master copies of
43  the code.</p>
44
45  <p><dfn>Note:</dfn> This document assumes that the repositories are
46  inaccessible to the external user, due to issues of security and
47  practicality.</p>
48
49  <h2 id="distribution">Creating a Distribution</h2>
50
51  <p>A system configured under FCM can be distributed by packaging a known
52  revision (usually corresponding to a stable release) into an archive (e.g. a
53  tarball) of directories and files. Various issues need to be considered:</p>
54
55  <ul>
56    <li>A distribution may contain a variety of different files including
57    source code, scripts, benchmark and validation tests, documentation,
58    etc.</li>
59
60    <li>A system may consist of several different <em>projects</em> which
61    should be put into separate directories in the distribution. Please refer
62    to the section <a href=
63    "../user_guide/system_admin.html#svn_design">Repository design</a> in the
64    FCM user guide for an explanation of what is meant by a project in this
65    context.</li>
66
67    <li>Some files in a project may not be included in the distribution. This
68    may be because they are of no interest to external users or because of
69    license restrictions. Such files will need to be filtered out when creating
70    the distribution.</li>
71
72    <li>The distribution may also contain some files which are not maintained
73    under FCM version control (test results for instance).</li>
74
75    <li>Some systems share code with other systems.
76
77      <ul>
78        <li>If a distribution is intended to be used standalone then the
79        necessary files from these other systems will need to be included. e.g.
80        The VAR system requires code from the OPS and GEN systems.</li>
81
82        <li>If the distribution is part of a wider collaboration then it is
83        likely that the files from the other systems will be distributed
84        separately. It is best if stable releases of the various systems can be
85        synchronised so that, for example, a VAR stable release uses code from
86        an OPS stable release which both use code from the same GEN
87        release.</li>
88      </ul>
89    </li>
90
91    <li>Release notes should be prepared to accompany a distribution which
92    explain, among other things, how the distribution is structured.</li>
93
94    <li>The distribution should contain a file which identifies the repository
95    revision(s) contained in the distribution.</li>
96
97    <li>System managers will probably wish to maintain a script which automates
98    the generation of these distributions.</li>
99  </ul>
100
101  <h2 id="feedback">Feeding Back Changes</h2>
102
103  <p>Although we would encourage all collaborators to make use of the FCM
104  system for version control, we recognise that they may already have their own
105  preferred systems in place. There is no particular problem with this. The
106  main requirement is that any proposed changes are provided as a modification
107  relative to the provided distribution. The changeset could be provided in the
108  form of a modified project tree or as a patchfile (refer to the later section
109  <a href="#patchfiles">Exchanging Changesets using Patchfiles</a> for further
110  discussion). If the change involves any renaming or removal of files or
111  directories then special instructions should be provided plus a script to
112  perform the changes.</p>
113
114  <p>At the central repository, the changeset should be applied to a branch
115  created from the repository revision which formed the basis of the changeset
116  (possibly making use of the Subversion utility <a href=
117  "http://svnbook.red-bean.com/en/1.4/svn.advanced.vendorbr.html#svn.advanced.vendorbr.svn_load_dirs">
118  svn_load_dirs.pl</a>). Note that extra care is needed with changesets
119  provided as modified project trees if there are any files in the project
120  which are excluded from the distribution. Once imported, the changeset should
121  then undergo any necessary testing or review before being merged into the
122  trunk.</p>
123
124  <h2 id="usingfcm">Collaborating Using FCM for Version Control</h2>
125
126  <p>There are a number of advantages if the FCM system is used for version
127  control by the collaborator. In particular it means that:</p>
128
129  <ul>
130    <li>Collaborators will be able to see all of the individual changesets
131    which went in to a new release rather than only being able to view each new
132    release as one big change.</li>
133
134    <li>The process of sending a proposed change to the central repository can
135    be standardised through the use of an <em>FCM patch</em> (explained
136    later).</li>
137
138    <li>The FCM Extract system can be fully utilised.</li>
139
140    <li>Common tools will help to ease communication. We will all use technical
141    terms to mean the same thing.</li>
142  </ul>
143
144  <p>This section explains the recommended way of using FCM in a
145  collaboration.</p>
146
147  <h3 id="initsvn">Initialising the Subversion Repositories</h3>
148
149  <p>The collaborator needs to set up a repository and import each of the
150  projects. Please see the section <a href=
151  "../user_guide/system_admin.html#svn_create">Creating a repository</a> in the
152  FCM user guide for advice. Collaborators may wish to use separate
153  repositories and Trac systems for each project or they may prefer to use a
154  single repository for all projects and use a single Trac system. Either
155  option should be fine so long as the same set of projects is retained.</p>
156
157  <p>After completing the initial import, the collaborator should have the
158  required set of projects available in Subversion where the initial version of
159  the trunk of each project corresponds with the initial stable release
160  provided in the distribution.</p>
161
162  <h3 id="prepchanges">Preparing Changes at the Collaborator's Site</h3>
163
164  <p>The recommended way of preparing changes is illustrated in Figure 1a:</p>
165
166  <p><strong>Figure 1a: working at the collaborator's
167  site</strong></p>
168
169  <p><img src="working-as-collaborator.png" alt=
170  "Figure 1a: working at the collaborator's site" class="img-polaroid" /></p>
171
172  <p>The collaborator will create a shared package branch from the latest
173  stable release on the trunk. This branch will contain all the changes that
174  will eventually be fed back to the central repository. Developers will also
175  create their own development branches. These may be branched from the latest
176  stable release on the trunk. Alternatively, if the change needs to build on
177  other changes then a branch can be created from the shared package branch.
178  When the changes are ready (i.e. tested, documented, reviewed, etc) then they
179  are merged into the shared package branch. The trunk is not used for the
180  shared changes as it is reserved for changes received from the central
181  repository.</p>
182
183  <p>Should it be required, a second shared package branch can be created from
184  the same point to contain any local modifications that will not be fed back
185  to the central repository. A configuration branch can then be used to combine
186  the local changes with those destined to be fed back. This is illustrated in
187  Figure 1b:</p>
188
189  <p><strong>Figure 1b: managing local changes</strong></p>
190
191  <p ><img src="managing-local-changes.png" alt=
192  "Figure 1b: managing local changes" class="img-polaroid" /></p>
193
194  <h3 id="feedbackfcm">Feeding Back Changes Using FCM</h3>
195
196  <p>Eventually, a series of changesets will exist on the first package branch.
197  These changes will be fed back to the central repository via an <em>FCM
198  patch</em>. This contains a series of differences associated with changesets
199  in a given branch of development, created by the <code>fcm mkpatch</code>
200  command. For further information about the command, please refer to its
201  <a href="../user_guide/command_ref.html#fcm-mkpatch">command
202  reference</a> in the FCM user guide.</p>
203
204  <p>At the central repository, the changeset will be applied to a branch
205  created from the repository revision which formed the basis of the changeset.
206  This is illustrated in Figure 2:</p>
207
208  <p><strong>Figure 2: feeding back changes</strong></p>
209  <p><img src="feeding-back-patch.png" alt=
210  "Figure 2: feeding back changes" class="img-polaroid" /></p>
211
212  <p>Patches will usually be exchanged in the form of a tarball. To apply the
213  patch it must first be extracted to a directory. In this directory there
214  should be a shell script called <code>fcm-import-patch</code>. A TARGET needs
215  to be specified when invoking the script. The TARGET must either be a URL or
216  a working copy of a valid project tree that can accept the import of the
217  patches. It is essential that this target matches the version of the project
218  from which the patch was created (usually this means a particular stable
219  release). The script contains a series of <code>cp</code> and
220  <code>svn</code> commands to import the changesets one by one. Note that the
221  changesets are committed automatically with no user interaction. It is worth
222  ensuring that an up to date backup of the repository is available in case of
223  problems.</p>
224
225  <h3 id="changescentral">Incorporating Changes into the Trunk of the Central
226  Repository</h3>
227
228  <p>Once the changes have undergone any necessary testing or review they can
229  be merged into the trunk. There are three ways of approaching this:</p>
230
231  <ol>
232    <li>As one changeset: all changes in the branch will be merged into the
233    trunk as a single changeset. This approach is the easiest and has the
234    advantage that any conflicts only need to be resolved once. However, the
235    drawback of this approach is that the logical changesets as fed back by the
236    collaborator will be combined into a large single changeset on the trunk,
237    which may not be the most desirable (although the logical changesets will
238    still be available to examine on the import branch). This is illustrated in
239    Figure 3a:
240    </li>
241  </ol>
242
243  <p><strong>Figure 3a: merging a patch in a single changeset</strong></p>
244
245  <p><img src="merging-patch-one.png" alt=
246  "Figure 3a: merging a patch in a single changeset"
247  class="img-polaroid" /></p>
248
249  <ol start="2">
250    <li>As multiple changesets: each changeset in the branch will be merged
251    into the trunk in order. This can be quite complicated and time consuming,
252    especially if you have a large number of changesets and there are a lot of
253    clashes. The advantage is that each logical changeset will retain its
254    logical identity, which may be more desirable in the long run, when you
255    come to inspect the history. This is illustrated in Figure 3b:
256    </li>
257  </ol>
258
259  <p><strong>Figure 3b: merging a patch in multiple changesets</strong></p>
260
261  <p><img src="merging-patch-multi.png" alt=
262  "Figure 3b: merging a patch in multiple changesets"
263  class="img-polaroid" /></p>
264
265  <ol start="3">
266    <li>As a mixture of the above: you may want to combine the above two
267    approaches when it makes sense to do so. For example, there may be a series
268    of small changesets that can be combined logically, or there may be a
269    changeset that fixes a bug introduced in the previous one. The bottom line
270    is that the project/system manager should use his/her own judgement in the
271    matter for what is best for the future of the project.</li>
272  </ol>
273
274  <h3 id="changescollab">Incorporating Updates at the Collaborator's Site</h3>
275
276  <p>Once a new stable release is available it will be supplied in the form of
277  a distribution tarball as described earlier. However, collaborators will also
278  be supplied with an <em>FCM patch</em> (as described earlier) for each
279  project containing all the changes made since the previous stable release.
280  Note that this assumes that stable releases are prepared on the trunk and not
281  in branches.</p>
282
283  <p>Each patch should be applied to the trunk of the collaborator's
284  repository. This means that the collaborator's trunk will always be mirroring
285  that of the central repository. This is illustrated in Figure 4:</p>
286
287  <p><strong>Figure 4: mirroring the trunk at the
288  collaborator's site</strong></p>
289
290  <p><img src="mirroring-trunk.png" alt=
291  "Figure 4: mirroring the trunk at the collaborator's site"
292  class="img-polaroid"/></p>
293
294  <p>In order to be certain that the patch has worked correctly, we recommend
295  that a check is performed to ensure that the new stable release on the trunk
296  matches the files provided in the distribution (preferably using a copy of
297  the repository for testing purposes before applying the patch to the live
298  repository).</p>
299
300  <h3 id="updatebranches">Updating Existing Branches</h3>
301
302  <p>Old branches that are still active at the collaborators site should be
303  updated to the latest stable release when it becomes available. Developers
304  should create a new branch from the latest stable release and then merge the
305  changes from the old branch to the new branch. The old branch should be
306  deleted once it is no longer required. This is illustrated in Figure 5a:</p>
307
308  <p><strong>Figure 5a: updating a branch to the latest
309  stable release</strong></p>
310
311  <p><img src="updating-branch.png" alt=
312  "Figure 5a: updating a branch to the latest stable release"
313  class="img-polaroid"/></p>
314
315  <p>Note that the merge will be easiest if the old branch was created from the
316  previous stable release. If it was created from the shared package branch
317  then a custom merge will be required to achieve the desired result (a normal
318  FCM merge command would choose the wrong base for comparison). This is
319  illustrated in Figure 5b:</p>
320
321  <p><strong>Figure 5b: updating a branch of the shared package
322  branch</strong></p>
323
324  <p><img src="updating-shared-branch.png" alt=
325  "Figure 5b: updating a branch of the shared package branch"
326  class="img-polaroid"/></p>
327
328  <h3 id="other">Other Scenarios</h3>
329
330  <p>The previous sections have only considered how developments on the trunk
331  of a central repository can be shared with a single collaborator. However,
332  the same techniques can be applied to more complex situations.</p>
333
334  <ul>
335    <li>If there are multiple external collaborators each working with their
336    own repository then hopefully it is clear that this does not alter things
337    in any way. Inevitably there will be an increased workload on the
338    maintainers of the central repository. There will also be an increased need
339    for coordination of planned code changes. However, the method of code
340    exchange is unaltered.</li>
341
342    <li>Sometimes there may be the need to collaborate on development of a
343    branch (i.e. to exchange code which is not yet ready to be incorporated
344    onto the trunk). The collaborator would maintain the trunk of their
345    repository as before, importing patches to keep their trunk alligned with
346    the stable releases from the central repository. In addition, they would
347    receive an <em>FCM patch</em> from the central repository representing the
348    changes on the shared branch relative to the stable release. The
349    collaborator should create a branch from the stable release and the patch
350    should then be imported onto this branch. They should then create a branch
351    from this branch on which to prepare their changes. When ready the changes
352    would be returned in the form of an <em>FCM patch</em>, and so on.
353    Hopefully it can be seen that the same process can be applied to this
354    shared branch as we have previously described for trunk developments.</li>
355  </ul>
356
357  <h3 id="alternative">An Alternative Branching Strategy</h3>
358
359  <p>We have described the branching strategy we believe will work best for
360  collaborators. However, this is by no means the only branching strategy that
361  can be used. In particular, some collaborators may prefer to keep the latest
362  copies of the code they are using on the trunk. This effectively means
363  getting rid of the shared package branches for shared and local changes and
364  merging all changes on to the trunk. A separate branch would be used for
365  keeping a pristine copy of the main site and merging changes from new stable
366  builds on to the trunk.</p>
367
368  <p>This approach is certainly possible and has the advantage that developers
369  at the collaborator's site may find it easier to work with. However there are
370  two disadvantages that need to be considered:</p>
371
372  <ol>
373    <li>Merging in changes from a new stable release may be more difficult. If
374    the new stable release includes changes which were fed back by the
375    collaborator then these will already be present on the collaborators trunk.
376    If these changes were modified in any way or if they overlap with other
377    changes then this will result in a conflict which could be tricky to
378    resolve.</li>
379
380    <li>Any changes which need to be fed back by the collaborator need to be
381    made relative to a stable release. However, changes will have been prepared
382    relative to some version of the trunk. This means that a separate branch
383    will need to be taken (from the branch containing the pristine copy of the
384    main site) and a custom merge will be required in order to achieve the
385    desired result.</li>
386  </ol>
387
388  <h3 id="patchfiles">Exchanging Changesets using Patchfiles</h3>
389
390  <p>In some cases, an <em>FCM patch</em> may not be the best way of exchanging
391  changesets. For instance, when distributing code changes which have not yet
392  been finalised, you probably wouldn't want to send a patch containing all the
393  individual commits to the branch on which the change is being developed. What
394  you want is a summary of the changes in a single changeset. In this case you
395  will often be better to use a patchfile (which can be applied using the Unix
396  command <code>patch</code>). A patchfile is simply the output from an
397  <code>fcm diff</code> command. For example:</p>
398  <pre>
399fcm diff --branch fcm:myproj-br/dev/frdm/r2134_my_branch &gt; my_patchfile
400</pre>
401
402  <p>The patchfile must be applied to a working copy of the project which
403  corresponds to the same revision from which the patchfile was generated. The
404  option <code>-p0</code> must be used with the <code>patch</code> command. For
405  example:</p>
406  <pre>
407patch -p0 &lt; my_patchfile
408</pre>
409
410  <p>Patchfiles have the advantage that they are simple to generate and
411  exchange and that they can combine the changes from a number of changsets
412  into one. However, they have a number of limitations such as:</p>
413
414  <ul>
415    <li>Binary files are ignored.</li>
416
417    <li>Deleted directories are ignored.</li>
418
419    <li>Deleted files are left as empty files.</li>
420
421    <li>Copied files appear as new files.</li>
422
423    <li>A moved file is treated as a deleted file and a new file.</li>
424  </ul>
425
426  <p>Fortunately these limitations will not be an issue for the majority of
427  changes and, where they are a problem, there are various options such as
428  providing additional instructions with the patchfile, using an <em>FCM
429  patch</em>, or exchanging a modified project tree.</p>
430
431  <h2 id="further">Further Considerations</h2>
432
433  <p>The previous sections have only considered the version control aspects of
434  a collaboration. This section lists some other aspects of the collaboration
435  which will need to be considered.</p>
436
437  <ul>
438    <li>The FCM build system can be used regardless of what version control
439    system is used. This avoids effort being wasted trying to maintain
440    compatibility with an alternate build system. It also ensures that any code
441    changes prepared by the collaborator are compatible with the coding
442    standards which the FCM build system requires. Even if there are good
443    reasons for the collaborator not to use FCM for version control, it is
444    highly recommended that the FCM build system is used (assuming that is what
445    is used at the central repository).</li>
446
447    <li>Coding standards should be agreed by all collaborators.</li>
448
449    <li>Working practices should be agreed which should define, amongst other
450    things, what level of testing, review and documentation is expected to
451    accompany any proposed change.</li>
452
453    <li>All parties in the collaboration should note the advice given in the
454    <a href="../user_guide/code_management.html#svn_problems">FCM user
455    guide</a> to avoid renaming files or directories unless you can ensure that
456    no-one is working in parallel on the affected areas of the project.</li>
457
458    <li><acronym title="intellectual property rights">IPR</acronym>, copyright
459    and license issues should be agreed by all collaborators.</li>
460  </ul>
461
462  </div>
463  </div>
464  </div>
465
466  <hr/>
467  <div class="container-fluid text-center">
468    <div class="row"><div class="col-md-12">
469    <address><small>
470      Copyright &copy; 2006-2021 British Crown (Met Office) &amp; Contributors.
471      <a href="http://www.metoffice.gov.uk">Met Office</a>.
472      See <a href="../etc/fcm-terms-of-use.html">Terms of Use</a>.<br />
473      This document is released under the British <a href=
474      "http://www.nationalarchives.gov.uk/doc/open-government-licence/" rel=
475      "license">Open Government Licence</a>.<br />
476    </small></address>
477    </div></div>
478  </div>
479
480  <script type="text/javascript" src="../etc/jquery.min.js"></script>
481  <script type="text/javascript" src="../etc/bootstrap/js/bootstrap.min.js"></script>
482  <script type="text/javascript" src="../etc/fcm.js"></script>
483  <script type="text/javascript" src="../etc/fcm-version.js"></script>
484</body>
485</html>
Note: See TracBrowser for help on using the repository browser.