source: LMDZ6/branches/IPSLCM6.0.13/tools/fcm/doc/collaboration/index.html @ 5429

Last change on this file since 5429 was 1578, checked in by jghattas, 13 years ago
  • Add fcm in LMDZ5/tools directory

It is no longer needed to have fcm in your environement PATH variable.
Now makelmdz_fcm takes by default this fcm. It is still possible to use
another fcm, using -fcm_path argument in makelmdz_fcm.
File size: 17.8 KB
Line 
1<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
2
3<html>
4<head>
5  <title>External Distribution &amp; Collaboration for FCM Projects</title>
6  <meta name="author" content="FCM development team">
7  <meta name="descriptions" content=
8  "External Distribution &amp; Collaboration">
9  <meta name="keywords" content="FCM, distribution, collaboration">
10  <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
11  <link rel="stylesheet" type="text/css" href="style.css">
12</head>
13
14<body>
15  <p align="right"><img src="logo.png" alt="Met Office logo" width="85" height=
16  "85"></p>
17
18  <h1>External Distribution &amp; Collaboration for FCM Projects</h1>
19
20  <p align="center">Last updated: 28 November 2006</a>
21
22  <p align="center">Met Office<br>
23  FitzRoy Road, Exeter<br>
24  Devon, EX1 3PB<br>
25  United Kingdom</p>
26
27  <p align="center">&copy; Crown Copyright 2006</p>
28
29  <p align="center">Questions regarding this document or permissions to quote
30  from it should be directed to the <a href=
31  "mailto:iprmanager@metoffice.gov.uk">IPR Manager</a>.</p>
32 
33  <script type="text/javascript">
34  <!--
35  var out = 'For printing, please use the '
36  out    += '<a href="fcm-collaboration.pdf">PDF<\/a> version of the document.'
37  document.write ('<p align="center">')
38  document.write (out)
39  document.write ('<\/p>')
40  //-->
41  </script>
42
43  <h2><a name="introduction" id="introduction">Introduction</a></h2>
44
45  <p>This document describes how projects configured under FCM can be
46  distributed externally. Particular attention is given to collaborative
47  distributions - where the external user regularly returns code for
48  consolidation into the central repositories which hold the master copies of the
49  code.</p>
50
51  <p><b>Note:</b> This document assumes that the repositories are inaccessible to
52  the external user, due to issues of security and practicality.</p>
53
54  <h2><a name="distribution" id="distribution">Creating a Distribution</a></h2>
55
56  <p>A system configured under FCM can be distributed by packaging a known
57  revision (usually corresponding to a stable release) into an archive (e.g. a
58  tarball) of directories and files. Various issues need to be considered:</p>
59
60  <ul>
61    <li>A distribution may contain a variety of different files including
62    source code, scripts, benchmark and validation tests, documentation,
63    etc.</li>
64
65    <li>A system may consist of several different <em>projects</em> which
66    should be put into separate directories in the distribution. Please refer
67    to the section <a href=
68    "../user_guide/system_admin.html#svn_design">Repository design</a> in the
69    FCM user guide for an explanation of what is meant by a project in this
70    context.</li>
71
72    <li>Some files in a project may not be included in the distribution. This
73    may be because they are of no interest to external users or because of
74    license restrictions. Such files will need to be filtered out when creating
75    the distribution.</li>
76
77    <li>The distribution may also contain some files which are not maintained
78    under FCM version control (test results for instance).</li>
79
80    <li>Some systems share code with other systems.
81
82      <ul>
83        <li>If a distribution is intended to be used standalone then the
84        necessary files from these other systems will need to be included. e.g.
85        The VAR system requires code from the OPS and GEN systems.</li>
86
87        <li>If the distribution is part of a wider collaboration then it is
88        likely that the files from the other systems will be distributed
89        separately. It is best if stable releases of the various systems can be
90        synchronised so that, for example, a VAR stable release uses code from
91        an OPS stable release which both use code from the same GEN release.</li>
92      </ul>
93    </li>
94
95    <li>Release notes should be prepared to accompany a distribution which
96    explain, among other things, how the distribution is structured.</li>
97
98    <li>The distribution should contain a file which identifies the repository
99    revision(s) contained in the distribution.</li>
100
101    <li>System managers will probably wish to maintain a script which automates
102    the generation of these distributions.</li>
103  </ul>
104
105  <h2><a name="feedback" id="feedback">Feeding Back Changes</a></h2>
106
107  <p>Although we would encourage all collaborators to make use of the FCM
108  system for version control, we recognise that they may already have their own
109  preferred systems in place. There is no particular problem with this. The
110  main requirement is that any proposed changes are provided as a modification
111  relative to the provided distribution. The changeset could be provided in
112  the form of a modified project tree or as a patchfile (refer to the man page
113  for the Unix command <em>patch</em> for details). If the change involves any
114  renaming or removal of files or directories then special instructions should be
115  provided plus a script to perform the changes.</p>
116
117  <p>At the central repository, the changeset should be applied to a branch
118  created from the repository revision which formed the basis of the changeset
119  (possibly making use of the Subversion utility
120  <a href="http://svnbook.red-bean.com/en/1.2/svn.advanced.vendorbr.html#svn.advanced.vendorbr.svn_load_dirs">
121  svn_load_dirs.pl</a>). Note that extra care is needed with changesets
122  provided as modified project trees if there are any files in the project which
123  are excluded from the distribution. Once imported, the changeset should then
124  undergo any necessary testing or review before being merged into the trunk.</p>
125
126  <h2><a name="usingfcm" id="usingfcm">Collaborating Using FCM for Version
127  Control</a></h2>
128
129  <p>There are a number of advantages if the FCM system is used for version
130  control by the collaborator. In particular it means that:</p>
131
132  <ul>
133    <li>Collaborators will be able to see all of the individual changesets
134    which went in to a new release rather than only being able to view each new
135    release as one big change.</li>
136
137    <li>The process of sending a proposed change to the central repository can be
138    standardised through the use of an "FCM patch" (explained later).</li>
139
140    <li>The FCM Extract system can be fully utilised.</li>
141
142    <li>Common tools will help to ease communication. We will all use technical
143    terms to mean the same thing.</li>
144  </ul>
145
146  <p>This section explains the recommended way of using FCM in a
147  collaboration.</p>
148
149  <h3><a name="initsvn" id="initsvn">Initialising the Subversion
150  Repositories</a></h3>
151
152  <p>The collaborator needs to set up a repository and import each of the
153  projects. Please see the section <a href=
154  "../user_guide/system_admin.html#svn_create">Creating a repository</a> in the
155  FCM user guide for advice. Collaborators may wish to use separate repositories
156  and Trac systems for each project or they may prefer to use a single
157  repository for all projects and use a single Trac system. Either option
158  should be fine so long as the same set of projects is retained.</p>
159
160  <p>After completing the initial import, the collaborator should have the
161  required set of projects available in Subversion where the initial version of
162  the trunk of each project corresponds with the initial stable release provided
163  in the distribution.</p>
164
165  <h3><a name="prepchanges" id="prepchanges">Preparing Changes at the
166  Collaborator's Site</a></h3>
167
168  <p>The recommended way of preparing changes is illustrated in Figure 1a:</p>
169
170  <p class="image"><img src="working-as-collaborator.png" alt=
171  "Figure 1a: working at the collaborator's site"><br>
172  Figure 1a: working at the collaborator's site</p>
173
174  <p>The collaborator will create a shared package branch from the latest
175  stable release on the trunk. This branch will contain all the changes that
176  will eventually be fed back to the central repository. Developers will also
177  create their own development branches. These may be branched from the latest
178  stable release on the trunk. Alternatively, if the change needs to build on
179  other changes then a branch can be created from the shared package branch. When
180  the changes are ready (i.e. tested, documented, reviewed, etc) then they are
181  merged into the shared package branch. The trunk is not used for the shared
182  changes as it is reserved for changes received from the central repository.</p>
183
184  <p>Should it be required, a second shared package branch can be created from
185  the same point to contain any local modifications that will not be fed back
186  to the central repository. A configuration branch can then be used to combine
187  the local changes with those destined to be fed back. This is illustrated in
188  Figure 1b:</p>
189
190  <p class="image"><img src="managing-local-changes.png" alt=
191  "Figure 1b: managing local changes"><br>
192  Figure 1b: managing local changes</p>
193
194  <h3><a name="feedbackfcm" id="feedbackfcm">Feeding Back Changes Using
195  FCM</a></h3>
196
197  <p>Eventually, a series of changesets will exist on the first package branch.
198  These changes will be fed back to the central repository via an "FCM patch".
199  This contains a series of differences associated with changesets in a given
200  branch of development, created by the <tt>fcm mkpatch</tt> command. For further
201  information about the command, please refer to its <a href=
202  "../user_guide/command_ref.html#fcm_svn_mkpatch">command reference</a> in the
203  FCM user guide.</p>
204
205  <p>At the central repository, the changeset will be applied to a branch
206  created from the repository revision which formed the basis of the changeset.
207  This is illustrated in Figure 2:</p>
208
209  <p class="image"><img src="feeding-back-patch.png" alt=
210  "Figure 2: feeding back changes"><br>
211  Figure 2: feeding back changes</p>
212
213  <p>Patches will usually be exchanged in the form of a tarball. To apply the
214  patch it must first be extracted to a directory. In this directory there
215  should be a shell script called <tt>fcm-import-patch</tt>. A TARGET needs to
216  be specified when invoking the script. The TARGET must either be a URL or a
217  working copy of a valid project tree that can accept the import of the
218  patches. It is essential that this target matches the version of the project
219  from which the patch was created (usually this means a particular stable
220  release). The script contains a series of <tt>cp</tt> and <tt>svn</tt>
221  commands to import the changesets one by one. Note that the changesets are
222  committed automatically with no user interaction. It is worth ensuring that
223  an up to date backup of the repository is available in case of problems.</p>
224
225  <h3><a name="changescentral" id="changescentral">Incorporating Changes into
226  the Trunk of the Central Repository</a></h3>
227
228  <p>Once the changes have undergone any necessary testing or review they can
229  be merged into the trunk. There are three ways of approaching this:</p>
230
231  <ol>
232    <li>As one changeset: all changes in the branch will be merged into the
233    trunk as a single changeset. This approach is the easiest and has the
234    advantage that any conflicts only need to be resolved once. However, the
235    drawback of this approach is that the logical changesets as fed back by the
236    collaborator will be combined into a large single changeset on the trunk,
237    which may not be the most desirable (although the logical changesets will
238    still be available to examine on the import branch). This is illustrated in
239    Figure 3a:
240
241      <p class="image"><img src="merging-patch-one.png" alt=
242      "Figure 3a: merging a patch in a single changeset"><br>
243      Figure 3a: merging a patch in a single changeset</p>
244    </li>
245
246    <li>As multiple changesets: each changeset in the branch will be merged
247    into the trunk in order. This can be quite complicated and time consuming,
248    especially if you have a large number of changesets and there are a lot of
249    clashes. The advantage is that each logical changeset will retain its
250    logical identity, which may be more desirable in the long run, when you
251    come to inspect the history. This is illustrated in Figure 3b:
252
253      <p class="image"><img src="merging-patch-multi.png" alt=
254      "Figure 3b: merging a patch in multiple changesets"><br>
255      Figure 3b: merging a patch in multiple changesets</p>
256    </li>
257
258    <li>As a mixture of the above: you may want to combine the above two
259    approaches when it makes sense to do so. For example, there may be a series
260    of small changesets that can be combined logically, or there may be a
261    changeset that fixes a bug introduced in the previous one. The bottom line
262    is that the project/system manager should use his/her own judgement in the
263    matter for what is best for the future of the project.</li>
264  </ol>
265
266  <h3><a name="changescollab" id="changescollab">Incorporating Updates at the
267  Collaborator's Site</a></h3>
268
269  <p>Once a new stable release is available it will be supplied in the form of
270  a distribution tarball as described earlier. However, collaborators will also
271  be supplied with an "FCM patch" (as described earlier) for each project
272  containing all the changes made since the previous stable release. Note that
273  this assumes that stable releases are prepared on the trunk and not in
274  branches.</p>
275
276  <p>Each patch should be applied to the trunk of the collaborator's
277  repository. This means that the collaborator's trunk will always be mirroring
278  that of the central repository. This is illustrated in Figure&nbsp;4:</p>
279
280  <p class="image"><img src="mirroring-trunk.png" alt=
281  "Figure 4: mirroring the trunk at the collaborator's site"><br>
282  Figure 4: mirroring the trunk at the collaborator's site</p>
283
284  <p>In order to be certain that the patch has worked correctly, we recommend
285  that a check is performed to ensure that the new stable release on the trunk
286  matches the files provided in the distribution.</p>
287
288  <h3><a name="updatebranches" id="updatebranches">Updating Existing
289  Branches</a></h3>
290
291  <p>Old branches that are still active at the collaborators site should be
292  updated to the latest stable release when it becomes available. Developers
293  should create a new branch from the latest stable release and then merge the
294  changes from the old branch to the new branch. The old branch should be
295  deleted once it is no longer required. This is illustrated in Figure 5a:</p>
296
297  <p class="image"><img src="updating-branch.png" alt=
298  "Figure 5a: updating a branch to the latest stable release"><br>
299  Figure 5a: updating a branch to the latest stable release</p>
300
301  <p>Note that the merge will be easiest if the old branch was created from the
302  previous stable release. If it was created from the shared package branch
303  then a custom merge will be required to achieve the desired result. This is
304  illustrated in Figure 5b:</p>
305
306  <p class="image"><img src="updating-shared-branch.png" alt=
307  "Figure 5b: updating a branch of the shared package branch"><br>
308  Figure 5b: updating a branch of the shared package branch</p>
309
310  <h3><a name="other" id="other">Other Scenarios</a></h3>
311
312  <p>The previous sections have only considered how developments on the trunk of
313  a central repository can be shared with a single collaborator. However, the
314  same techniques can be applied to more complex situations.</p>
315
316  <ul>
317    <li>If there are multiple external collaborators each working with their own
318    repository then hopefully it is clear that this does not alter things in any
319    way. Inevitably there will be an increased workload on the maintainers of the
320    central repository. There will also be an increased need for coordination of
321    planned code changes. However, the method of code exchange is unaltered.</li>
322
323    <li>Sometimes there may be the need to collaborate on development of a branch
324    (i.e. to exchange code which is not yet ready to be incorporated onto the
325    trunk). The collaborator would maintain the trunk of their repository as
326    before, importing patches to keep their trunk alligned with the stable
327    releases from the central repository. In addition, they would receive an FCM
328    patch from the central repository representing the changes on the shared
329    branch relative to the stable release. The collaborator should create a
330    branch from the stable release and the patch should then be imported onto
331    this branch. They should then create a branch from this branch on which to
332    prepare their changes. When ready the changes would be returned in the form
333    of an FCM patch, and so on. Hopefully it can be seen that the same process
334    can be applied to this shared branch as we have previously described for
335    trunk developments.</li>
336  </ul>
337
338  <h2><a name="further" id="other">Further Considerations</a></h2>
339
340  <p>The previous sections have only considered the version control aspects of a
341  collaboration. This section lists some other aspects of the collaboration which
342  will need to be considered.</p>
343
344  <ul>
345    <li>The FCM build system can be used regardless of what version control
346    system is used. This avoids effort being wasted trying to maintain
347    compatibility with an alternate build system. It also ensures that any code
348    changes prepared by the collaborator are compatible with the coding standards
349    which the FCM build system requires. Even if there are good reasons for the
350    collaborator not to use FCM for version control, it is highly recommended
351    that the FCM build system is used (assuming that is what is used at the
352    central repository).</li>
353
354    <li>Coding standards should be agreed by all collaborators.</li>
355
356    <li>Working practises should be agreed which should define, amongst other
357    things, what level of testing, review and documentation is expected to
358    accompany any proposed change.</li>
359
360    <li>All parties in the collaboration should note the advice given in the
361    <a href="../user_guide/code_management.html#svn_problems">FCM user guide</a>
362    to avoid renaming files or directories unless you can ensure that no-one is
363    working in parallel on the affected areas of the project.</li>
364
365    <li>IPR, copyright and license issues should be agreed by all collaborators.</li>
366  </ul>
367
368</body>
369</html>
Note: See TracBrowser for help on using the repository browser.