| 1 | <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> |
|---|
| 2 | |
|---|
| 3 | <html> |
|---|
| 4 | <head> |
|---|
| 5 | <title>External Distribution & Collaboration for FCM Projects</title> |
|---|
| 6 | <meta name="author" content="FCM development team"> |
|---|
| 7 | <meta name="descriptions" content= |
|---|
| 8 | "External Distribution & Collaboration"> |
|---|
| 9 | <meta name="keywords" content="FCM, distribution, collaboration"> |
|---|
| 10 | <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> |
|---|
| 11 | <link rel="stylesheet" type="text/css" href="style.css"> |
|---|
| 12 | </head> |
|---|
| 13 | |
|---|
| 14 | <body> |
|---|
| 15 | <p align="right"><img src="logo.png" alt="Met Office logo" width="85" height= |
|---|
| 16 | "85"></p> |
|---|
| 17 | |
|---|
| 18 | <h1>External Distribution & Collaboration for FCM Projects</h1> |
|---|
| 19 | |
|---|
| 20 | <p align="center">Last updated: 28 November 2006</a> |
|---|
| 21 | |
|---|
| 22 | <p align="center">Met Office<br> |
|---|
| 23 | FitzRoy Road, Exeter<br> |
|---|
| 24 | Devon, EX1 3PB<br> |
|---|
| 25 | United Kingdom</p> |
|---|
| 26 | |
|---|
| 27 | <p align="center">© Crown Copyright 2006</p> |
|---|
| 28 | |
|---|
| 29 | <p align="center">Questions regarding this document or permissions to quote |
|---|
| 30 | from it should be directed to the <a href= |
|---|
| 31 | "mailto:iprmanager@metoffice.gov.uk">IPR Manager</a>.</p> |
|---|
| 32 | |
|---|
| 33 | <script type="text/javascript"> |
|---|
| 34 | <!-- |
|---|
| 35 | var out = 'For printing, please use the ' |
|---|
| 36 | out += '<a href="fcm-collaboration.pdf">PDF<\/a> version of the document.' |
|---|
| 37 | document.write ('<p align="center">') |
|---|
| 38 | document.write (out) |
|---|
| 39 | document.write ('<\/p>') |
|---|
| 40 | //--> |
|---|
| 41 | </script> |
|---|
| 42 | |
|---|
| 43 | <h2><a name="introduction" id="introduction">Introduction</a></h2> |
|---|
| 44 | |
|---|
| 45 | <p>This document describes how projects configured under FCM can be |
|---|
| 46 | distributed externally. Particular attention is given to collaborative |
|---|
| 47 | distributions - where the external user regularly returns code for |
|---|
| 48 | consolidation into the central repositories which hold the master copies of the |
|---|
| 49 | code.</p> |
|---|
| 50 | |
|---|
| 51 | <p><b>Note:</b> This document assumes that the repositories are inaccessible to |
|---|
| 52 | the external user, due to issues of security and practicality.</p> |
|---|
| 53 | |
|---|
| 54 | <h2><a name="distribution" id="distribution">Creating a Distribution</a></h2> |
|---|
| 55 | |
|---|
| 56 | <p>A system configured under FCM can be distributed by packaging a known |
|---|
| 57 | revision (usually corresponding to a stable release) into an archive (e.g. a |
|---|
| 58 | tarball) of directories and files. Various issues need to be considered:</p> |
|---|
| 59 | |
|---|
| 60 | <ul> |
|---|
| 61 | <li>A distribution may contain a variety of different files including |
|---|
| 62 | source code, scripts, benchmark and validation tests, documentation, |
|---|
| 63 | etc.</li> |
|---|
| 64 | |
|---|
| 65 | <li>A system may consist of several different <em>projects</em> which |
|---|
| 66 | should be put into separate directories in the distribution. Please refer |
|---|
| 67 | to the section <a href= |
|---|
| 68 | "../user_guide/system_admin.html#svn_design">Repository design</a> in the |
|---|
| 69 | FCM user guide for an explanation of what is meant by a project in this |
|---|
| 70 | context.</li> |
|---|
| 71 | |
|---|
| 72 | <li>Some files in a project may not be included in the distribution. This |
|---|
| 73 | may be because they are of no interest to external users or because of |
|---|
| 74 | license restrictions. Such files will need to be filtered out when creating |
|---|
| 75 | the distribution.</li> |
|---|
| 76 | |
|---|
| 77 | <li>The distribution may also contain some files which are not maintained |
|---|
| 78 | under FCM version control (test results for instance).</li> |
|---|
| 79 | |
|---|
| 80 | <li>Some systems share code with other systems. |
|---|
| 81 | |
|---|
| 82 | <ul> |
|---|
| 83 | <li>If a distribution is intended to be used standalone then the |
|---|
| 84 | necessary files from these other systems will need to be included. e.g. |
|---|
| 85 | The VAR system requires code from the OPS and GEN systems.</li> |
|---|
| 86 | |
|---|
| 87 | <li>If the distribution is part of a wider collaboration then it is |
|---|
| 88 | likely that the files from the other systems will be distributed |
|---|
| 89 | separately. It is best if stable releases of the various systems can be |
|---|
| 90 | synchronised so that, for example, a VAR stable release uses code from |
|---|
| 91 | an OPS stable release which both use code from the same GEN release.</li> |
|---|
| 92 | </ul> |
|---|
| 93 | </li> |
|---|
| 94 | |
|---|
| 95 | <li>Release notes should be prepared to accompany a distribution which |
|---|
| 96 | explain, among other things, how the distribution is structured.</li> |
|---|
| 97 | |
|---|
| 98 | <li>The distribution should contain a file which identifies the repository |
|---|
| 99 | revision(s) contained in the distribution.</li> |
|---|
| 100 | |
|---|
| 101 | <li>System managers will probably wish to maintain a script which automates |
|---|
| 102 | the generation of these distributions.</li> |
|---|
| 103 | </ul> |
|---|
| 104 | |
|---|
| 105 | <h2><a name="feedback" id="feedback">Feeding Back Changes</a></h2> |
|---|
| 106 | |
|---|
| 107 | <p>Although we would encourage all collaborators to make use of the FCM |
|---|
| 108 | system for version control, we recognise that they may already have their own |
|---|
| 109 | preferred systems in place. There is no particular problem with this. The |
|---|
| 110 | main requirement is that any proposed changes are provided as a modification |
|---|
| 111 | relative to the provided distribution. The changeset could be provided in |
|---|
| 112 | the form of a modified project tree or as a patchfile (refer to the man page |
|---|
| 113 | for the Unix command <em>patch</em> for details). If the change involves any |
|---|
| 114 | renaming or removal of files or directories then special instructions should be |
|---|
| 115 | provided plus a script to perform the changes.</p> |
|---|
| 116 | |
|---|
| 117 | <p>At the central repository, the changeset should be applied to a branch |
|---|
| 118 | created from the repository revision which formed the basis of the changeset |
|---|
| 119 | (possibly making use of the Subversion utility |
|---|
| 120 | <a href="http://svnbook.red-bean.com/en/1.2/svn.advanced.vendorbr.html#svn.advanced.vendorbr.svn_load_dirs"> |
|---|
| 121 | svn_load_dirs.pl</a>). Note that extra care is needed with changesets |
|---|
| 122 | provided as modified project trees if there are any files in the project which |
|---|
| 123 | are excluded from the distribution. Once imported, the changeset should then |
|---|
| 124 | undergo any necessary testing or review before being merged into the trunk.</p> |
|---|
| 125 | |
|---|
| 126 | <h2><a name="usingfcm" id="usingfcm">Collaborating Using FCM for Version |
|---|
| 127 | Control</a></h2> |
|---|
| 128 | |
|---|
| 129 | <p>There are a number of advantages if the FCM system is used for version |
|---|
| 130 | control by the collaborator. In particular it means that:</p> |
|---|
| 131 | |
|---|
| 132 | <ul> |
|---|
| 133 | <li>Collaborators will be able to see all of the individual changesets |
|---|
| 134 | which went in to a new release rather than only being able to view each new |
|---|
| 135 | release as one big change.</li> |
|---|
| 136 | |
|---|
| 137 | <li>The process of sending a proposed change to the central repository can be |
|---|
| 138 | standardised through the use of an "FCM patch" (explained later).</li> |
|---|
| 139 | |
|---|
| 140 | <li>The FCM Extract system can be fully utilised.</li> |
|---|
| 141 | |
|---|
| 142 | <li>Common tools will help to ease communication. We will all use technical |
|---|
| 143 | terms to mean the same thing.</li> |
|---|
| 144 | </ul> |
|---|
| 145 | |
|---|
| 146 | <p>This section explains the recommended way of using FCM in a |
|---|
| 147 | collaboration.</p> |
|---|
| 148 | |
|---|
| 149 | <h3><a name="initsvn" id="initsvn">Initialising the Subversion |
|---|
| 150 | Repositories</a></h3> |
|---|
| 151 | |
|---|
| 152 | <p>The collaborator needs to set up a repository and import each of the |
|---|
| 153 | projects. Please see the section <a href= |
|---|
| 154 | "../user_guide/system_admin.html#svn_create">Creating a repository</a> in the |
|---|
| 155 | FCM user guide for advice. Collaborators may wish to use separate repositories |
|---|
| 156 | and Trac systems for each project or they may prefer to use a single |
|---|
| 157 | repository for all projects and use a single Trac system. Either option |
|---|
| 158 | should be fine so long as the same set of projects is retained.</p> |
|---|
| 159 | |
|---|
| 160 | <p>After completing the initial import, the collaborator should have the |
|---|
| 161 | required set of projects available in Subversion where the initial version of |
|---|
| 162 | the trunk of each project corresponds with the initial stable release provided |
|---|
| 163 | in the distribution.</p> |
|---|
| 164 | |
|---|
| 165 | <h3><a name="prepchanges" id="prepchanges">Preparing Changes at the |
|---|
| 166 | Collaborator's Site</a></h3> |
|---|
| 167 | |
|---|
| 168 | <p>The recommended way of preparing changes is illustrated in Figure 1a:</p> |
|---|
| 169 | |
|---|
| 170 | <p class="image"><img src="working-as-collaborator.png" alt= |
|---|
| 171 | "Figure 1a: working at the collaborator's site"><br> |
|---|
| 172 | Figure 1a: working at the collaborator's site</p> |
|---|
| 173 | |
|---|
| 174 | <p>The collaborator will create a shared package branch from the latest |
|---|
| 175 | stable release on the trunk. This branch will contain all the changes that |
|---|
| 176 | will eventually be fed back to the central repository. Developers will also |
|---|
| 177 | create their own development branches. These may be branched from the latest |
|---|
| 178 | stable release on the trunk. Alternatively, if the change needs to build on |
|---|
| 179 | other changes then a branch can be created from the shared package branch. When |
|---|
| 180 | the changes are ready (i.e. tested, documented, reviewed, etc) then they are |
|---|
| 181 | merged into the shared package branch. The trunk is not used for the shared |
|---|
| 182 | changes as it is reserved for changes received from the central repository.</p> |
|---|
| 183 | |
|---|
| 184 | <p>Should it be required, a second shared package branch can be created from |
|---|
| 185 | the same point to contain any local modifications that will not be fed back |
|---|
| 186 | to the central repository. A configuration branch can then be used to combine |
|---|
| 187 | the local changes with those destined to be fed back. This is illustrated in |
|---|
| 188 | Figure 1b:</p> |
|---|
| 189 | |
|---|
| 190 | <p class="image"><img src="managing-local-changes.png" alt= |
|---|
| 191 | "Figure 1b: managing local changes"><br> |
|---|
| 192 | Figure 1b: managing local changes</p> |
|---|
| 193 | |
|---|
| 194 | <h3><a name="feedbackfcm" id="feedbackfcm">Feeding Back Changes Using |
|---|
| 195 | FCM</a></h3> |
|---|
| 196 | |
|---|
| 197 | <p>Eventually, a series of changesets will exist on the first package branch. |
|---|
| 198 | These changes will be fed back to the central repository via an "FCM patch". |
|---|
| 199 | This contains a series of differences associated with changesets in a given |
|---|
| 200 | branch of development, created by the <tt>fcm mkpatch</tt> command. For further |
|---|
| 201 | information about the command, please refer to its <a href= |
|---|
| 202 | "../user_guide/command_ref.html#fcm_svn_mkpatch">command reference</a> in the |
|---|
| 203 | FCM user guide.</p> |
|---|
| 204 | |
|---|
| 205 | <p>At the central repository, the changeset will be applied to a branch |
|---|
| 206 | created from the repository revision which formed the basis of the changeset. |
|---|
| 207 | This is illustrated in Figure 2:</p> |
|---|
| 208 | |
|---|
| 209 | <p class="image"><img src="feeding-back-patch.png" alt= |
|---|
| 210 | "Figure 2: feeding back changes"><br> |
|---|
| 211 | Figure 2: feeding back changes</p> |
|---|
| 212 | |
|---|
| 213 | <p>Patches will usually be exchanged in the form of a tarball. To apply the |
|---|
| 214 | patch it must first be extracted to a directory. In this directory there |
|---|
| 215 | should be a shell script called <tt>fcm-import-patch</tt>. A TARGET needs to |
|---|
| 216 | be specified when invoking the script. The TARGET must either be a URL or a |
|---|
| 217 | working copy of a valid project tree that can accept the import of the |
|---|
| 218 | patches. It is essential that this target matches the version of the project |
|---|
| 219 | from which the patch was created (usually this means a particular stable |
|---|
| 220 | release). The script contains a series of <tt>cp</tt> and <tt>svn</tt> |
|---|
| 221 | commands to import the changesets one by one. Note that the changesets are |
|---|
| 222 | committed automatically with no user interaction. It is worth ensuring that |
|---|
| 223 | an up to date backup of the repository is available in case of problems.</p> |
|---|
| 224 | |
|---|
| 225 | <h3><a name="changescentral" id="changescentral">Incorporating Changes into |
|---|
| 226 | the Trunk of the Central Repository</a></h3> |
|---|
| 227 | |
|---|
| 228 | <p>Once the changes have undergone any necessary testing or review they can |
|---|
| 229 | be merged into the trunk. There are three ways of approaching this:</p> |
|---|
| 230 | |
|---|
| 231 | <ol> |
|---|
| 232 | <li>As one changeset: all changes in the branch will be merged into the |
|---|
| 233 | trunk as a single changeset. This approach is the easiest and has the |
|---|
| 234 | advantage that any conflicts only need to be resolved once. However, the |
|---|
| 235 | drawback of this approach is that the logical changesets as fed back by the |
|---|
| 236 | collaborator will be combined into a large single changeset on the trunk, |
|---|
| 237 | which may not be the most desirable (although the logical changesets will |
|---|
| 238 | still be available to examine on the import branch). This is illustrated in |
|---|
| 239 | Figure 3a: |
|---|
| 240 | |
|---|
| 241 | <p class="image"><img src="merging-patch-one.png" alt= |
|---|
| 242 | "Figure 3a: merging a patch in a single changeset"><br> |
|---|
| 243 | Figure 3a: merging a patch in a single changeset</p> |
|---|
| 244 | </li> |
|---|
| 245 | |
|---|
| 246 | <li>As multiple changesets: each changeset in the branch will be merged |
|---|
| 247 | into the trunk in order. This can be quite complicated and time consuming, |
|---|
| 248 | especially if you have a large number of changesets and there are a lot of |
|---|
| 249 | clashes. The advantage is that each logical changeset will retain its |
|---|
| 250 | logical identity, which may be more desirable in the long run, when you |
|---|
| 251 | come to inspect the history. This is illustrated in Figure 3b: |
|---|
| 252 | |
|---|
| 253 | <p class="image"><img src="merging-patch-multi.png" alt= |
|---|
| 254 | "Figure 3b: merging a patch in multiple changesets"><br> |
|---|
| 255 | Figure 3b: merging a patch in multiple changesets</p> |
|---|
| 256 | </li> |
|---|
| 257 | |
|---|
| 258 | <li>As a mixture of the above: you may want to combine the above two |
|---|
| 259 | approaches when it makes sense to do so. For example, there may be a series |
|---|
| 260 | of small changesets that can be combined logically, or there may be a |
|---|
| 261 | changeset that fixes a bug introduced in the previous one. The bottom line |
|---|
| 262 | is that the project/system manager should use his/her own judgement in the |
|---|
| 263 | matter for what is best for the future of the project.</li> |
|---|
| 264 | </ol> |
|---|
| 265 | |
|---|
| 266 | <h3><a name="changescollab" id="changescollab">Incorporating Updates at the |
|---|
| 267 | Collaborator's Site</a></h3> |
|---|
| 268 | |
|---|
| 269 | <p>Once a new stable release is available it will be supplied in the form of |
|---|
| 270 | a distribution tarball as described earlier. However, collaborators will also |
|---|
| 271 | be supplied with an "FCM patch" (as described earlier) for each project |
|---|
| 272 | containing all the changes made since the previous stable release. Note that |
|---|
| 273 | this assumes that stable releases are prepared on the trunk and not in |
|---|
| 274 | branches.</p> |
|---|
| 275 | |
|---|
| 276 | <p>Each patch should be applied to the trunk of the collaborator's |
|---|
| 277 | repository. This means that the collaborator's trunk will always be mirroring |
|---|
| 278 | that of the central repository. This is illustrated in Figure 4:</p> |
|---|
| 279 | |
|---|
| 280 | <p class="image"><img src="mirroring-trunk.png" alt= |
|---|
| 281 | "Figure 4: mirroring the trunk at the collaborator's site"><br> |
|---|
| 282 | Figure 4: mirroring the trunk at the collaborator's site</p> |
|---|
| 283 | |
|---|
| 284 | <p>In order to be certain that the patch has worked correctly, we recommend |
|---|
| 285 | that a check is performed to ensure that the new stable release on the trunk |
|---|
| 286 | matches the files provided in the distribution.</p> |
|---|
| 287 | |
|---|
| 288 | <h3><a name="updatebranches" id="updatebranches">Updating Existing |
|---|
| 289 | Branches</a></h3> |
|---|
| 290 | |
|---|
| 291 | <p>Old branches that are still active at the collaborators site should be |
|---|
| 292 | updated to the latest stable release when it becomes available. Developers |
|---|
| 293 | should create a new branch from the latest stable release and then merge the |
|---|
| 294 | changes from the old branch to the new branch. The old branch should be |
|---|
| 295 | deleted once it is no longer required. This is illustrated in Figure 5a:</p> |
|---|
| 296 | |
|---|
| 297 | <p class="image"><img src="updating-branch.png" alt= |
|---|
| 298 | "Figure 5a: updating a branch to the latest stable release"><br> |
|---|
| 299 | Figure 5a: updating a branch to the latest stable release</p> |
|---|
| 300 | |
|---|
| 301 | <p>Note that the merge will be easiest if the old branch was created from the |
|---|
| 302 | previous stable release. If it was created from the shared package branch |
|---|
| 303 | then a custom merge will be required to achieve the desired result. This is |
|---|
| 304 | illustrated in Figure 5b:</p> |
|---|
| 305 | |
|---|
| 306 | <p class="image"><img src="updating-shared-branch.png" alt= |
|---|
| 307 | "Figure 5b: updating a branch of the shared package branch"><br> |
|---|
| 308 | Figure 5b: updating a branch of the shared package branch</p> |
|---|
| 309 | |
|---|
| 310 | <h3><a name="other" id="other">Other Scenarios</a></h3> |
|---|
| 311 | |
|---|
| 312 | <p>The previous sections have only considered how developments on the trunk of |
|---|
| 313 | a central repository can be shared with a single collaborator. However, the |
|---|
| 314 | same techniques can be applied to more complex situations.</p> |
|---|
| 315 | |
|---|
| 316 | <ul> |
|---|
| 317 | <li>If there are multiple external collaborators each working with their own |
|---|
| 318 | repository then hopefully it is clear that this does not alter things in any |
|---|
| 319 | way. Inevitably there will be an increased workload on the maintainers of the |
|---|
| 320 | central repository. There will also be an increased need for coordination of |
|---|
| 321 | planned code changes. However, the method of code exchange is unaltered.</li> |
|---|
| 322 | |
|---|
| 323 | <li>Sometimes there may be the need to collaborate on development of a branch |
|---|
| 324 | (i.e. to exchange code which is not yet ready to be incorporated onto the |
|---|
| 325 | trunk). The collaborator would maintain the trunk of their repository as |
|---|
| 326 | before, importing patches to keep their trunk alligned with the stable |
|---|
| 327 | releases from the central repository. In addition, they would receive an FCM |
|---|
| 328 | patch from the central repository representing the changes on the shared |
|---|
| 329 | branch relative to the stable release. The collaborator should create a |
|---|
| 330 | branch from the stable release and the patch should then be imported onto |
|---|
| 331 | this branch. They should then create a branch from this branch on which to |
|---|
| 332 | prepare their changes. When ready the changes would be returned in the form |
|---|
| 333 | of an FCM patch, and so on. Hopefully it can be seen that the same process |
|---|
| 334 | can be applied to this shared branch as we have previously described for |
|---|
| 335 | trunk developments.</li> |
|---|
| 336 | </ul> |
|---|
| 337 | |
|---|
| 338 | <h2><a name="further" id="other">Further Considerations</a></h2> |
|---|
| 339 | |
|---|
| 340 | <p>The previous sections have only considered the version control aspects of a |
|---|
| 341 | collaboration. This section lists some other aspects of the collaboration which |
|---|
| 342 | will need to be considered.</p> |
|---|
| 343 | |
|---|
| 344 | <ul> |
|---|
| 345 | <li>The FCM build system can be used regardless of what version control |
|---|
| 346 | system is used. This avoids effort being wasted trying to maintain |
|---|
| 347 | compatibility with an alternate build system. It also ensures that any code |
|---|
| 348 | changes prepared by the collaborator are compatible with the coding standards |
|---|
| 349 | which the FCM build system requires. Even if there are good reasons for the |
|---|
| 350 | collaborator not to use FCM for version control, it is highly recommended |
|---|
| 351 | that the FCM build system is used (assuming that is what is used at the |
|---|
| 352 | central repository).</li> |
|---|
| 353 | |
|---|
| 354 | <li>Coding standards should be agreed by all collaborators.</li> |
|---|
| 355 | |
|---|
| 356 | <li>Working practises should be agreed which should define, amongst other |
|---|
| 357 | things, what level of testing, review and documentation is expected to |
|---|
| 358 | accompany any proposed change.</li> |
|---|
| 359 | |
|---|
| 360 | <li>All parties in the collaboration should note the advice given in the |
|---|
| 361 | <a href="../user_guide/code_management.html#svn_problems">FCM user guide</a> |
|---|
| 362 | to avoid renaming files or directories unless you can ensure that no-one is |
|---|
| 363 | working in parallel on the affected areas of the project.</li> |
|---|
| 364 | |
|---|
| 365 | <li>IPR, copyright and license issues should be agreed by all collaborators.</li> |
|---|
| 366 | </ul> |
|---|
| 367 | |
|---|
| 368 | </body> |
|---|
| 369 | </html> |
|---|
