[5129] | 1 | <!DOCTYPE html> |
---|
| 2 | <html> |
---|
| 3 | <head> |
---|
| 4 | <title>FCM: User Guide: Annex: The FCM 1 Extract System</title> |
---|
| 5 | <meta name="author" content="FCM team" /> |
---|
| 6 | <meta name="viewport" content="width=device-width, initial-scale=1.0" /> |
---|
| 7 | <link rel="icon" href="../etc/fcm-icon.png" type="image/png" /> |
---|
| 8 | <link rel="shortcut icon" href="../etc/fcm-icon.png" type="image/png" /> |
---|
| 9 | <link href="../etc/bootstrap/css/bootstrap.min.css" rel="stylesheet" media="screen" /> |
---|
| 10 | <link href="../etc/fcm.css" rel="stylesheet" media="screen" /> |
---|
| 11 | </head> |
---|
| 12 | <body> |
---|
| 13 | <div class="navbar navbar-inverse"> |
---|
| 14 | <div class="container-fluid"> |
---|
| 15 | <div class="navbar-header"> |
---|
| 16 | <a class="navbar-brand" href=".."><span class="fcm-version">FCM</span></a> |
---|
| 17 | </div> |
---|
| 18 | <div class="collapse navbar-collapse"> |
---|
| 19 | <ul class="nav navbar-nav"> |
---|
| 20 | <li><a href="../installation/">Installation</a></li> |
---|
| 21 | |
---|
| 22 | <li><a class="active" href="#">User Guide</a></li> |
---|
| 23 | </ul> |
---|
| 24 | </div> |
---|
| 25 | </div> |
---|
| 26 | </div> |
---|
| 27 | |
---|
| 28 | <div class="page-header"> |
---|
| 29 | <div class="fcm-page-content pull-right well well-sm"></div> |
---|
| 30 | <h1>FCM: User Guide: Annex: The FCM 1 Extract System</h1> |
---|
| 31 | </div> |
---|
| 32 | |
---|
| 33 | <div class="container"> |
---|
| 34 | <div class="row"> |
---|
| 35 | <div class="col-md-12"> |
---|
| 36 | |
---|
| 37 | <h2 id="introduction">Introduction</h2> |
---|
| 38 | |
---|
| 39 | <p><em>The FCM 1 extract system is deprecated. The documentation for the |
---|
| 40 | current extract system can be found at <a href="make.html">FCM |
---|
| 41 | Make</a>.</em></p> |
---|
| 42 | |
---|
| 43 | <p>The extract system provides an interface between the revision control |
---|
| 44 | system (currently Subversion) and the build system. Where appropriate, it |
---|
| 45 | extracts code from the repository and other user-defined locations to a |
---|
| 46 | directory tree suitable for feeding into the build system. In this chapter, |
---|
| 47 | we shall use many examples to explain how to use the extract system. At the |
---|
| 48 | end of this chapter, you will be able to extract code from the local file |
---|
| 49 | system as well as from different branches of different repository URLs. You |
---|
| 50 | will also learn how to mirror code to an alternate destination. Finally, you |
---|
| 51 | will be given an introduction on how to specify configurations for the build |
---|
| 52 | system via the extract configuration file. (For further information on the |
---|
| 53 | build system, please see the next chapter <a href="build.html">The Build |
---|
| 54 | System</a>.) The last section of the chapter tells you what you can do in the |
---|
| 55 | case when Subversion is not available.</p> |
---|
| 56 | |
---|
| 57 | <h2 id="command">The Extract Command</h2> |
---|
| 58 | |
---|
| 59 | <p>To invoke the extract system, simply issue the command:</p> |
---|
| 60 | <pre> |
---|
| 61 | fcm extract |
---|
| 62 | </pre> |
---|
| 63 | |
---|
| 64 | <p>By default, the extract system searches for an extract configuration file |
---|
| 65 | <samp>ext.cfg</samp> in <samp>$PWD</samp> and then <samp>$PWD/cfg</samp>. If |
---|
| 66 | an extract configuration file is not found in these directories, the command |
---|
| 67 | fails with an error. If an extract configuration file is found, the system |
---|
| 68 | will use the configuration specified in the file to perform the current |
---|
| 69 | extract.</p> |
---|
| 70 | |
---|
| 71 | <p>If the destination of the extract does not exist, the system performs a |
---|
| 72 | new full extract to the destination. If a previous extract already exists at |
---|
| 73 | the destination, the system performs an incremental extract, updating any |
---|
| 74 | modifications if necessary. If a full (fresh) extract is required for |
---|
| 75 | whatever reason, you can invoke the extract system using the <code>-f</code> |
---|
| 76 | option, (i.e. the command becomes <code>fcm extract -f</code>). If you simply |
---|
| 77 | want to remove all the items generated by a previous extract in the |
---|
| 78 | destination, you can invoke the extract system using the <code>--clean</code> |
---|
| 79 | option.</p> |
---|
| 80 | |
---|
| 81 | <p>For further information on the extract command, please see <a href= |
---|
| 82 | "command_ref.html#fcm-extract">FCM Command Reference > fcm extract</a>.</p> |
---|
| 83 | |
---|
| 84 | <h2 id="simple">Simple Usage</h2> |
---|
| 85 | |
---|
| 86 | <p>The extract configuration file is the main user interface of the extract |
---|
| 87 | system. It is a line based text file. For a complete set of extract |
---|
| 88 | configuration file declarations, please refer to the <a href= |
---|
| 89 | "annex_ext_cfg.html">Annex: Declarations in FCM extract configuration |
---|
| 90 | file</a>.</p> |
---|
| 91 | |
---|
| 92 | <h3 id="simple_local">Extract from a local path</h3> |
---|
| 93 | |
---|
| 94 | <p>A simple example of a basic extract configuration file is given below:</p> |
---|
| 95 | <pre id="example_1"> |
---|
| 96 | # Example 1 |
---|
| 97 | # ---------------------------------------------------------------------- |
---|
| 98 | cfg::type ext # line 1 |
---|
| 99 | cfg::version 1.0 # line 2 |
---|
| 100 | # line 3 |
---|
| 101 | dest $PWD # line 4 |
---|
| 102 | # line 5 |
---|
| 103 | repos::var::user $HOME/var # line 6 |
---|
| 104 | # line 7 |
---|
| 105 | expsrc::var::user code # line 8 |
---|
| 106 | </pre> |
---|
| 107 | |
---|
| 108 | <p>The above demonstrates how to use the extract system to extract code from |
---|
| 109 | a local user directory. Here is an explanation of what each line does:</p> |
---|
| 110 | |
---|
| 111 | <ul> |
---|
| 112 | <li><dfn>line 1</dfn>: the label <code>CFG::TYPE</code> declares the type |
---|
| 113 | of the configuration file. The value <samp>ext</samp> tells the system that |
---|
| 114 | it is an extract configuration file.</li> |
---|
| 115 | |
---|
| 116 | <li><dfn>line 2</dfn>: the label <code>CFG::VERSION</code> declares the |
---|
| 117 | version of the extract configuration file. The current default is |
---|
| 118 | <samp>1.0</samp>. Although it is not currently used, if we have to change |
---|
| 119 | the format of the configuration file at a later stage, we shall be able to |
---|
| 120 | use this number to determine whether we are reading a file with an older |
---|
| 121 | format or one with a newer format.</li> |
---|
| 122 | |
---|
| 123 | <li><dfn>line 3</dfn>: a blank line or a line beginning with a |
---|
| 124 | <code>#</code> is a comment, and is ignored by the interpreter.</li> |
---|
| 125 | |
---|
| 126 | <li><dfn>line 4</dfn>: the label <code>DEST</code> declares the destination |
---|
| 127 | root directory of this extract. The value <samp>$PWD</samp> expands to the |
---|
| 128 | current working directory.</li> |
---|
| 129 | |
---|
| 130 | <li><dfn>line 5</dfn>: comment line, ignored.</li> |
---|
| 131 | |
---|
| 132 | <li><dfn>line 6</dfn>: the label |
---|
| 133 | <code>REPOS::<pck>::<branch></code> declares the top level URL |
---|
| 134 | or path of a repository. The package name of the repository is given by |
---|
| 135 | <pck>. In our example, we choose <samp>var</samp> as the name of the |
---|
| 136 | package. (You can choose any name you like, however, it is usually sensible |
---|
| 137 | to use a package name that matches the name of the project or system you |
---|
| 138 | are working with.) The branch name in the repository is given by |
---|
| 139 | <branch>. (Again, you can choose any name you like, however, it is |
---|
| 140 | usually sensible to use a name such as <samp>base</samp>, <samp>user</samp> |
---|
| 141 | or something that matches your branch name.) In our example, the word |
---|
| 142 | <samp>user</samp> is normally used to denote a local user directory. Hence |
---|
| 143 | the statement declares that the repository path for the <samp>var</samp> |
---|
| 144 | package in the <samp>user</samp> branch can be found at |
---|
| 145 | <samp>$HOME/var</samp>.</li> |
---|
| 146 | |
---|
| 147 | <li><dfn>line 7</dfn>: comment line, ignored.</li> |
---|
| 148 | |
---|
| 149 | <li><dfn>line 8</dfn>: the label |
---|
| 150 | <code>EXPSRC::<pck>::<branch></code> declares an |
---|
| 151 | <em>expandable</em> source directory for the package <pck> in the |
---|
| 152 | branch <branch>. In our example, the package name is |
---|
| 153 | <samp>var</samp>, and the branch name is <samp>user</samp>. These match the |
---|
| 154 | package and the branch names of the repository declaration in line 6. It |
---|
| 155 | means that the source directory declaration is associated with the path |
---|
| 156 | <samp>$HOME/var</samp>. The value of the declaration <samp>code</samp> is |
---|
| 157 | therefore a sub-directory under <samp>$HOME/var</samp>. By declaring a |
---|
| 158 | source directory using an <code>EXPSRC</code> label, the system |
---|
| 159 | automatically searches for all sub-directories (recursively) under the |
---|
| 160 | declared source directory.</li> |
---|
| 161 | </ul> |
---|
| 162 | |
---|
| 163 | <p>Invoking the extract system using the above configuration file will |
---|
| 164 | extract all sub-directories under <samp>$HOME/var/code</samp> to |
---|
| 165 | <samp>$PWD/src/var/code</samp>. Note: the extract system ignores symbolic |
---|
| 166 | links and hidden files, (i.e. file names beginning with a <samp>.</samp>). It |
---|
| 167 | will write a build configuration file to <samp>$PWD/cfg/bld.cfg</samp>. The |
---|
| 168 | configuration used for this extract will be written to the configuration file |
---|
| 169 | at <samp>$PWD/cfg/ext.cfg</samp>.</p> |
---|
| 170 | |
---|
| 171 | <dl> |
---|
| 172 | <dt>Note - incremental extract</dt> |
---|
| 173 | |
---|
| 174 | <dd>Suppose you have already performed an extract using the above |
---|
| 175 | configuration file. At a later time, you have made some changes to some of |
---|
| 176 | the files in the source directory. Re-running the extract system on the |
---|
| 177 | same configuration will trigger an incremental extract. In an incremental |
---|
| 178 | extract, the system will update only those files that are modified. If the |
---|
| 179 | last modified time (or last commit revision) of a source file in the |
---|
| 180 | current extract differs from that in the previous extract, the system will |
---|
| 181 | attempt a content comparison. The system updates the destination only if |
---|
| 182 | the content and/or file access permission of the source differs from that |
---|
| 183 | of the destination.</dd> |
---|
| 184 | </dl> |
---|
| 185 | |
---|
| 186 | <h3 id="simple_url">Extract from a Subversion URL</h3> |
---|
| 187 | |
---|
| 188 | <p>The next example demonstrates how to extract from a Subversion repository |
---|
| 189 | URL:</p> |
---|
| 190 | <pre id="example_2"> |
---|
| 191 | # Example 2 |
---|
| 192 | # ---------------------------------------------------------------------- |
---|
| 193 | cfg::type ext # line 1 |
---|
| 194 | cfg::version 1.0 # line 2 |
---|
| 195 | # line 3 |
---|
| 196 | dest $PWD # line 4 |
---|
| 197 | # line 5 |
---|
| 198 | repos::var::base svn://server/var/trunk # line 6 |
---|
| 199 | revision::var::base 1234 # line 7 |
---|
| 200 | # line 8 |
---|
| 201 | expsrc::var::base code # line 9 |
---|
| 202 | </pre> |
---|
| 203 | |
---|
| 204 | <ul> |
---|
| 205 | <li><dfn>line 1-5</dfn>: same as <a href="#example_1">example 1</a>.</li> |
---|
| 206 | |
---|
| 207 | <li><dfn>line 6</dfn>: the line declares the repository location of the |
---|
| 208 | <samp>base</samp> branch of the <samp>var</samp> package to be the |
---|
| 209 | Subversion URL <samp>svn://server/var/trunk</samp>.</li> |
---|
| 210 | |
---|
| 211 | <li><dfn>line 7</dfn>: the label |
---|
| 212 | <code>REVISION::<pck>::<branch></code> declares the revision of |
---|
| 213 | the repository associated with the package <pck> in the branch |
---|
| 214 | <branch>. The current line tells the extract system to use revision |
---|
| 215 | 1234 of <samp>svn://server/var/trunk</samp>. It is worth noting that the |
---|
| 216 | declared revision must be a revision when the declared branch exists. The |
---|
| 217 | actual revision used is the last changed revision of the declared one. If |
---|
| 218 | the revision is not declared, the default is to use the last changed |
---|
| 219 | revision at the HEAD of the branch.</li> |
---|
| 220 | |
---|
| 221 | <li><dfn>line 8</dfn>: comment line, ignored.</li> |
---|
| 222 | |
---|
| 223 | <li><dfn>line 9</dfn>: the line declares an expandable source directory in |
---|
| 224 | the repository <samp>svn://server/var/trunk</samp>.</li> |
---|
| 225 | </ul> |
---|
| 226 | |
---|
| 227 | <p>Invoking the extract system using the above configuration file will |
---|
| 228 | extract all sub-directories under <samp>svn://server/var/trunk/code</samp> to |
---|
| 229 | <samp>$PWD/src/var/code</samp>. It will write a build configuration file to |
---|
| 230 | <samp>$PWD/cfg/bld.cfg</samp>. The configuration used for this extract will |
---|
| 231 | be written to the configuration file at <samp>$PWD/cfg/ext.cfg</samp>.</p> |
---|
| 232 | |
---|
| 233 | <dl> |
---|
| 234 | <dt>EXPSRC or SRC?</dt> |
---|
| 235 | |
---|
| 236 | <dd> |
---|
| 237 | <p>So far, we have only declared source directories using the |
---|
| 238 | <code>EXPSRC</code> statement, which stands for <em>expandable source |
---|
| 239 | directory</em>. A source directory declared using this statement will |
---|
| 240 | trigger the system to search recursively for any sub-directories under |
---|
| 241 | the declared one. Any sub-directories containing regular source files |
---|
| 242 | will be included in the extract. Symbolic links, hidden files and empty |
---|
| 243 | directories (or those containing only symbolic links and/or hidden files) |
---|
| 244 | are ignored.</p> |
---|
| 245 | |
---|
| 246 | <p>If you do not want the system to search for sub-directories underneath |
---|
| 247 | your declared source directory, you can declare your source directory |
---|
| 248 | using the <code>SRC</code> statement. The <code>SRC</code> statement is |
---|
| 249 | essentially the same as <code>EXPSRC</code> except that it does not |
---|
| 250 | trigger the automatic recursive search for source directories. In fact, |
---|
| 251 | the system implements the <code>EXPSRC</code> statement by expanding it |
---|
| 252 | into a list of <code>SRC</code> statements.</p> |
---|
| 253 | </dd> |
---|
| 254 | |
---|
| 255 | <dt>Package and sub-package</dt> |
---|
| 256 | |
---|
| 257 | <dd> |
---|
| 258 | <p>The second field of a repository, revision or source directory |
---|
| 259 | declaration label is the name of the container package. It is a name |
---|
| 260 | selected by the user to identify the system or project he/she is working |
---|
| 261 | on. (Therefore, it is often sensible to choose an identifier that matches |
---|
| 262 | the name of the project or system.) The package name provides a unique |
---|
| 263 | namespace for a file container. Source directories are automatically |
---|
| 264 | arranged into sub-packages, using the names of the sub-directories as the |
---|
| 265 | names of the sub-packages. For example, the declaration at line 9 in |
---|
| 266 | <a href="#example_2">example 2</a> will put the source directory in the |
---|
| 267 | <samp>var/code</samp> sub-package automatically.</p> |
---|
| 268 | |
---|
| 269 | <p>Note that, in additional to slash <code>/</code>, double colon |
---|
| 270 | <code>::</code> and double underscore <code>__</code> (internal only) |
---|
| 271 | also act as delimiters for package names. Please avoid using them for |
---|
| 272 | naming your files and directories.</p> |
---|
| 273 | |
---|
| 274 | <p>You can declare a sub-package name explicitly in your source directory |
---|
| 275 | statement. For example, the following two lines are equivalent:</p> |
---|
| 276 | <pre> |
---|
| 277 | src::var::base code/VarMod_Surface |
---|
| 278 | src::var/code/VarMod_Surface::base code/VarMod_Surface |
---|
| 279 | </pre> |
---|
| 280 | |
---|
| 281 | <p>Explicit sub-package declaration should not be used normally, as it |
---|
| 282 | requires a lot more typing (although there are some situations where it |
---|
| 283 | can be useful, e.g. if you need to re-define the package name).</p> |
---|
| 284 | |
---|
| 285 | <p>Currently, the extract system only supports non-space characters in |
---|
| 286 | the package name, as the space character is used as a delimiter between |
---|
| 287 | the declaration label and its value. If there are spaces in the path name |
---|
| 288 | to a file or directory, you should explicity re-define the package name |
---|
| 289 | of that path to a package name with no space using the above method. |
---|
| 290 | However, we recommend that only non-space characters are used for naming |
---|
| 291 | directories and files to make life simpler.</p> |
---|
| 292 | </dd> |
---|
| 293 | </dl> |
---|
| 294 | |
---|
| 295 | <dl> |
---|
| 296 | <dt>The expanded extract configuration file</dt> |
---|
| 297 | |
---|
| 298 | <dd> |
---|
| 299 | <p>At the end of a successful extract, the configuration used by the |
---|
| 300 | current extract is written in <samp>cfg/ext.cfg</samp> under the extract |
---|
| 301 | destination root. This file is an <em>expanded</em> version of the |
---|
| 302 | original, with changes in the following declarations:</p> |
---|
| 303 | |
---|
| 304 | <ul> |
---|
| 305 | <li>All revision keywords are converted into revision numbers.</li> |
---|
| 306 | |
---|
| 307 | <li>If a revision is not defined for a repository, it is set to the |
---|
| 308 | corresponding revision number of the HEAD revision.</li> |
---|
| 309 | |
---|
| 310 | <li>All URL keywords are converted into the full URLs.</li> |
---|
| 311 | |
---|
| 312 | <li>All <code>EXPSRC</code> declarations are expanded into |
---|
| 313 | <code>SRC</code> declarations.</li> |
---|
| 314 | |
---|
| 315 | <li>All other variables are expanded.</li> |
---|
| 316 | </ul> |
---|
| 317 | |
---|
| 318 | <p>With this file, it should be possible for a later extract to re-create |
---|
| 319 | the current configuration even if the contents of the repository have |
---|
| 320 | changed. (This applies only to code stored in the repository.)</p> |
---|
| 321 | </dd> |
---|
| 322 | </dl> |
---|
| 323 | |
---|
| 324 | <h3 id="simple_mirror">Mirror code to an alternate location</h3> |
---|
| 325 | |
---|
| 326 | <p>The next example demonstrates how to extract from a repository and mirror |
---|
| 327 | the code to an alternate location. It is essentially the same as <a href= |
---|
| 328 | "#example_2">example 2</a>, except that it has three new lines to describe |
---|
| 329 | how the system can mirror the extracted code to an alternate location.</p> |
---|
| 330 | <pre id="example_3"> |
---|
| 331 | # Example 3 |
---|
| 332 | # ---------------------------------------------------------------------- |
---|
| 333 | cfg::type ext |
---|
| 334 | cfg::version 1.0 |
---|
| 335 | |
---|
| 336 | dest $PWD |
---|
| 337 | |
---|
| 338 | rdest::machine tx01 # line 6 |
---|
| 339 | rdest::logname frva # line 7 |
---|
| 340 | rdest /scratch/frva/extract/example3 # line 8 |
---|
| 341 | |
---|
| 342 | repos::var::base svn://server/var/trunk |
---|
| 343 | revision::var::base 1234 |
---|
| 344 | |
---|
| 345 | expsrc::var::base code |
---|
| 346 | </pre> |
---|
| 347 | |
---|
| 348 | <p>Here is an explanation of what each line does:</p> |
---|
| 349 | |
---|
| 350 | <ul> |
---|
| 351 | <li><dfn>line 6</dfn>: <code>RDEST::MACHINE</code> declares the target |
---|
| 352 | machine to which the code will be mirrored. The example mirrors the code to |
---|
| 353 | the machine named <samp>tx01</samp>.</li> |
---|
| 354 | |
---|
| 355 | <li><dfn>line 7</dfn>: <code>RDEST::LOGNAME</code> declares the user name |
---|
| 356 | of the target machine, to which the user has login access. If this is not |
---|
| 357 | declared, the system uses the login name of the current user on the local |
---|
| 358 | machine.</li> |
---|
| 359 | |
---|
| 360 | <li><dfn>line 8</dfn>: <code>RDEST</code> declares the root directory of |
---|
| 361 | the alternate destination, where the mirror version of the extract will be |
---|
| 362 | sent.</li> |
---|
| 363 | </ul> |
---|
| 364 | |
---|
| 365 | <p>Invoking the extract system on the above configuration will trigger an |
---|
| 366 | extract similar to that given in <a href="#example_2">example 2</a>, but it |
---|
| 367 | will also attempt to mirror the contents at <samp>$PWD/src/var/code</samp> to |
---|
| 368 | <samp>/scratch/frva/extract/example3/src</samp> on the alternate destination. |
---|
| 369 | It will also mirror the expanded extract configuration file |
---|
| 370 | <samp>$PWD/cfg/ext.cfg</samp> to |
---|
| 371 | <samp>/scratch/frva/extract/example3/cfg/ext.cfg</samp> and |
---|
| 372 | <samp>$PWD/cfg/bld.cfg</samp> to |
---|
| 373 | <samp>/scratch/frva/extract/example3/cfg/bld.cfg</samp>. It is also worth |
---|
| 374 | noting that the content of the build configuration file will be slightly |
---|
| 375 | different, since it will include directory names appropriate for the |
---|
| 376 | alternate destination.</p> |
---|
| 377 | |
---|
| 378 | <dl> |
---|
| 379 | <dt>Note - mirroring command</dt> |
---|
| 380 | |
---|
| 381 | <dd> |
---|
| 382 | <p>The extract system currently supports <code>rdist</code> and |
---|
| 383 | <code>rsync</code> as its mirroring tool. The default is |
---|
| 384 | <code>rsync</code>. To use <code>rdist</code> instead of |
---|
| 385 | <code>rsync</code>, add the following line to your extract configuration |
---|
| 386 | file:</p> |
---|
| 387 | <pre> |
---|
| 388 | rdest::mirror_cmd rdist |
---|
| 389 | </pre> |
---|
| 390 | |
---|
| 391 | <p>If <code>rsync</code> is used to mirror an extract, the system needs to |
---|
| 392 | issue a separate remote shell command to create the container directory of |
---|
| 393 | the mirror destination. The default is to issue a shell command in the |
---|
| 394 | form <samp>ssh -n -oBatchMode=yes LOGNAME@MACHINE mkdir -p DEST</samp>. |
---|
| 395 | The following declarations can be used to modify the command:</p> |
---|
| 396 | <pre> |
---|
| 397 | # Examples using the default settings: |
---|
| 398 | rdest::rsh_mkdir_rsh ssh |
---|
| 399 | rdest::rsh_mkdir_rshflags -n -oBatchMode=yes |
---|
| 400 | rdest::rsh_mkdir_mkdir mkdir |
---|
| 401 | rdest::rsh_mkdir_mkdirflags -p |
---|
| 402 | </pre> |
---|
| 403 | |
---|
| 404 | <p>In addition, the default <code>rsync</code> shell command is |
---|
| 405 | <samp>rsync -a --exclude='.*' --delete-excluded --timeout=900 --rsh='ssh |
---|
| 406 | -oBatchMode=yes' SOURCE DEST</samp>. The following declarations can be |
---|
| 407 | used to modify the command:</p> |
---|
| 408 | <pre> |
---|
| 409 | # Examples using the default settings: |
---|
| 410 | rdest::rsync rsync |
---|
| 411 | rdest::rsyncflags -a --exclude='.*' --delete-excluded --timeout=900 \ |
---|
| 412 | --rsh='ssh -oBatchMode=yes' |
---|
| 413 | </pre> |
---|
| 414 | </dd> |
---|
| 415 | </dl> |
---|
| 416 | |
---|
| 417 | <h2 id="advanced">Advanced Usage</h2> |
---|
| 418 | |
---|
| 419 | <h3 id="advanced_multi">Extract from multiple repositories</h3> |
---|
| 420 | |
---|
| 421 | <p>So far, we have only extracted from a single location. The extract system |
---|
| 422 | is not much use if that is the only thing it can do. In fact, the extract |
---|
| 423 | system supports extract of multiple source directories from multiple branches |
---|
| 424 | in multiple repositories. The following configuration file is an example of |
---|
| 425 | how to extract from multiple repositories:</p> |
---|
| 426 | <pre id="example_4"> |
---|
| 427 | # Example 4 |
---|
| 428 | # ---------------------------------------------------------------------- |
---|
| 429 | cfg::type ext |
---|
| 430 | cfg::version 1.0 |
---|
| 431 | |
---|
| 432 | dest $PWD |
---|
| 433 | |
---|
| 434 | repos::var::base fcm:var_tr # line 6 |
---|
| 435 | repos::ops::base fcm:ops_tr # line 7 |
---|
| 436 | repos::gen::base fcm:gen_tr # line 8 |
---|
| 437 | |
---|
| 438 | revision::gen::base 2468 # line 10 |
---|
| 439 | |
---|
| 440 | expsrc::var::base src/code # line 12 |
---|
| 441 | expsrc::var::base src/scripts # line 13 |
---|
| 442 | expsrc::ops::base src/code # line 14 |
---|
| 443 | src::gen::base src/code/GenMod_Constants # line 15 |
---|
| 444 | src::gen::base src/code/GenMod_Control # line 16 |
---|
| 445 | src::gen::base src/code/GenMod_FortranIO # line 17 |
---|
| 446 | src::gen::base src/code/GenMod_GetEnv # line 18 |
---|
| 447 | src::gen::base src/code/GenMod_ModelIO # line 19 |
---|
| 448 | src::gen::base src/code/GenMod_ObsInfo # line 20 |
---|
| 449 | src::gen::base src/code/GenMod_Platform # line 21 |
---|
| 450 | src::gen::base src/code/GenMod_Reporting # line 22 |
---|
| 451 | src::gen::base src/code/GenMod_Trace # line 23 |
---|
| 452 | src::gen::base src/code/GenMod_UMConstants # line 24 |
---|
| 453 | src::gen::base src/code/GenMod_Utilities # line 25 |
---|
| 454 | </pre> |
---|
| 455 | |
---|
| 456 | <p>Here is an explanation of what each line does:</p> |
---|
| 457 | |
---|
| 458 | <ul> |
---|
| 459 | <li><dfn>line 6-8</dfn>: these lines declare the repositories for the |
---|
| 460 | <samp>base</samp> branches of the <samp>var</samp>, <samp>ops</samp> and |
---|
| 461 | <samp>gen</samp> packages respectively. It is worth noting that the values |
---|
| 462 | of the declarations are no longer Subversion URLs but are FCM URL keywords. |
---|
| 463 | These keywords are normally declared in the central configuration file of |
---|
| 464 | the FCM system, and will be expanded into the corresponding Subversion URLs |
---|
| 465 | by the FCM system. For further information on URL keywords, please see |
---|
| 466 | <a href="code_management.html#svn_basic_keywords">Code Management System |
---|
| 467 | > Using Subversion > Basic Command Line Usage > Repository & |
---|
| 468 | Revision Keywords</a>.</li> |
---|
| 469 | |
---|
| 470 | <li><dfn>line 10</dfn>: this line declares the revision number for the |
---|
| 471 | <samp>base</samp> branch of the <samp>gen</samp> package, i.e. for the |
---|
| 472 | <samp>fcm:gen_tr</samp> repository. It is worth noting that the revision |
---|
| 473 | numbers for the <samp>var</samp> and <samp>ops</samp> packages have not |
---|
| 474 | been declared. By default, their revision numbers will be set to the last |
---|
| 475 | changed revision at the HEAD.</li> |
---|
| 476 | |
---|
| 477 | <li><dfn>line 12-14</dfn>: these line declares the source directories for |
---|
| 478 | the <samp>base</samp> branches of the <samp>var</samp> and <samp>ops</samp> |
---|
| 479 | packages. For the <samp>var</samp> package, we are extracting everything |
---|
| 480 | from the <samp>code</samp> and the <samp>scripts</samp> sub-directory. For |
---|
| 481 | the <samp>ops</samp> package, we are extracting everything from the |
---|
| 482 | <samp>code</samp> directory.</li> |
---|
| 483 | |
---|
| 484 | <li><dfn>line 15-25</dfn>: these line declares the source directories for |
---|
| 485 | the <samp>base</samp> branch of the <samp>gen</samp> package. The source |
---|
| 486 | directories declared will not be searched for sub-directories underneath |
---|
| 487 | the declared directories.</li> |
---|
| 488 | </ul> |
---|
| 489 | |
---|
| 490 | <p>We shall end up with a directory tree such as:</p> |
---|
| 491 | <pre> |
---|
| 492 | $PWD |
---|
| 493 | | |
---|
| 494 | |--- cfg |
---|
| 495 | | | |
---|
| 496 | | |--- bld.cfg |
---|
| 497 | | |--- ext.cfg |
---|
| 498 | | |
---|
| 499 | |--- src |
---|
| 500 | | |
---|
| 501 | |--- gen |
---|
| 502 | | | |
---|
| 503 | | |--- code |
---|
| 504 | | | |
---|
| 505 | | |--- GenMod_Constants |
---|
| 506 | | |--- GenMod_Control |
---|
| 507 | | |--- GenMod_FortranIO |
---|
| 508 | | |--- GenMod_GetEnv |
---|
| 509 | | |--- GenMod_ModelIO |
---|
| 510 | | |--- GenMod_ObsInfo |
---|
| 511 | | |--- GenMod_Platform |
---|
| 512 | | |--- GenMod_Reporting |
---|
| 513 | | |--- GenMod_Trace |
---|
| 514 | | |--- GenMod_UMConstants |
---|
| 515 | | |--- GenMod_Utilities |
---|
| 516 | | |
---|
| 517 | |--- ops |
---|
| 518 | | | |
---|
| 519 | | |--- code |
---|
| 520 | | | |
---|
| 521 | | |--- ... |
---|
| 522 | | |
---|
| 523 | |--- var |
---|
| 524 | | |
---|
| 525 | |--- code |
---|
| 526 | | | |
---|
| 527 | | |--- ... |
---|
| 528 | | |
---|
| 529 | |--- scripts |
---|
| 530 | | |
---|
| 531 | |--- ... |
---|
| 532 | </pre> |
---|
| 533 | |
---|
| 534 | <dl> |
---|
| 535 | <dt>Note - revision number</dt> |
---|
| 536 | |
---|
| 537 | <dd> |
---|
| 538 | <p>As seen in the above example, if a revision number is not specified |
---|
| 539 | for a repository URL, it defaults to the last changed revision at the |
---|
| 540 | HEAD of the branch. The revision number can also be declared in other |
---|
| 541 | ways:</p> |
---|
| 542 | |
---|
| 543 | <ul> |
---|
| 544 | <li>Any revision arguments acceptable by Subversion are allowed. You |
---|
| 545 | can use a valid revision number, a date between a pair of curly |
---|
| 546 | brackets (e.g. <samp>{2005-05-01T12:00}</samp>) or the keyword HEAD. |
---|
| 547 | However, please do not use the keywords BASE, COMMITTED or PREV as |
---|
| 548 | these are reserved for working copy only.</li> |
---|
| 549 | |
---|
| 550 | <li>FCM revision keywords are allowed. These must be defined for the |
---|
| 551 | corresponding repository URLs in either the central or the user FCM |
---|
| 552 | configuration file. For further information on revision keywords, |
---|
| 553 | please see <a href="code_management.html#svn_basic_keywords">Code |
---|
| 554 | Management > Using Subversion > Basic Command Line Usage > |
---|
| 555 | Repository & Revision Keywords</a>.</li> |
---|
| 556 | |
---|
| 557 | <li>Do not use the keyword USER, as it is used internally by the |
---|
| 558 | extract system.</li> |
---|
| 559 | </ul> |
---|
| 560 | |
---|
| 561 | <p>If a revision number is specified for a branch, the actual revision |
---|
| 562 | used by the extract system is the last changed revision of the branch, |
---|
| 563 | which may differ from the declared revision. While this behaviour is |
---|
| 564 | useful in most situations, some users may find it confusing to work with. |
---|
| 565 | It is possible to alter this behaviour so that extract will fail if the |
---|
| 566 | declared revision does not correspond to a changeset of the declared |
---|
| 567 | branch. Make the following declaration to switch on this checking:</p> |
---|
| 568 | <pre> |
---|
| 569 | revmatch true |
---|
| 570 | </pre> |
---|
| 571 | </dd> |
---|
| 572 | </dl> |
---|
| 573 | |
---|
| 574 | <h3 id="advanced_branches">Extract from multiple branches</h3> |
---|
| 575 | |
---|
| 576 | <p>We have so far dealt with a single branch in any package. The extract |
---|
| 577 | system can be used to <em>combine</em> changes from different branches of a |
---|
| 578 | package. An example is given below:</p> |
---|
| 579 | <pre id="example_5"> |
---|
| 580 | # Example 5 |
---|
| 581 | # ---------------------------------------------------------------------- |
---|
| 582 | cfg::type ext |
---|
| 583 | cfg::version 1.0 |
---|
| 584 | |
---|
| 585 | dest $PWD |
---|
| 586 | |
---|
| 587 | repos::var::base fcm:var_tr |
---|
| 588 | repos::ops::base fcm:ops_tr |
---|
| 589 | repos::gen::base fcm:gen_tr |
---|
| 590 | |
---|
| 591 | revision::gen::base 2468 |
---|
| 592 | |
---|
| 593 | expsrc::var::base src/code |
---|
| 594 | expsrc::var::base src/scripts |
---|
| 595 | expsrc::ops::base src/code |
---|
| 596 | src::gen::base src/code/GenMod_Constants |
---|
| 597 | src::gen::base src/code/GenMod_Control |
---|
| 598 | src::gen::base src/code/GenMod_FortranIO |
---|
| 599 | src::gen::base src/code/GenMod_GetEnv |
---|
| 600 | src::gen::base src/code/GenMod_ModelIO |
---|
| 601 | src::gen::base src/code/GenMod_ObsInfo |
---|
| 602 | src::gen::base src/code/GenMod_Platform |
---|
| 603 | src::gen::base src/code/GenMod_Reporting |
---|
| 604 | src::gen::base src/code/GenMod_Trace |
---|
| 605 | src::gen::base src/code/GenMod_UMConstants |
---|
| 606 | src::gen::base src/code/GenMod_Utilities |
---|
| 607 | |
---|
| 608 | repos::var::branch1 fcm:var_br/frva/r1234_new_stuff # line 27 |
---|
| 609 | repos::var::branch2 fcm:var_br/frva/r1516_bug_fix # line 28 |
---|
| 610 | repos::ops::branch1 fcm:ops_br/opsrc/r3188_good_stuff # line 29 |
---|
| 611 | </pre> |
---|
| 612 | |
---|
| 613 | <p>The configuration file in <a href="#example_5">example 5</a> is similar to |
---|
| 614 | that of <a href="#example_4">example 4</a> except for the last three lines. |
---|
| 615 | Here is an explanation of what they do:</p> |
---|
| 616 | |
---|
| 617 | <ul> |
---|
| 618 | <li> |
---|
| 619 | <dfn>line 27</dfn>: this line declares a repository URL for the |
---|
| 620 | <samp>branch1</samp> branch of the <samp>var</samp> package. From the URL |
---|
| 621 | of the branch, we know that the branch was created by the user |
---|
| 622 | <samp>frva</samp> based on the trunk at revision at 1234. The description |
---|
| 623 | of the branch is <samp>branch1</samp>. The following points are worth |
---|
| 624 | noting: |
---|
| 625 | |
---|
| 626 | <ul> |
---|
| 627 | <li>By declaring a new branch with the same package name to a |
---|
| 628 | previously declared branch, it is assumed that both branches reside in |
---|
| 629 | the same Subversion repository.</li> |
---|
| 630 | |
---|
| 631 | <li>No revision is declared for this URL, so the default is used which |
---|
| 632 | is the last changed revision at the HEAD of the branch.</li> |
---|
| 633 | |
---|
| 634 | <li>No source directory is declared for this URL. By default, if no |
---|
| 635 | source directory is declared for a branch repository, it will attempt |
---|
| 636 | to use the same set of source directories as the first declared branch |
---|
| 637 | of the package. In this case, the source directories declared for the |
---|
| 638 | <samp>base</samp> branch of the <samp>var</samp> package will be |
---|
| 639 | used.</li> |
---|
| 640 | </ul> |
---|
| 641 | </li> |
---|
| 642 | |
---|
| 643 | <li><dfn>line 28</dfn>: this line declares another branch called |
---|
| 644 | <samp>branch2</samp> for the <samp>var</samp> package. No source directory |
---|
| 645 | is declared for this URL either, so it will use the same set of source |
---|
| 646 | directories declared for the <samp>base</samp> branch.</li> |
---|
| 647 | |
---|
| 648 | <li><dfn>line 29</dfn>: this line declares a branch called |
---|
| 649 | <samp>branch1</samp> for the <samp>ops</samp> package. It will use the same |
---|
| 650 | set of source directories declared for the <samp>ops</samp> package |
---|
| 651 | <samp>base</samp> branch.</li> |
---|
| 652 | </ul> |
---|
| 653 | |
---|
| 654 | <p>When we invoke the extract system, it will attempt to extract from the |
---|
| 655 | first declared branch of a package, if the last changed revision of the |
---|
| 656 | source directory is the same in all the branches. However, if the last |
---|
| 657 | changed revision of the source directory differs for different branches, the |
---|
| 658 | system will attempt to obtain an extract priority list for each source |
---|
| 659 | directory, using the following logic:</p> |
---|
| 660 | |
---|
| 661 | <ol> |
---|
| 662 | <li>The system looks for source directory packages from the first declared |
---|
| 663 | branch to the last declared branch.</li> |
---|
| 664 | |
---|
| 665 | <li>The branch in which a source directory package is first declared is the |
---|
| 666 | <samp>base</samp> branch of the source directory package.</li> |
---|
| 667 | |
---|
| 668 | <li>The last changed revision of a source directory package in a |
---|
| 669 | subsequently declared repository branch is compared with that of the base |
---|
| 670 | branch. If the last changed revision is the same as that of the base |
---|
| 671 | branch, the source directory of this branch is discarded. Otherwise, it is |
---|
| 672 | placed at the end of the extract priority list.</li> |
---|
| 673 | </ol> |
---|
| 674 | |
---|
| 675 | <p>For the <samp>var</samp> package in the above example, let us assume that |
---|
| 676 | we have three source directory packages X, Y and Z under <samp>code</samp>, |
---|
| 677 | and their last changed revisions under <samp>base</samp> are 100. Let's say |
---|
| 678 | we have committed some changes to X and Z in the <samp>branch1</samp> branch |
---|
| 679 | at revision 102, and other changes to Y and Z in the <samp>branch2</samp> |
---|
| 680 | branch at revision 104, the extract priority lists for X, Y and Z will look |
---|
| 681 | like:</p> |
---|
| 682 | |
---|
| 683 | <ul> |
---|
| 684 | <li>X: base (100, base), branch1 (102), branch2 (100, discarded)</li> |
---|
| 685 | |
---|
| 686 | <li>Y: base (100, base), branch1 (100, discarded), branch2 (104)</li> |
---|
| 687 | |
---|
| 688 | <li>Z: base (100, base), branch1 (102), branch2 (104)</li> |
---|
| 689 | </ul> |
---|
| 690 | |
---|
| 691 | <p>Once we have an extract priority list for a source directory, we can begin |
---|
| 692 | extracting source files in the source directory. The source directory of the |
---|
| 693 | base branch is extracted first, followed by that in the subsequent branches. |
---|
| 694 | If a source file in a subsequent branch has the same content as the that in |
---|
| 695 | the base branch, it is discarded. Otherwise, the following logic determines |
---|
| 696 | the branch to use:</p> |
---|
| 697 | |
---|
| 698 | <ol> |
---|
| 699 | <li>If a source file is modified in only one subsequent branch, the source |
---|
| 700 | file in that branch is extracted.</li> |
---|
| 701 | |
---|
| 702 | <li>If a source file is modified in two or more subsequent branches, but |
---|
| 703 | their modifications are the same, then the source file in the first |
---|
| 704 | modification is used.</li> |
---|
| 705 | |
---|
| 706 | <li>If a source file is modified in two or more subsequent branches and |
---|
| 707 | their modifications differ, then the behaviour depends on the "conflict |
---|
| 708 | mode" setting, which can be <code>fail</code>, <code>merge</code> (default) |
---|
| 709 | and <code>override</code>. If the conflict mode is <code>fail</code>, the |
---|
| 710 | extract fails. If the conflict mode is <code>merge</code>, the system will |
---|
| 711 | attempt to merge the changes using a tool such as <code>diff3</code>. The |
---|
| 712 | result of the merge will be used to update the destination. The extract |
---|
| 713 | fails only if there are unresolved conflicts in the merge. (In which case, |
---|
| 714 | the conflict should be resolved using the version control system before |
---|
| 715 | re-running the extract system.) If the conflict mode is |
---|
| 716 | <code>override</code>, the change in the latest declared branch takes |
---|
| 717 | precedence, and the changes in all other branches will be ignored. The |
---|
| 718 | conflict mode can be changed using the <code>CONFLICT</code> declaration in |
---|
| 719 | the extract configuration file. E.g: |
---|
| 720 | <pre> |
---|
| 721 | conflict fail |
---|
| 722 | </pre> |
---|
| 723 | </li> |
---|
| 724 | </ol> |
---|
| 725 | |
---|
| 726 | <p>Once the system has established which source files to use, it determines |
---|
| 727 | whether the destination file is out of date or not. The destination file is |
---|
| 728 | out of date if it does not exist or if its content differs from the version |
---|
| 729 | of the source file we are using. The system only updates the destination if |
---|
| 730 | it is considered to be out of date.</p> |
---|
| 731 | |
---|
| 732 | <p>The extract system can also combine changes from branches in the Subversion |
---|
| 733 | repository and the local file system. This is demonstrated in the next |
---|
| 734 | example.</p> |
---|
| 735 | <pre id="example_6"> |
---|
| 736 | # Example 6 |
---|
| 737 | # ---------------------------------------------------------------------- |
---|
| 738 | cfg::type ext |
---|
| 739 | cfg::version 1.0 |
---|
| 740 | |
---|
| 741 | dest $PWD |
---|
| 742 | |
---|
| 743 | repos::var::base fcm:var_tr |
---|
| 744 | repos::ops::base fcm:ops_tr |
---|
| 745 | repos::gen::base fcm:gen_tr |
---|
| 746 | |
---|
| 747 | revision::gen::base 2468 |
---|
| 748 | |
---|
| 749 | expsrc::var::base src/code |
---|
| 750 | expsrc::var::base src/scripts |
---|
| 751 | expsrc::ops::base src/code |
---|
| 752 | src::gen::base src/code/GenMod_Constants |
---|
| 753 | src::gen::base src/code/GenMod_Control |
---|
| 754 | src::gen::base src/code/GenMod_FortranIO |
---|
| 755 | src::gen::base src/code/GenMod_GetEnv |
---|
| 756 | src::gen::base src/code/GenMod_ModelIO |
---|
| 757 | src::gen::base src/code/GenMod_ObsInfo |
---|
| 758 | src::gen::base src/code/GenMod_Platform |
---|
| 759 | src::gen::base src/code/GenMod_Reporting |
---|
| 760 | src::gen::base src/code/GenMod_Trace |
---|
| 761 | src::gen::base src/code/GenMod_UMConstants |
---|
| 762 | src::gen::base src/code/GenMod_Utilities |
---|
| 763 | |
---|
| 764 | repos::var::branch1 fcm:var_br/frva/r1234_new_stuff |
---|
| 765 | repos::var::branch2 fcm:var_br/frva/r1516_bug_fix |
---|
| 766 | repos::ops::branch1 fcm:ops_br/opsrc/r3188_good_stuff |
---|
| 767 | |
---|
| 768 | repos::var::user $HOME/var # line 31 |
---|
| 769 | repos::gen::user $HOME/gen # line 32 |
---|
| 770 | </pre> |
---|
| 771 | |
---|
| 772 | <p><a href="#example_6">Example 6</a> is similar to <a href= |
---|
| 773 | "#example_5">example 5</a> except that it is also extracting from local |
---|
| 774 | directories. Here is an explanation of the lines:</p> |
---|
| 775 | |
---|
| 776 | <ul> |
---|
| 777 | <li><dfn>line 31-32</dfn>: these line declare the repositories for the |
---|
| 778 | <samp>user</samp> branches of the <samp>var</samp> and <samp>gen</samp> |
---|
| 779 | packages respectively. Both are local paths at the local file system. There |
---|
| 780 | are no declarations for source directories for the <samp>user</samp> |
---|
| 781 | branches, so they use the same set of source directories of the first |
---|
| 782 | declared branches, the <samp>base</samp> branches in both cases.</li> |
---|
| 783 | </ul> |
---|
| 784 | |
---|
| 785 | <dl> |
---|
| 786 | <dt>Note - the INC declaration</dt> |
---|
| 787 | |
---|
| 788 | <dd> |
---|
| 789 | You have probably realised that the above examples have many repeated |
---|
| 790 | lines. To avoid having repeated lines in multiple extract configuration |
---|
| 791 | files, you can use <code>INC</code> declarations to include other extract |
---|
| 792 | configuration files. For example, if the configuration file of example 5 |
---|
| 793 | is stored in the file <samp>$HOME/example5/ext.cfg</samp>, line 1 to 29 |
---|
| 794 | of <a href="#example_6">example 6</a> can be replaced with an |
---|
| 795 | <code>INC</code> declaration. <a href="#example_6">Example 6</a> can then |
---|
| 796 | be written as: |
---|
| 797 | <pre> |
---|
| 798 | inc $HOME/example5/ext.cfg |
---|
| 799 | |
---|
| 800 | repos::var::user $HOME/var |
---|
| 801 | repos::gen::user $HOME/gen |
---|
| 802 | </pre> |
---|
| 803 | |
---|
| 804 | <p>Note: the <code>INC</code> declaration supports the special |
---|
| 805 | environment variable <var>$HERE</var>. If this variable is already set in |
---|
| 806 | the environment, it acts as a normal environment variable. However, if it |
---|
| 807 | is not set, it will be expanded into the container directory of the |
---|
| 808 | current extract configuration file. This feature is particularly useful |
---|
| 809 | if you are including a hierarchy of extract configurations from files in |
---|
| 810 | the same container directory in a repository.</p> |
---|
| 811 | </dd> |
---|
| 812 | </dl> |
---|
| 813 | |
---|
| 814 | <h3 id="advanced_inherit">Inherit from a previous extract</h3> |
---|
| 815 | |
---|
| 816 | <p>All the examples above dealt with standalone extract, that is, the current |
---|
| 817 | extract is independent of any other extract. If a previous extract exists in |
---|
| 818 | another location, the extract system can inherit from this previous extract |
---|
| 819 | in your current extract. This works like a normal incremental extract, except |
---|
| 820 | that your extract will only contain the changes you have specified (compared |
---|
| 821 | with the inherited extract) instead of the full source directory tree. This |
---|
| 822 | type of incremental extract is useful in several ways. For instance:</p> |
---|
| 823 | |
---|
| 824 | <ul> |
---|
| 825 | <li>It is fast, because you only have to extract and mirror files that you |
---|
| 826 | have changed.</li> |
---|
| 827 | |
---|
| 828 | <li>The subsequent build will also be fast, since it will use incremental |
---|
| 829 | build.</li> |
---|
| 830 | |
---|
| 831 | <li>You do not need write access to the original extract. A system |
---|
| 832 | administrator can set up a stable version in a central account, which |
---|
| 833 | developers can then inherit from.</li> |
---|
| 834 | |
---|
| 835 | <li>You want an incremental extract, but you need to leave the original |
---|
| 836 | extract unmodified.</li> |
---|
| 837 | </ul> |
---|
| 838 | |
---|
| 839 | <p>The following example is based on <a href="#example_4">example 4</a> and |
---|
| 840 | <a href="#example_6">example 6</a>. The assumption is that an extract has |
---|
| 841 | already been performed at the directory <samp>~frva/var/vn22.0</samp> based |
---|
| 842 | on the configuration file in <a href="#example_4">example 4</a>.</p> |
---|
| 843 | <pre id="example_7"> |
---|
| 844 | # Example 7 |
---|
| 845 | # ---------------------------------------------------------------------- |
---|
| 846 | cfg::type ext |
---|
| 847 | cfg::version 1.0 |
---|
| 848 | |
---|
| 849 | dest $PWD |
---|
| 850 | |
---|
| 851 | use ~frva/var/vn22.0 # line 6 |
---|
| 852 | |
---|
| 853 | repos::var::branch1 fcm:var_br/frva/r1234_new_stuff # line 8 |
---|
| 854 | repos::var::branch2 fcm:var_br/frva/r1516_bug_fix # line 9 |
---|
| 855 | repos::ops::branch1 fcm:ops_br/opsrc/r3188_good_stuff # line 10 |
---|
| 856 | |
---|
| 857 | repos::var::user $HOME/var # line 12 |
---|
| 858 | repos::gen::user $HOME/gen # line 13 |
---|
| 859 | </pre> |
---|
| 860 | |
---|
| 861 | <ul> |
---|
| 862 | <li><dfn>line 6</dfn>: this line replaces line 1 to 25 of <a href= |
---|
| 863 | "#example_6">example 6</a>. It declares that the current extract should |
---|
| 864 | inherit from the previous extract located at |
---|
| 865 | <samp>~frva/var/vn22.0</samp>.</li> |
---|
| 866 | </ul> |
---|
| 867 | |
---|
| 868 | <p>Running the extract system using the above configuration will trigger an |
---|
| 869 | incremental extract, as if you are running an incremental extract having |
---|
| 870 | modified the configuration file in <a href="#example_4">example 4</a> to that |
---|
| 871 | of <a href="#example_6">example 6</a>. The only difference is that the |
---|
| 872 | original extract using the <a href="#example_4">example 4</a> configuration |
---|
| 873 | will be left untouched at <samp>~frva/var/vn22.0</samp>, and the new extract |
---|
| 874 | will contain only the changes in the branches declared from line 8 to 13.</p> |
---|
| 875 | |
---|
| 876 | <p>Note: extract inheritance allows you to add more branches to a package, |
---|
| 877 | but you should not redefine the <code>REPOS</code>, <code>REVISION</code>, |
---|
| 878 | <code>EXPSRC</code> or <code>SRC</code> declarations of a branch that is |
---|
| 879 | already declared (and already extracted) in the inherited extract. Although |
---|
| 880 | the system will not stop you from doing so, you may end up with an extract |
---|
| 881 | that does not quite do what it is supposed to do. For example, if the |
---|
| 882 | <samp>base</samp> branch in the <samp>foo</samp> package |
---|
| 883 | (<tt>repos::foo::base</tt>) is already defined and extracted in an extract |
---|
| 884 | you are inheriting from, you should not redefine any of the |
---|
| 885 | <tt>*::foo::base</tt> declarations in your current extract. However, you are |
---|
| 886 | free to add more branches for the same package with new labels (e.g. |
---|
| 887 | <tt>repos::foo::b1</tt>), and indeed new packages that are not already |
---|
| 888 | defined in the inherited extract (e.g. <tt>repos::bar::base</tt>).</p> |
---|
| 889 | |
---|
| 890 | <p>If you are setting up an extract to be inherited, you do not have to |
---|
| 891 | perform a build. If you don't you will still gain the benefit of incremental |
---|
| 892 | file extract, but you will be performing a full build of the code.</p> |
---|
| 893 | |
---|
| 894 | <dl> |
---|
| 895 | <dt>Note - inherit and mirror</dt> |
---|
| 896 | |
---|
| 897 | <dd> |
---|
| 898 | <p>It is worth bearing in mind that <tt>rdest::*</tt> settings are not |
---|
| 899 | inherited. If mirroring is required in the inheriting extract, it will |
---|
| 900 | require its own set of <tt>rdest::*</tt> declarations.</p> |
---|
| 901 | |
---|
| 902 | <p>The system will, however, assume that a mirrored version of the |
---|
| 903 | inherited extract is available for inheritance from the mirrored |
---|
| 904 | destination of the current extract.</p> |
---|
| 905 | |
---|
| 906 | <p>E.g.: Consider an extract at <samp>/path/to/inherited/</samp> and an |
---|
| 907 | inheriting extract at <samp>/path/to/current/</samp>. If the former does |
---|
| 908 | not have a mirror, the latter should not have one either. If the former |
---|
| 909 | mirrors to <samp>machine@/path/to/inherited/mirror/</samp> and the latter |
---|
| 910 | mirrors to <samp>machine@/path/to/current/mirror/</samp>, the system will |
---|
| 911 | assume that the subsequent build at |
---|
| 912 | <samp>machine@/path/to/current/mirror/</samp> can inherit from the build |
---|
| 913 | at <samp>machine@/path/to/inherited/mirror/</samp>. This is illustrated |
---|
| 914 | below:</p> |
---|
| 915 | |
---|
| 916 | <pre> |
---|
| 917 | /path/to/current/ => at machine: /path/to/current/mirror/ |
---|
| 918 | use /path/to/inherited/ => at machine: use /path/to/inherited/mirror/ |
---|
| 919 | </pre> |
---|
| 920 | </dd> |
---|
| 921 | </dl> |
---|
| 922 | |
---|
| 923 | <h3 id="advanced_build">Extract - Build Configuration</h3> |
---|
| 924 | |
---|
| 925 | <p>Configuration settings for feeding into the build system can be declared |
---|
| 926 | through the extract configuration file using the <code>BLD::</code> prefix. |
---|
| 927 | Any line in an extract configuration containing a label with such a prefix |
---|
| 928 | will be considered a build system variable. At the end of a successful |
---|
| 929 | extract, the system strips out the <code>BLD::</code> prefix before writing |
---|
| 930 | these variables to the build configuration file. Some example entries are |
---|
| 931 | given between line 17 and 22 in the following configuration file:</p> |
---|
| 932 | <pre id="example_8"> |
---|
| 933 | # Example 8 |
---|
| 934 | # ---------------------------------------------------------------------- |
---|
| 935 | cfg::type ext |
---|
| 936 | cfg::version 1.0 |
---|
| 937 | |
---|
| 938 | dest $PWD |
---|
| 939 | |
---|
| 940 | repos::var::base fcm:var_tr |
---|
| 941 | repos::ops::base fcm:ops_tr |
---|
| 942 | repos::gen::base fcm:gen_tr |
---|
| 943 | |
---|
| 944 | revision::gen::base 2468 |
---|
| 945 | |
---|
| 946 | expsrc::var::base src/code |
---|
| 947 | expsrc::var::base src/scripts |
---|
| 948 | expsrc::ops::base src/code |
---|
| 949 | src::gen::base src/code/GenMod_Constants |
---|
| 950 | src::gen::base src/code/GenMod_Control |
---|
| 951 | src::gen::base src/code/GenMod_FortranIO |
---|
| 952 | src::gen::base src/code/GenMod_GetEnv |
---|
| 953 | src::gen::base src/code/GenMod_ModelIO |
---|
| 954 | src::gen::base src/code/GenMod_ObsInfo |
---|
| 955 | src::gen::base src/code/GenMod_Platform |
---|
| 956 | src::gen::base src/code/GenMod_Reporting |
---|
| 957 | src::gen::base src/code/GenMod_Trace |
---|
| 958 | src::gen::base src/code/GenMod_UMConstants |
---|
| 959 | src::gen::base src/code/GenMod_Utilities |
---|
| 960 | |
---|
| 961 | bld::target VarProg_AnalysePF.exe # line 27 |
---|
| 962 | |
---|
| 963 | bld::tool::fc sxmpif90 # line 29 |
---|
| 964 | bld::tool::cc sxmpic++ # line 30 |
---|
| 965 | bld::tool::ld sxmpif90 # line 31 |
---|
| 966 | </pre> |
---|
| 967 | |
---|
| 968 | <p>The above example is essentially the same as <a href="#example_4">example |
---|
| 969 | 4</a>, apart from the additional build configuration. The following is a |
---|
| 970 | simple explanation of what the lines represent: (For detail of the build |
---|
| 971 | system, please see the next chapter on <a href="build.html">The Build |
---|
| 972 | System</a>.)</p> |
---|
| 973 | |
---|
| 974 | <ul> |
---|
| 975 | <li><dfn>line 27</dfn>: the line declares a default target of the |
---|
| 976 | build.</li> |
---|
| 977 | |
---|
| 978 | <li><dfn>line 29-31</dfn>: the lines declare the Fortran compiler, the C |
---|
| 979 | compiler and the linker respectively.</li> |
---|
| 980 | </ul> |
---|
| 981 | |
---|
| 982 | <dl> |
---|
| 983 | <dt>Note - use of variables</dt> |
---|
| 984 | |
---|
| 985 | <dd> |
---|
| 986 | <p>When you start using the extract system to define compiler flags for |
---|
| 987 | the build system, you may end up having to make a lot of long and |
---|
| 988 | repetitive declarations. In this case, you may want to define variables |
---|
| 989 | to replace the repetitive parts of the declarations.</p> |
---|
| 990 | |
---|
| 991 | <p>Environment variables whose names contain only upper case latin |
---|
| 992 | alphabets, numbers and underscores can be referenced in a declaration |
---|
| 993 | value via the syntax <code>$NAME</code> or <code>${NAME}</code>. For |
---|
| 994 | example:</p> |
---|
| 995 | <pre> |
---|
| 996 | repos::um::base ${HOME}/svn-wc/um |
---|
| 997 | bld::tool::fflags $MY_FFLAGS |
---|
| 998 | </pre> |
---|
| 999 | |
---|
| 1000 | <p>You can define a user variable by making a declaration with a label |
---|
| 1001 | that begins with a percent sign <code>%</code>. The value of a user |
---|
| 1002 | variable remains in memory until the end of the current file is reached. |
---|
| 1003 | You can reference a user variable in a declaration value via the syntax |
---|
| 1004 | <code>%NAME</code> or <code>%{NAME}</code>. For example:</p> |
---|
| 1005 | <pre> |
---|
| 1006 | # Declare a variable %fred |
---|
| 1007 | %fred -Cdebug -eC -Wf,-init heap=nan stack=nan |
---|
| 1008 | |
---|
| 1009 | bld::tool::fflags %fred |
---|
| 1010 | # bld::tool::fflags -Cdebug -eC -Wf,-init heap=nan stack=nan |
---|
| 1011 | |
---|
| 1012 | bld::tool::fflags::foo %fred -f0 |
---|
| 1013 | # bld::tool::fflags::foo -Cdebug -eC -Wf,-init heap=nan stack=nan -f0 |
---|
| 1014 | |
---|
| 1015 | bld::tool::fflags::bar -w %fred |
---|
| 1016 | # bld::tool::fflags::bar -w -Cdebug -eC -Wf,-init heap=nan stack=nan |
---|
| 1017 | </pre> |
---|
| 1018 | |
---|
| 1019 | <p>Further to this, each declaration results in an internal variable of |
---|
| 1020 | the same name and you can also refer to any of these internal variables in |
---|
| 1021 | the same way. So, the example given above could also be written as |
---|
| 1022 | follows:</p> |
---|
| 1023 | <pre> |
---|
| 1024 | bld::tool::fflags -Cdebug -eC -Wf,-init heap=nan stack=nan |
---|
| 1025 | bld::tool::fflags::foo %bld::tool::fflags -f0 |
---|
| 1026 | bld::tool::fflags::bar -w %bld::tool::fflags |
---|
| 1027 | </pre> |
---|
| 1028 | </dd> |
---|
| 1029 | |
---|
| 1030 | <dt>Note - as-parsed configuration</dt> |
---|
| 1031 | |
---|
| 1032 | <dd> |
---|
| 1033 | <p>If you use a hierarchy of <code>INC</code> declarations or variables, |
---|
| 1034 | you may end up with a configuration file that is difficult to understand. |
---|
| 1035 | To help you with this, the extract system generates an as-parsed |
---|
| 1036 | configuration file at <samp>cfg/parsed_ext.cfg</samp> of the destination. |
---|
| 1037 | The content of the as-parsed configuration file is what the extract |
---|
| 1038 | system actually reads. It should contain everything in your original |
---|
| 1039 | extract configuration file, except that all <code>INC</code> |
---|
| 1040 | declarations, environment variables and user/internal variables are |
---|
| 1041 | expanded.</p> |
---|
| 1042 | </dd> |
---|
| 1043 | </dl> |
---|
| 1044 | |
---|
| 1045 | <h2 id="verbose">Diagnostic verbose level</h2> |
---|
| 1046 | |
---|
| 1047 | <p>The amount of diagnostic messages generated by the extract system is |
---|
| 1048 | normally set to a level suitable for normal everyday operation. This is the |
---|
| 1049 | default diagnostic verbose level 1. If you want a minimum amount of |
---|
| 1050 | diagnostic messages, you should set the verbose level to 0. If you want more |
---|
| 1051 | diagnostic messages, you can set the verbose level to 2 or 3. You can modify |
---|
| 1052 | the verbose level in two ways. The first way is to set the environment |
---|
| 1053 | variable <var>FCM_VERBOSE</var> to the desired verbose level. The second way |
---|
| 1054 | is to invoke the extract system with the <code>-v <level></code> |
---|
| 1055 | option. (If set, the command line option overrides the environment |
---|
| 1056 | variable.)</p> |
---|
| 1057 | |
---|
| 1058 | <p>The following is a list of diagnostic output at each verbose level:</p> |
---|
| 1059 | |
---|
| 1060 | <dl> |
---|
| 1061 | <dt>Level 0</dt> |
---|
| 1062 | |
---|
| 1063 | <dd> |
---|
| 1064 | <ul> |
---|
| 1065 | <li>Report the time taken to extract the code.</li> |
---|
| 1066 | |
---|
| 1067 | <li>Report the time taken to mirror the code.</li> |
---|
| 1068 | |
---|
| 1069 | <li>If <code>rdist</code> is used to mirror the code, run the command |
---|
| 1070 | with the <code>-q</code> option.</li> |
---|
| 1071 | </ul> |
---|
| 1072 | </dd> |
---|
| 1073 | |
---|
| 1074 | <dt>Level 1</dt> |
---|
| 1075 | |
---|
| 1076 | <dd> |
---|
| 1077 | <ul> |
---|
| 1078 | <li>Everything at verbose level 0.</li> |
---|
| 1079 | |
---|
| 1080 | <li>Report the name of the extract configuration file.</li> |
---|
| 1081 | |
---|
| 1082 | <li>Report the location of the extract destination.</li> |
---|
| 1083 | |
---|
| 1084 | <li>Report date/time at the beginning of the extract step.</li> |
---|
| 1085 | |
---|
| 1086 | <li>If the revision specified for a repository branch is not its last |
---|
| 1087 | changed revision, print an information statement to inform the user of |
---|
| 1088 | the last changed revision of the branch.</li> |
---|
| 1089 | |
---|
| 1090 | <li>Summarises the destination status and the source status.</li> |
---|
| 1091 | |
---|
| 1092 | <li>Report date/time at the beginning of the mirror step.</li> |
---|
| 1093 | |
---|
| 1094 | <li>Report the location of the alternate destination.</li> |
---|
| 1095 | |
---|
| 1096 | <li>Report total time.</li> |
---|
| 1097 | </ul> |
---|
| 1098 | </dd> |
---|
| 1099 | |
---|
| 1100 | <dt>Level 2</dt> |
---|
| 1101 | |
---|
| 1102 | <dd> |
---|
| 1103 | <ul> |
---|
| 1104 | <li>Everything at verbose level 1.</li> |
---|
| 1105 | |
---|
| 1106 | <li>If the revision specified for a repository branch is not current |
---|
| 1107 | (i.e. the specified revision number is less than the revision number of |
---|
| 1108 | the last commit revision), print an information statement to inform the |
---|
| 1109 | user of the last commit revision of the branch.</li> |
---|
| 1110 | |
---|
| 1111 | <li>Report the detail of each change in the destination.</li> |
---|
| 1112 | |
---|
| 1113 | <li>If <code>rdist</code> is used to mirror the code, run the command |
---|
| 1114 | without the <code>-q</code> option.</li> |
---|
| 1115 | </ul> |
---|
| 1116 | </dd> |
---|
| 1117 | |
---|
| 1118 | <dt>Level 3</dt> |
---|
| 1119 | |
---|
| 1120 | <dd> |
---|
| 1121 | <ul> |
---|
| 1122 | <li>Everything at verbose level 2.</li> |
---|
| 1123 | |
---|
| 1124 | <li>Report all shell commands invoked by the extract system with |
---|
| 1125 | timestamp.</li> |
---|
| 1126 | |
---|
| 1127 | <li>If <code>rdist</code> is used to mirror the code, print the |
---|
| 1128 | <samp>distfile</samp> supplied to the command.</li> |
---|
| 1129 | |
---|
| 1130 | <li>If <code>rsync</code> is used to mirror the code, invoke the |
---|
| 1131 | command with the <code>-v</code> option.</li> |
---|
| 1132 | </ul> |
---|
| 1133 | </dd> |
---|
| 1134 | </dl> |
---|
| 1135 | |
---|
| 1136 | <h2 id="nosvn">When Subversion Is Not Available</h2> |
---|
| 1137 | |
---|
| 1138 | <p>The extract system can still be used if Subversion is not available. |
---|
| 1139 | Clearly, you can only use local repositories. However, you can still do |
---|
| 1140 | incremental extract, mirror an extract to an alternate location, or combine |
---|
| 1141 | code from multiple local repositories.</p> |
---|
| 1142 | |
---|
| 1143 | <p>If you are using Subversion but your server is down then clearly there is |
---|
| 1144 | little you can do. However, if you already have an extract then you can |
---|
| 1145 | re-run <code>fcm extract</code> as long as the extract configuration file |
---|
| 1146 | only refers to fixed revisions. If this is not the case then you can always |
---|
| 1147 | use the expanded extract configuration file which can be found in |
---|
| 1148 | <samp>cfg/ext.cfg</samp> under the extract destination root. This means that |
---|
| 1149 | you can continue to makes changes to local code and do incremental extracts |
---|
| 1150 | even whilst your Subversion server is down.</p> |
---|
| 1151 | |
---|
| 1152 | </div> |
---|
| 1153 | </div> |
---|
| 1154 | </div> |
---|
| 1155 | |
---|
| 1156 | <hr/> |
---|
| 1157 | <div class="container-fluid text-center"> |
---|
| 1158 | <div class="row"><div class="col-md-12"> |
---|
| 1159 | <address><small> |
---|
| 1160 | Copyright © 2006-2021 British Crown (Met Office) & Contributors. |
---|
| 1161 | <a href="http://www.metoffice.gov.uk">Met Office</a>. |
---|
| 1162 | See <a href="../etc/fcm-terms-of-use.html">Terms of Use</a>.<br /> |
---|
| 1163 | This document is released under the British <a href= |
---|
| 1164 | "http://www.nationalarchives.gov.uk/doc/open-government-licence/" rel= |
---|
| 1165 | "license">Open Government Licence</a>.<br /> |
---|
| 1166 | </small></address> |
---|
| 1167 | </div></div> |
---|
| 1168 | </div> |
---|
| 1169 | |
---|
| 1170 | <script type="text/javascript" src="../etc/jquery.min.js"></script> |
---|
| 1171 | <script type="text/javascript" src="../etc/bootstrap/js/bootstrap.min.js"></script> |
---|
| 1172 | <script type="text/javascript" src="../etc/fcm.js"></script> |
---|
| 1173 | <script type="text/javascript" src="../etc/fcm-version.js"></script> |
---|
| 1174 | </body> |
---|
| 1175 | </html> |
---|