source: LMDZ6/branches/Amaury_dev/tools/fcm/doc/user_guide/system_admin.html

Last change on this file was 5129, checked in by abarral, 18 months ago

Re-add removed by mistake fcm
File size: 25.0 KB
Line 
1<!DOCTYPE html>
2<html>
3<head>
4  <title>FCM: User Guide: System Administration</title>
5  <meta name="author" content="FCM team" />
6  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7  <link rel="icon" href="../etc/fcm-icon.png" type="image/png" />
8  <link rel="shortcut icon" href="../etc/fcm-icon.png" type="image/png" />
9  <link href="../etc/bootstrap/css/bootstrap.min.css" rel="stylesheet" media="screen" />
10  <link href="../etc/fcm.css" rel="stylesheet" media="screen" />
11</head>
12<body>
13  <div class="navbar navbar-inverse">
14    <div class="container-fluid">
15      <div class="navbar-header">
16        <a class="navbar-brand" href=".."><span class="fcm-version">FCM</span></a>
17      </div>
18      <div class="collapse navbar-collapse">
19        <ul class="nav navbar-nav">
20          <li><a href="../installation/">Installation</a></li>
21
22          <li><a class="active" href="#">User Guide</a></li>
23        </ul>
24      </div>
25    </div>
26  </div>
27
28  <div class="page-header">
29    <div class="fcm-page-content pull-right well well-sm"></div>
30    <h1>FCM: User Guide: System Administration</h1>
31  </div>
32
33  <div class="container">
34  <div class="row">
35  <div class="col-md-12">
36
37  <h2 id="introduction">Introduction</h2>
38
39  <p>This chapter provides an administration guide for managers of projects or
40  systems which are using FCM.</p>
41
42  <p>Note that, where this section refers to the <em>FCM team</em> this applies
43  only to Met Office users. Users at other sites will either need to refer to the
44  equivalent team within their organisation or will need to perfom these tasks
45  themselves.</p>
46
47  <h2 id="svn">Subversion</h2>
48
49  <h3 id="svn_layout">Repository Layout</h3>
50
51  <p>In theory you can set up your repository to have any random layouts.
52  However, many <code>fcm</code> commands have to make assumptions on a set of
53  working practices in order to function. The easiest way is to go with the
54  default:</p>
55  <pre>
56&lt;root&gt;
57    |
58    |-- &lt;project 1&gt;
59    |       |
60    |       |-- trunk
61    |       |-- branches
62    |       |-- tags
63    |
64    |-- &lt;project 2&gt;
65    |       |
66    |       |-- trunk
67    |       |-- branches
68    |       |-- tags
69    |
70    |-- ...
71</pre>
72 
73  <p>In the default layout, each project is a sub-directory under the repository
74  root. Each project has a <code>trunk</code> sub-directory, and optionally a
75  <code>branches</code> sub-directory and a <code>tags</code> sub-directory. The
76  main line of development of the project lives directly under the
77  <code>trunk</code> sub-directory. A branch lives 3 levels under the
78  <code>branches</code> sub-directory. A tag lives 1 level under the
79  <code>tags</code> sub-directory.</p>
80
81  <p>FCM allows you to customise the layout for each repository by adding an
82  <code>fcm:layout</code> property at the HEAD of root of the repository.
83  E.g.:</p>
84
85  <pre>
86(shell prompt)$ fcm co -q -N svn://host/repos repos-root
87(shell prompt)$ cd repos-root
88(shell prompt)$ fcm pe fcm:layout .
89</pre>
90
91  <p>The default settings are given in the following. In the editor started by
92  <code>fcm pe</code>, add these settings and modify the VALUE of each KEY=VALUE
93  pair.</p>
94
95  <pre>
96depth-project =
97depth-branch = 3
98depth-tag = 1
99dir-trunk = trunk
100dir-branch = branches
101dir-tag = tags
102level-owner-branch = 2
103level-owner-tag =
104template-branch = {category}/{owner}/{name_prefix}{name}
105template-tag =
106</pre>
107
108  <p>The settings will become effective when you <code>fcm commit</code> them.
109  An empty VALUE denotes an undefined value. The meanings of the settings are
110  described below:</p>
111
112  <dl>
113    <dt><code>depth-project</code></dt>
114
115    <dd>Number of sub-directories expected to be used by the name of a project.
116    An undefined value means that a project can live under an
117    arbitrary number of sub-directories (or directly) below the repository
118    root.</dd>
119
120    <dt><code>depth-branch</code></dt>
121
122    <dd>Number of sub-directories (under the sub-directory defined by
123    <code>dir-branch</code>) expected to be used by the name of a branch. This
124    setting must be defined.</dd>
125
126    <dt><code>depth-tag</code></dt>
127
128    <dd>Number of sub-directories (under the sub-directory defined by
129    <code>dir-tag</code>) expected to be used by the name of a tag. This
130    setting must be defined.</dd>
131
132    <dt><code>dir-trunk</code></dt>
133
134    <dd>The sub-directory (under the project) where the trunk source tree lives.
135    This setting must be defined.</dd>
136
137    <dt><code>dir-branch</code></dt>
138
139    <dd>The sub-directory (under the project) where (the sub-directories
140    containing) all the branch source trees live.</dd>
141
142    <dt><code>dir-tag</code></dt>
143
144    <dd>The sub-directory (under the project) where (the sub-directories
145    containing) all the tag source trees live.</dd>
146
147    <dt><code>level-owner-branch</code></dt>
148
149    <dd>The sub-directory level in the name of a branch containing its owner.</dd>
150
151    <dt><code>level-owner-tag</code></dt>
152
153    <dd>The sub-directory level in the name of a tag containing its owner.</dd>
154
155    <dt><code>template-branch</code></dt>
156
157    <dd>The template string to construct a branch name.</dd>
158
159    <dt><code>template-tag</code></dt>
160
161    <dd>The template string to construct a tag name.</dd>
162  </dl>
163
164  <p>You will need to decide whether to use a single project tree for your
165  system or whether to use multiple projects.</p>
166
167  <p>Advantages of a single project tree:</p>
168
169  <ul>
170    <li>Changes to any part of the system can always be committed as a single
171    logical changeset. If you split your system into multiple projects then you
172    may have occasions when a logical change involves more than one project and
173    hence requires multiple commits (and branches).</li>
174  </ul>
175
176  <p>Disadvantages of a single project tree:</p>
177
178  <ul>
179    <li>If you have a large system then your working copies may become very
180    large and unwieldy. Basic commands such as <code>checkout</code> and
181    <code>status</code> can become frustratingly slow if your working copy is
182    too large.</li>
183
184    <li>Depending on how you work, you may end up doing lots more merges of
185    files that are unrelated to your work.</li>
186  </ul>
187
188  <p>One common approach is to split the admin type files (e.g. site
189  configurations that are unrelated to the main release) into a separate project
190  from the core system files. If you include any large data files under version
191  control you may also want to use a separate project for them to avoid making
192  your working copies very large when editing code.</p>
193
194  <p>Note that there is often no obvious right or wrong answer so you just have
195  to make a decision and see how it works out. You can always re-arrange your
196  repository in the future (although be aware that this will break any changes
197  being prepared on branches at the time).</p>
198
199  <p>You also need to decide whether your system requires its own repository
200  (or multiple repositories) or whether it can share with another system.</p>
201
202  <ul>
203    <li>The main disadvantage of having separate repositories for each system
204    is the maintenance overhead (although this is almost all automated by the
205    FCM team so is not a big deal).</li>
206
207    <li>We normally configure a single Trac environment per repository. If the
208    repository contains multiple systems then it makes it difficult to use the
209    Trac milestones to handle system releases. However, Trac now supports
210    restricting itself to a sub-directory within a repository so, again, this
211    is not a big deal.</li>
212
213    <li>If you share a repository with other systems then your revision numbers
214    can increase even when there are no changes to your system. This doesn't
215    matter but some people don't like it.</li>
216  </ul>
217
218  <p>For simplicity, in most cases you will probably want your own repository
219  for your system.</p>
220
221  <p>You will not normally want to have multiple repositories for a system. One
222  exception may be if you are storing large data files where you might not want
223  to keep all the old versions for ever. Removing old versions can't be done
224  without changing all the revision numbers which would mess up all your code
225  history and Trac tickets. Storing the large data files in a separate
226  repository reduces the impact if you do decide to remove old versions in the
227  future. One disadvantage of this approach is that, for the moment at least,
228  Trac only handles one repository so you will need a separate Trac environment for
229  the data files.</p>
230
231  <p>For further details please see the section <a href=
232  "http://svnbook.red-bean.com/en/1.8/svn.reposadmin.planning.html#svn.reposadmin.projects.chooselayout">
233  Planning Your Repository Organization</a> from the Subversion book.</p>
234
235  <h3 id="svn_create">Creating a Repository</h3>
236
237  <p>Normally the FCM team will help you to set up your initial repository.
238  However, it is quite simple if you need to do it yourself. First you need to
239  issue the command <code>svnadmin create /path/to/repos</code>. This creates an
240  empty repository which is now ready to accept an initial import. To do so, you
241  should create a directory tree in a suitable location, and issue the <code>fcm
242  project-create</code> command. At the root of the repository should be the
243  project directories. Each project should then contain the <samp>trunk</samp>
244  sub-directory.  The sub-directories <samp>branches</samp> and
245  <samp>tags</samp> are optional.  You can import your source files to the
246  <samp>trunk</samp> after the project is created. For example, if your
247  directory tree is located at <samp>$HOME/foo</samp>, you will do the following
248  to import it to a new repository:</p>
249  <pre>
250(SHELL PROMPT)$ svnadmin create FOO_svn
251(SHELL PROMPT)$ fcm project-create FOO file://$PWD/FOO_svn
252(SHELL PROMPT)$ fcm checkout file://$PWD/FOO_svn/FOO $HOME/svn-wc/foo
253(SHELL PROMPT)$ cd $HOME/svn-wc/foo
254(SHELL PROMPT)$ cp -r $HOME/foo/* .
255(SHELL PROMPT)$ fcm add *
256(SHELL PROMPT)$ fcm status
257(SHELL PROMPT)$ fcm commit
258</pre>
259
260  <p>Note that the <code>svnadmin</code> command takes a <var>PATH</var> as an
261  argument, as opposed to a URL for the <code>svn</code> command.</p>
262
263  <p>For further details please see the section <a href=
264  "http://svnbook.red-bean.com/en/1.8/svn.reposadmin.planning.html#svn.reposadmin.projects.chooselayout">
265  Planning Your Repository Organization</a> from the Subversion book.</p>
266
267  <h3 id="svn_access">Access Control</h3>
268
269  <p>Restrictions such as preventing anonymous read access or restricting write
270  access to the trunk to a limited set of users can be arranged if
271  necessary.</p>
272
273  <h3 id="svn_hosting">Repository Hosting</h3>
274
275  <p>The FCM team will organise the hosting of your repository. A number of
276  facilities will be set up for you as standard.</p>
277
278  <p>Your repository will be set up on a central FCM server and access will be
279  provided via <code>svnserve</code> (which we use in preference to
280  <cite>Apache</cite> for performance reasons). The FCM team will advise you of
281  the URL, and put in place standard hook scripts and backup procedures.</p>
282
283  <p>Note that if you want to use a Subversion repository for your own
284  individual use there is no need to get the FCM team to host it. You can
285  simply create your repository and then use a <code>file://</code> URL to
286  access it.</p>
287
288  <h2 id="trac">Trac</h2>
289
290  <h3 id="trac_config">Trac Configuration</h3>
291
292  <p>Normally the FCM team will set up your Trac environment for you. This
293  section describes some things you may wish to be configured. This can be done
294  when the Trac environment is set up or later if you are unsure what you will
295  require at first.</p>
296
297  <h4 id="trac_access">Access Control</h4>
298
299  <p>You will not normally want to allow anonymous users to make changes to
300  your Trac environment since this means that changes may not get identified with a
301  userid. The FCM team will normally set up your Trac environment such that any
302  authenticated users can make changes. Further restrictions such as
303  restricting write access to named accounts or preventing anonymous read
304  access can be arranged if necessary.</p>
305
306  <p>The system manager will normally be given <var>TRAC_ADMIN</var>
307  privileges. This allows them to do additional things which normal users
308  cannot do such as:</p>
309
310  <ul>
311    <li>Delete wiki pages (the latest version or the entire page).</li>
312
313    <li>Add or modify milestones, components and versions.</li>
314
315    <li>Modify ticket descriptions and delete ticket attachments.</li>
316
317    <li>Make wiki pages read-only.</li>
318
319    <li>Alter the permissions.</li>
320  </ul>
321
322  <p>For further details please see the section <a href=
323  "http://trac.edgewall.org/wiki/TracPermissions">Trac Permissions</a> from the
324  Trac documentation.</p>
325
326  <h4 id="trac_email">Email Notification</h4>
327
328  <p>By default, each Trac environment is configured such that the owner and
329  reporter and anyone on the <var>CC</var> list are notified whenever a change
330  is made to a ticket. If system mangers wish to be notified of all ticket
331  changes then this can also be configured. Alternatively, email notifications
332  can be disabled if they are not wanted.</p>
333
334  <h4 id="trac_misc">Other Configurable Items</h4>
335
336  <p>There are lots of other things that can be configured in your Trac
337  environment
338  such as:</p>
339
340  <ul>
341    <li>Custom fields</li>
342
343    <li>System icon</li>
344
345    <li>Stylesheets</li>
346  </ul>
347
348  <p>For further details please see the sections <a href=
349  "http://trac.edgewall.org/wiki/TracIni">The Trac Configuration File</a> and
350  <a href="http://trac.edgewall.org/wiki/TracTickets">The Trac Ticket
351  System</a> from the Trac documentation.</p>
352
353  <h3 id="trac_hosting">Trac Hosting</h3>
354
355  <p>The FCM team will organise the hosting of your Trac environment. It will be set
356  up on the same server that hosts your Subversion repository and access will
357  be provided via a web server. The FCM team will advise you of the URL, and put
358  in place the backup procedures.</p>
359
360  <h2 id="fcm-keywords">FCM Keywords</h2>
361
362  <p>When you set up a repository for a new project, you will normally want the
363  FCM team to set up a URL keyword for it in the FCM central configuration
364  file. The name of the project should be a short string containing only word
365  characters.</p>
366
367  <p>Individual projects can store revision keywords using the Subversion
368  property <code>fcm:revision</code> at registered URLs. Using the UM as an
369  example: if <samp>UM</samp> is a registered URL keyword, you can add the
370  <code>fcm:revision</code> property at the head of the UM project by doing a
371  non-recursive checkout. E.g.:</p>
372  <pre>
373(prompt)$ fcm co -q -N fcm:um um
374(prompt)$ fcm pe fcm:revision um
375</pre>
376
377  <p>In the editor, add the following and <code>fcm commit</code>:</p>
378  <pre>
379vn6.3 = 402
380vn6.4 = 1396
381vn6.5 = 2599
382vn6.6 = 4913
383vn7.0 = 6163
384</pre>
385
386  <p>In a subsequent invocation of <code>fcm</code>, if a revision keyword is
387  specified for a URL in the UM namespace, the command will attempt to load it
388  from the <code>fcm:revision</code> property at the head of the UM project.
389  Revision keywords can also be defined in the FCM central configuration file
390  if you prefer.</p>
391
392  <p>If the project has an associated Trac browser, you can also declare
393  browser URL mapping in the central configuration file. This allows FCM to
394  associate the Subversion URL with a Trac browser URL. There is an automatic
395  default for mapping URLs hosted by the FCM team at the Met Office. External
396  users of FCM may want to adjust this default for their site.</p>
397
398  <p>To change the default browser URL mapping, you need to make some
399  <code>browser.*[namespace] = value</code> declarations in your site's
400  <samp>$FCM/etc/fcm/keyword.cfg</samp> file. There are 3 components to this
401  declaration: <var>browser.comp-pat</var>, <var>browser.loc-tmpl</var> and
402  <var>browser.rev-tmpl</var>. The <var>browser.comp-pat</var> is a regular
403  expression, which is used to separate the scheme-specific part of a version
404  control system URL into a number of components by capturing its substrings.
405  These components are then used to fill in the numbered fields in the
406  <var>browser.loc-tmpl</var>. The template should have one more field than the
407  number of components captured by <var>browser.comp-pat</var>. The last field
408  is used to place the revision, which is generated via the
409  <var>browser.rev-tmpl</var>. This template should have a single numbered
410  field for filling in the revision number. This is best demonstrated by an
411  example. Consider the declarations:</p>
412  <pre>
413browser.comp-pat = (?msx-i:\A//([^/]+)/(.*)\z)
414browser.loc-tmpl = http://{1}/intertrac/source:{2}{3}
415browser.rev-tmpl = @{1}
416</pre>
417
418  <p>If we have a Subversion URL <samp>svn://repos/path/to/a/file</samp>, the
419  <var>browser.comp-pat</var> will capture the components [<samp>repos</samp>,
420  <samp>path/to/a/file</samp>]. When this is applied to the
421  <code>browser.loc-tmpl</code>, <var>{1}</var> will be translated to
422  <samp>repos</samp> and <var>{2}</var> will be translated to
423  <samp>path/to/a/file</samp>. A revision is not given in this case, and so
424  <var>{3}</var> is inserted with an empty string. The result is
425  <samp>http://repos/intertrac/path/to/a/file</samp>. If the revision is
426  <samp>1357</samp>, the <var>browser.rev-tmpl</var> will be used to translate
427  it to <samp>@1357</samp>, which is then inserted to <var>{3}</var> of the
428  <var>browser.loc-tmpl</var>. The result is therefore
429  <samp>http://repos/intertrac/path/to/a/file@1357</samp>.</p>
430
431  <p>For more information on how to set up the keywords, please refer to
432  <a href="code_management.html#svn_basic_keywords">Repository &amp; Revision
433  Keywords</a> and the <a href="annex_cfg.html">Annex: FCM Configuration
434  File</a>.</p>
435
436  <h2 id="make-cfg">FCM Make Configuration</h2>
437
438  <p>The <code>fcm make</code> command (for invoking the extract and build
439  systems) is very flexibile and can be used in lots of different ways. It is
440  therefore difficult to give specific advice explaining how to configure them.
441  However, based on experience with a number of systems, the following general
442  advice can be offered.</p>
443
444  <ul>
445    <li>Standard FCM Make configuration files should be defined and stored
446    within the repository. Users then include these files into their
447    configurations, before applying their local changes.</li>
448
449    <li>The files should be designed to include one another in a hierarchy. For
450    example, you may have one core file which defines all the repository and
451    source locations plus a series of platform/compiler specific files which
452    include the core file. More complex setups are also possible if you need to
453    cater for other options such as different optimisation levels, 32/64 bit,
454    etc.</li>
455
456    <li>When including other configuration files, always make use of the
457    special <var>$HERE</var> variable (rather than, for instance, referring to
458    a fixed repository location). When your configuration file is parsed, this
459    special variable is normally expanded into the container directory of the
460    current configuration file. This means that the include statements should
461    work correctly whether you are referring to configuration files in the
462    repository trunk, in a branch or in a local working copy.</li>
463
464    <li>Make good use of variables (e.g. <samp>$name_spaces</samp>) to simplify
465    repetitive declarations and make your configuration files easier to
466    maintain.</li>
467
468    <li>Use continuation lines to split long lines and make them easier to
469    read.</li>
470  </ul>
471
472  <p>Probably the best advice is to look at what has already been set up for
473  other systems. The FCM team can advise on the best systems to examine.</p>
474
475  <p>When you create a stable build you should keep a FCM Make configuration
476  file that can reproduce the build. One easy way to do this is to create your
477  build using the standard configuration files and the latest versions of the
478  code.  You can then save the configuration file which is created on
479  success.</p>
480
481  <h2 id="alternate_versions">Maintaining Alternate Versions of Namelists and
482  Data Files</h2>
483
484  <p>Sometimes it is useful to be able to access particular revisions of some
485  directories from a FCM repository without having to go via Subversion.
486  Typical examples are namelist or data files used as inputs to a program. The
487  <code>fcm export-items</code> command is designed to help with this. It can
488  be used to maintain a set of extracted version directories from a FCM
489  repository. The command has the following options:</p>
490
491  <dl>
492    <dt><code>--config-file=PATH</code>, <code>--file=PATH</code>, <code>-f
493    PATH</code></dt>
494
495    <dd>Specifies the path to the configuration file.
496    (default=<samp>$PWD/fcm-export-items.cfg</samp>)</dd>
497
498    <dt><code>--directory=PATH</code>, <code>-C PATH</code></dt>
499
500    <dd>Specifies the path to the destination. (default=<samp>$PWD</samp>)</dd>
501
502    <dt><code>--new</code></dt>
503
504    <dd>Specifies the new mode. In this mode, everything is re-exported.
505    Otherwise, the system runs in incremental mode, in which the version
506    directories are only updated if they do not already exist.</dd>
507  </dl>
508
509  <p>The 1st argument SOURCE should be the URL of a branch in a Subversion
510  repository with the standard FCM layout.</p>
511
512  <p>The configuration file should be in the deprecated FCM 1 configuration
513  format. The label in each entry should be a path relative to the source URL.
514  If the path ends in <samp>*</samp> then the path is expanded recursively and
515  any sub-directories containing regular files are added to the list of relative
516  paths to extract. The value may be empty, or it may be a list of space
517  separated <em>conditions</em>. Each condition is a conditional operator
518  (<code>&gt;</code>, <code>&gt;=</code>, <code>&lt;</code>,
519  <code>&lt;=</code>, <code>==</code> or <code>!=</code>) followed by a
520  revision number. The command uses the revision log to determine the revisions
521  at which the relative path has been updated in the source URL. If these
522  revisions also satisfy the conditions set by the user, they will be
523  considered in the export.</p>
524
525  <p>Example:</p>
526  <pre>
527(SHELL PROMPT)$ cat &gt;fcm-export-items.cfg &lt;&lt;EOF
528namelists/VerNL_AreaDefinition   &gt;1000 !=1234
529namelists/VerNL_GRIBToPPCode     &gt;=600 &lt;3000
530namelists/VerNL_StationList     
531elements/*                       &gt;1000
532EOF
533(SHELL PROMPT)$ fcm export-items fcm:ver_tr
534</pre>
535
536  <p>N.B.</p>
537
538  <ol>
539    <li>Each time a sub-directory is revised, the script assigns a sequential
540    <em>v</em> number for the item. Each <em>v</em> number for a sub-directory,
541    therefore, is associated with a revision number. For each exported
542    revision directory, there is a corresponding <em>v</em> number symbolic
543    link pointing to it.</li>
544
545    <li>The system also creates a symbolic link <samp>latest</samp> to point to
546    the latest exported revision directory.</li>
547  </ol>
548
549  <h2 id="work-practice">Defining Working Practices and Policies</h2>
550
551  <p>Some options on working practices and policies are defined in the chapter
552  on <a href="working_practices.html">Code Management Working Practices</a>.
553  Individual projects should document the approach they have adopted. In
554  addition, each project may also need to define its own working practices and
555  policies to suit its local need. For example each project may need to
556  specify:</p>
557
558  <ul>
559    <li>Whether changes are allowed directly on the trunk or whether branches
560    have to be used in all cases.</li>
561
562    <li>Whether all users are allowed to make changes to the trunk.</li>
563
564    <li>Whether Trac tickets have to be raised for all changes to the
565    trunk.</li>
566
567    <li>Whether Trac tickets should be raised for all support queries or
568    whether a Trac ticket should only be raised once there is an agreed
569    "issue".</li>
570
571    <li>Whether branches should normally be made from the latest code or from a
572    stable release.</li>
573
574    <li>Whether a user is allowed to resolve conflicts directly when merging a
575    branch into the trunk or whether he/she should merge the trunk into the
576    branch and resolve the conflicts in the branch first.</li>
577
578    <li>Whether all code changes to the trunk need to be reviewed.</li>
579
580    <li>What testing is required before changes can be merged to the
581    trunk.</li>
582
583    <li>Whether history entries are maintained in source files or whether
584    individual source files changes need to be described in the Subversion log
585    message.</li>
586
587    <li>Branch deletion policy.</li>
588
589    <li>Whether any files in the project require locking before being
590    changed.</li>
591  </ul>
592
593  </div>
594  </div>
595  </div>
596
597  <hr/>
598  <div class="container-fluid text-center">
599    <div class="row"><div class="col-md-12">
600    <address><small>
601      Copyright &copy; 2006-2021 British Crown (Met Office) &amp; Contributors.
602      <a href="http://www.metoffice.gov.uk">Met Office</a>.
603      See <a href="../etc/fcm-terms-of-use.html">Terms of Use</a>.<br />
604      This document is released under the British <a href=
605      "http://www.nationalarchives.gov.uk/doc/open-government-licence/" rel=
606      "license">Open Government Licence</a>.<br />
607    </small></address>
608    </div></div>
609  </div>
610
611  <script type="text/javascript" src="../etc/jquery.min.js"></script>
612  <script type="text/javascript" src="../etc/bootstrap/js/bootstrap.min.js"></script>
613  <script type="text/javascript" src="../etc/fcm.js"></script>
614  <script type="text/javascript" src="../etc/fcm-version.js"></script>
615</body>
616</html>
Note: See TracBrowser for help on using the repository browser.