FCM 1.3 Release Notes

These are the release notes for FCM release 1.3. You can use this release of FCM freely under the terms of the FCM LICENSE, which you should receive with the distribution of this release.

Note that FCM now requires Subversion 1.4.0 or later (previous releases only required Subversion 1.2.0).

FCM is maintained by the FCM team at the Met Office. Please feedback any bug reports or feature requests to us by e-mail.

What's New?

Build and extract:

The syntax for declaring the extract/build destinations are unified. The DEST (or DEST::ROOTDIR) declaration should be used to declare the root of the extract/build destination. The DIR::ROOT declaration is deprecated.
Users can no longer declare sub-directories in a destination. Declarations such as DEST::CFGDIR, DEST::SRCDIR, etc are no longer supported.
In configuration files, words or fields in a label can now be delimited by a slash (/) as well as a double colon (::). To improve readability, the convention is to only use slash as the delimiter when referring to package names.
In a true or false declaration in a configuration file, the system will now accept the following values as false: 0, <empty string>, false, no and off. (It used to accept <any non empty string> for true and 0, or <empty string> for false.)
The build and extract caches are now located separately in .cache/.bld/ and .cache/.ext/ respectively. In most cases they will be upgraded automatically when you perform the next incremental build/extract. However, if you use inherited build/extract, you must upgrade the inherited build/extract before they can be used. To upgrade an inherited extract, issue the following commands:
```
cd /path/to/inherited/extract/
cd .cache/
mkdir .ext/
cp -r -p .config * .ext/
```
To upgrade an inherited build, issue the following commands:
```
cd /path/to/inherited/build/
fcm bld
```
Extra/common stages in the build/extract processes: parse configuration and set up destination.
The build/extract destination (and the remote destination for extract) will now be printed in verbose mode 1 or above.
A new command line option --clean for removing files generated by previous extracts/builds.
An as-parsed configuration file will be generated for each run (if the file differs from the previous run).

Build:

It is now more efficient to make SRC declarations for files instead of the container directories. Container directories can still be declared, but they will be expanded by the configuration file parser into a list of source files within it. In addition, SRC declarations no longer require the specification of package names - if it is a relative path of the src/ sub-directory. If a relative path is specified, it will be assumed a relative path of the src/ sub-directory.
For declarations such as EXCL_DEP, PP, TOOL, if a package name is associated with the declaration, the system will fail if the package is not declared or defined.
The system will now detect changes in EXCL_DEP, INFILE_EXT and OUTFILE_EXT declarations in an incremental build.
It is now possible to make PP declarations down to the file level. (Previously, it could only be done down to the level of the container directories.)
The package name for a file used to be its root name (i.e. its basename without the extension). It is now its basename, (although the system will continue to support declarations down to the file's root name).
The declarations TYPE and DEP can now be used to define the type and dependencies of a source file. (This replaces most functionalities of the package configuration file.)
The build package configuration file is no longer supported.
The system now generates a single Makefile in the root location of the destination. Hard coded directories should appear only once at the top of the Makefile, provided that source files are only located in the src/ sub-directory of the build destination. (A build used to have a top level Makefile, which included a *.mk file for each source directory in the bld/ sub-directory of the destination.)
The fcm_env.ksh file is renamed fcm_env.sh, which should work with the Bourne shell. (fcm_env.ksh is still available via a symbolic link but is deprecated.)
The --archive or -a option will now archive your build directories using the command tar -czf FILE DIR. Consequently, the system will dearchive them using tar -xzf FILE. If you have been using this option in your previous builds, you should extract the archives manually using tar -xf FILE before running fcm build in incremental mode. Remove the old TAR files on success.
The linker command now defaults to the compiler of the language of the program source file. (The default used to be ld.)

Code management commands:

Allow other graphical merge tools to be used in place of xxdiff by modifying bin/fcm_graphic_merge.
fcm commit will issue extra warning when a user attempts to commit to or remove a branch belonging to another user.
The system has been modified to account for the improved support for peg revisions using the svn log command in Subversion 1.4. Unfortunately this means that Subversion 1.4.x clients are now required.
fcm diff now supports the --summarize option which was introduced in Subversion 1.4.
Added alternate branching strategy in the External Distribution & Collaboration for FCM Projects document.
A number of limitations with the fcm mkpatch command have been fixed. It will also use unified diffs where possible in order to reduce the size of the patch and to make it more readable.

Extract:

The ROOTDIR part of the RDEST::ROOTDIR declaration is now optional.
The MIRROR declaration is deprecated, and replaced by the RDEST::MIRROR_CMD declaration.
The build configuration file generated by extract should no longer contain hard coded paths - except for USE declarations and those protected by the BLD prefix.
The VERSION declaration is deprecated, and replaced by the REVISION declaration.
Added a new REVMATCH declaration for the extract configuration file. If you specify a revision (other than HEAD) for a branch, and this revision is not associated with a changeset for this branch, the system will normally inform you of this discrepancy. By setting REVMATCH to "true", however, the discrepancy will cause extract to fail.
Extract used to fail if the same file is modified by two different branches (compared with the base branch). It now attempts to merge the changes using diff3 -E -m. It only fails if there are unresolved conflicts.
Consequently, the OVERRIDE declaration is deprecated, and replaced by the CONFLICT declaration, which can be set to fail, merge (default) or override.

Minor enhancements & Bug Fixes

Build:

If there is no source file to build, report an error at the beginning of the build process.

Code management commands:

Fixed: FCM URL keyword not expanded when it is specified with an equal sign in an option.
Fixed: typo in the output of fcm branch --info.

Extract:

Improved logic for better performance.
Allow mirroring to use rsync with an alternate remote shell.
fcm cmp-ext-cfg: Improved support for InterTrac links.

General:

Fixed: problems parsing Subversion peg revision with the FCM URL keywords.
The general shell used by FCM is changed from /usr/bin/ksh to /bin/sh to improve portability.
Various other very minor enhancements and bug fixes.

Known Issues

The following are known issues with this release of FCM which we plan to address in later releases:

FCM build does not handle changes in an include file correctly in an inherited build if the include file resides in the same directory as the source file including it, and the source file remains unchanged. This is due to the fact that most pre-processor/compiler commands search the directory containing the source file for include files first before they search elsewhere. We are hoping to find a solution to this problem before the next release.

System Requirements

Perl

The core part of FCM is a set of Perl scripts and modules. For the build system to work, you need the following modules installed:

Carp
Cwd
File::Basename
File::Compare
File::Find
File::Path
File::Spec::Functions
File::Spec
FindBin
Getopt::Long
POSIX

The code management commands and extract system need the following additional modules installed:

File::Temp
Getopt::Long
HTTP::Date
XML::DOM

To use the simple GUI for some of the code management commands, you also need the following modules:

Tk::ROText
Tk

At the Met Office we are currently using the complete FCM system with Perl 5.8.x. In addition the build system is being used with Perl 5.6.x.

Subversion

To use the code management commands (and relevant parts of the extract system) you need to have Subversion installed.

FCM makes extensive use of peg revisions in both the code management and extract systems. This requires Subversion 1.4.0.
At the Met Office we are currently using Subversion 1.4.3.

Note that the extract system can mirror extracted code to a remote platform for building. Therefore it is only necessary to have Subversion installed on the platform where you do your code development. If you use other platforms purely for building and running then you do not need to have Subversion installed on these platforms.

Trac

The use of Trac is entirely optional (although highly recommended if you are using Subversion).

The --trac and --wiki options to the fcm diff --branch command allow you to view branch differences using Trac. This requires Trac 0.10.
At the Met Office we are currently using Trac 0.10.3.

Other Requirements

The fcm conflicts command requires xxdiff. At the Met Office we are currently using version 3.1. The fcm diff --graphical command also uses xxdiff by default although other graphical diff tools can also be used.

The extract system can use diff3, which is part of GNU diffutils, to merge together changes where the same file is modified by two different branches (compared with the base branch). At the Met Office we are currently using version 2.8.1.

The build system requires GNU make. At the Met Office we are currently using version 3.79.x and 3.80.

Optionally, the build system can use f90aib to generate interface files. However, there is also a built in Perl based interface file generator which is quicker and better in most cases so you are unlikely to need f90aib unless you hit a problem with some particular code.

FCM is intended to run on a Unix/Linux system. It is currently used at the Met Office on Linux (Red Hat Enterprise 2.1 and 4.5) and HP-UX 11.00.

Installation

FCM is distributed in the form of a compressed tar file. Un-pack the tar file into an appropriate location on your system. Then add the bin/ directory into your PATH. Once you have done this you should now have full access to the FCM system, assuming that you have met the requirements described in the previous section.

If you wish to define keywords for your systems you will need to create a file etc/fcm.cfg. An example file, fcm.cfg.eg, is provided which is a copy of the file currently used at the Met Office. For further details please refer to the section FCM keywords in the System Admin chapter of the User Guide.

The doc/ directory contains all the system documentation.

doc/release_notes/ contains these release notes. It also contains the release notes for all previous versions which may be useful if you have skipped any versions.
doc/user_guide/ contains the FCM User Guide.
doc/design/ contains the FCM Detailed Design document (currently in draft form).
doc/standards/ contains the FCM Perl and Fortran coding standards. The Perl standard describes the standards followed by the FCM code. The Fortran standard contains some specific advice on the best way of writing Fortran code for use with FCM as well as more general advice on good practise.
doc/collaboration/ contains the External Distribution & Collaboration for FCM Projects document which discusses how projects configured under FCM can be distributed externally.

The tutorial/ directory contains the files necessary to set up a tutorial repository. This will allow you to follow the tutorial section in the User Guide.

The file tutorial/repos/tutorial.dump should be loaded into an empty repository using the svnadmin load command.
The hook scripts in tutorial/hook/ should then be installed in this repository in order to prevent any commits to the trunk. Note that the configuration file svnperms.conf assumes that the tutorial repository is called tutorial_svn. Please edit this file if you use a different name. You also need to install the Subversion utility svnperms.py in order for this to work.
The repository should be configured to allow users write access. You may find it easiest to simply allow anonymous access.
A Trac system should be configured associated with the Tutorial repository. You then need to allow users write access. You may find it easiest to set up a number of guest accounts for this purpose.

The templates/ directory contains various example scripts which you may find useful. Note that these scripts are all specific to the Met Office and may contain hard coded paths and email addresses. They are provided in the hope that you may find them useful as templates for setting up similar scripts of your own. However, they should only be used after careful review to adapt them to your environment. The contents are as follows:

templates/hook/pre-commit

This script restricts write-access to the repository by checking the following:

It executes the Subversion utility svnperms.py if it, and the associated svnperms.conf file, exist. This utility checks whether the author of the current transaction has enough permission to write to particular paths in the repository.
It checks the disk space required by the current transaction. It fails the commit if it requires more than 5Mb of disk space.

templates/hook/post-commit

A simple post-commit hook script which runs the script post-commit-background in the background.

templates/hook/post-commit-background

This script runs in the background after each commit

It updates a <repos>.latest file with the latest revision number.
It creates a dump of the new revision.
It calls the script background_updates.pl if it exists.

This script is installed as standard in all our repositories.

templates/hook/background_updates.pl

An example of how you may want to set up a background_updates.pl script to perform post-commit tasks for a specific repository. This script uses a lock file to prevent multiple commits in quick succession from causing problems.

templates/hook/pre-revprop-change

A simple pre-revprop-change hook script which runs the script pre-revprop-change.pl.

templates/hook/pre-revprop-change.pl

If a user attempts to modify the log message of a changeset and he/she is not the original author of the changeset, this script will e-mail the original author. You can also set up a watch facility to monitor changes of log messages that affect particular paths in the repository. For further details please refer to the section Watching changes in log messages in the System Admin chapter of the User Guide.

templates/hook/post-revprop-change

A simple post-revprop-change hook script which runs the script post-revprop-change.py.

templates/hook/post-revprop-change.py

This hook script updates the Trac SQLite database following a successful change in the log message.

templates/utils/cron_template.sh

An example of how you might set up a cron job to make use of the <repos>.latest file.

templates/utils/daily_cron

The cron job which we run each night. It verifies and backs up each of our repositories, housekeeps the revision dumps created by post-commit-background and backs up each of our Trac systems. It also handles the distribution of FCM to various platforms at the Met Office.

templates/utils/fcm_add_trac.pl

This script sets up a new Trac system and applies some configuration options which we use by default at the Met Office.

templates/utils/recover_svn.pl

This script allows us to recover all of our Subversion repositories by using the nightly backups and the repository dumps.

FCM 1.3 Release Notes 30 January 2008