FCM Detailed Design > Extract System
Extract System
In this chapter, we shall discuss in detail the design of the extract
system. For information of how to use the extract system, please see: FCM System User Guide > The Extract
System.
The extract system extracts source from different locations in a version
control system (currently only Subversion), combining them with source from
the local file system to give a source tree suitable for feeding into the
build system. The system is written in a set of Perl modules and has a similar
interface to the build system. It shares the same command line interface and
many other utilities with the code management system and the build system.
The following inputs are required by the system:
- the location of the destination.
- the location of the source, e.g a URL/revision to one or more branches
in the version control system and/or paths in the local file system.
- configurations for the build system.
The system provides the following outputs:
- a source tree suitable for feeding into the build system.
- a configuration file for the build system.
- an expanded version of the current configuration file, so that the
current extract can be inherited by a subsequent extract.
To do its job, the extract system executes the following tasks in
order:
- Parse the extract configuration file. There are 2 stages in this
process. The file is first read by a primary parser, which process the lines
into label/value pairs. The label/value pairs are then parsed in groups
(according to the labels) by the extract system, which uses the declaration
values to initiate its variables. Inherited extract configuration files are
parsed in the same way.
- Check the destination and the location of any inherited extracts to
ensure that they do not contain any lock files. For inherited extracts,
ensure that their destinations exist. For a full extract, remove any items
created by previous extracts. In any case, set up the destination if it does
not exist.
- Expand declarations for URL keywords and revision keywords
- Expand declarations for expandable source directories using svn ls
-R. Discard any directories with no source files in them.
- Establish the extract priority when dealing with multiple branches. For
each directory containing source files, generate a table of specified
revision of each branch against the last commit revision of the directory.
Discard a directory in a branch if it is unchanged against that of the
base. For each directory, the extract priority is established by putting the
directory of the base branch at the bottom of the list, followed by the
remaining directories for the branches in order of when they are declared.
Directories from the local file system always takes precedence, and so are
placed at the top of the list.
- Export each directory in the above list from the version control system
into the cache.
- For each directory, compare the files in each branch according to the
priority list. Discard a file in a branch if it has the same content as
that of the base. If a file is changed by 2 or more branches, fail the
extract unless override mode is specified. In which case, issue a warning
and the file in the last declared branch is used.
- Copy the file from the cache to the destination if the file does not
exist in the destination or if its content differs. Remove a file in the
destination if it does not exist in any branch of the cache.
- Write an expanded extract configuration file.
- Write of a build configuration file.
- Mirror the source and configuration files to an alternate location, if
necessary.