Src/config plan for installed software

Note: This is a very preliminary document; suggestions are welcome and appreciated on all points discussed.

Goals

The goals of this Src/configuration system are to:

• allow us to determine precisely what source pool was used to build any of the installed software. This includes information about both:
  • where the sources originally came from, and
  • what local modifications were made to the sources.

• allow us to determine precisely what binaries are installed. This includes information such as:
  • the set of configuration options used in compiling the software
  • where the binaries came from, if not built locally.
  • any changes made to binaries as installed (such as binary edits)

Assumptions

This document assumes that software is already being installed according to the recommendations made in the /s document. When reference is made to "the package's directory", we're referring to the directory /s/package-name.

Plan of Action

The set of goals above will be discussed one-by-one, with a suggested plan of action for each.

Determining where sources originally came from

• A URL pointing to the original sources for the package should be placed in a well-known canonical place. I recommend that a file named ChangeLog be created in each package directory for this purpose, and contain such a pointer. (The ChangeLog file will also contain other information, which is detailed below.)

  Some other name for the URL-containing file is fine; I first considered a file (called URL or ANCESTRY) containing only this pointer, but saw no advantage over combining it with other information in the ChangeLog.

  The primary distribution site is the preferred one to use as the URL for the package; by "primary", I mean the site at which a new version of the software package will normally be released. This will simplify in the future the task of checking whether a new version of a package is available.

Determining what local modifications have been made to sources

It is very useful for local modification information be available on two different levels of granularity. For example, sometimes a simple "executive summary" type of information is desirable: a couple of sentences describing what's changed. It's also important that a set of local patches be simple to obtain. These two different needs are handled by two different solutions:

• Executive summary:

  Each package should have a ChangeLog type of file; this should detail any changes that are made to a package at a relatively high level of abstraction. The sorts of messages that are appropriate are along the same lines as what would be reported in a weekly staff meeting.

  Here's a sample of what a ChangeLog file might contain:

  
  Wed May 24 10:00:09 1995  Jon Cargille  
  
  	* Applied patches found on site <URL> rebuilt for all
  	  architectures
  
  Mon May 22 11:34:55 1995  Jon Cargille  
  
  	* Fetched version XX.YY.ZZ from <URL>
            built for all supported architectures
  

• Use CVS to track details of local modifications:

  There are several reasons to choose CVS over RCS; the most obvious such is that CVS is much better at handling software trees, as opposed to single directories of sources. This translates into a higher probability that people will correctly save away an entire unmodified copy of a package.

  CVS should be used in the following way:

  • An unmodified copy of the entire package's distribution tree should be stored before any modifications are made. (This corresponds to an import command in CVS-speak.) This guarantees that we can easily see how the package was originally distributed, without having to grab a fresh distribution tar-ball and compare.
  • Each significant change to the package should be saved in the CVS repository (called a commit in CVS lingo). This includes changes to the configuration of the packages, as well as source changes. This guarantees that we can easily generate a set of all the local changes that have been made.
  • Each time a "commit" is done, an entry should also be added to the package's ChangeLog. The same entry can generally be used both for CVS's logs, and the ChangeLog.

  Note that various versions of the same software package will be imported into the CVS tree as different packages. This will cause some extra space usage, but will pay off significantly in the long run. The advantages are:

  • importing a newer packages is much simpler (one command), as opposed to a potential rats-nest of dealing with filenames that have changed, new files, files that have been removed, etc. This is one of the area that CVS is weak in, and by treating each new version as a separate package in the CVS tree, we avoid its area of weakness.
  • removing old versions of sources from the CVS repository becomes possible and easy (one command); if the sources are imported on top of one another (i.e. over-layed), removing old versions could be a nightmare. So in spite of the higher disk-space utilization in the short term, the long-term disk-space utilization should be lower.

Determining what configuration was used in compiling a package

There are a number of common but different configuration methods; each of these will have to be dealt with somewhat differently.

• Packages configured with "configure" (generated by GNU autoconf)

  GNU configure are probably the easiest to deal with; if a package configured automatically once, it is likely to do so again. If a non-standard configure command is required (i.e. any command-line options to configure), they should be detailed in the ChangeLog file. If any of the configure-generated scripts are hand-edited, they should be committed to the CVS repository (both unmodified and modified versions) as well as described in the ChangeLog.

• Packages with human-generated configure scripts

  Any package that comes with its own custom configure scripts should be handled in the same way as packages using GNU configure. Any configuration file that is hand-edited should be committed to the repository (both modified and unmodified versions), and described in the ChangeLog. Any command-line options that are passed to the configuration scripts should be described in the ChangeLog.

  If a package requires interactive input during the configuration process (such as prompting for the desired locations of various files, as news software does), a typescript file should be generated of the configuration process using the script command. This should be committed to the CVS repository as well.

• Packages with no auto-configuration

  Packages that must be configured by hand-editing the source files (i.e. "config.h") and Makefiles should just have their modified Makefiles and source files committed to the CVS repository.

Can anyone point out common configuration methods that I've missed?

Are there packages for which the above would not capture enough information to repeat the build? You guys have more experience installing packages than I, so I'd be happy to have some feedback on this...

Determining where binaries came from, if not built locally

• A URL for the original binaries on the distribution site should be saved, in the same way a URL for the original sources would be handled.

Determining what changes have been made to binaries as installed

• Changes to a binary should be detailed in the ChangeLog file.

Using CVS for source control

Creating the CVSROOT

Each package will have its own CVSROOT area, located in /s/pkgname/src/CVSROOT. The CVSROOT can be created and initialized with the following command:

> setenv CVSROOT /s/pkgname/src/CVSROOT
> cd /s/cvs/src/cvs-1.3
> ./cvsinit

Importing the distribution

It is important that each package's original distribution be imported into CVS, so that we can track all modifications that are made locally to each package.

The initial distribution should be imported immediately after the distribution is extracted, as demonstrated by the following example:

> echo $CVSROOT
/s/pkgname-x.y.z/src/CVSROOT
> cd /s/pkgname-x.y.z/src
> tar xzvf pkgname-x.y.z.tgz
> cd pkgname-x.y.z
> cvs import -I ! -m "pkgname x.y.z distribution" pkgname-x.y.z VENDOR PKGNAME_X_Y_Z

Thus, the import line I used to import GNU-make was:

> cvs import -I ! -m "GNU-make 3.74 distribution" make-3.74 GNU GNUMAKE_3_74

where:
	make-3.74	was the directory that I wanted to import
	GNU		was the vendor
	GNUMAKE_3_74	was the version tag

Note that the "import" command does update the imported tree at all. In other words, the equivalent of the "checkout" is not performed. So the first thing you want to do after the import is blow away the tree, and extract the version you just imported, so that additions and changes can be tracked.

So, to continue the above example:

> cd ..
> \rm -rf pkgname-x.y.z
> cvs co pkgname-x.y.z