C.3. Upgrade Procedures

This section covers the specific steps involved in running abuild --upgrade-trees to upgrade the Abuild.conf and Abuild.backing files in your tree. Recall that this is only one step of the overall upgrade process, though it is the most significant step.

Note

The abuild --upgrade-trees process will create some new files and will remove or modify some old files, always saving the old versions. When you run abuild --upgrade-trees, it is highly recommend that you capture the output using script or tee so you can see a log of exactly which files were removed, added, and changed by the process.

C.3.1. High-level Summary of Upgrade Process

Here is an outline of the basic process:

  • Change your current directory (cd) to a directory that is above all your build trees and that you wish to use as the new forest root. This should be a common ancestor of all the trees you wish to upgrade.

  • Run abuild --upgrade-trees.

  • Abuild analyze all Abuild.conf files that it finds at or below your starting directory. It will then generate a file called abuild.upgrade-data that you will have to edit. Edit the file as described below.

  • Once you have filled in all required information in abuild.upgrade-data, rerun abuild --upgrade-trees. This time, it will perform the upgrade by rewriting any Abuild.conf or Abuild.backing that needs to be rewritten. The original file will be renamed to Abuild.conf-1_0 or Abuild.backing-1_0. When you are satisfied with the upgrade, you can delete the *-1_0 files, as those files are never used by abuild. You should also be sure to remove deleted files and check in added and modified files with your version control system. Remember that, in addition to modifying files, some files may be added or removed.

Generally, if you have a backing area, you should upgrade the backing area first. If your backing areas are set up such that each tree backs to the corresponding tree in the backing area and if you have not added any new trees in your area, the upgrade of your regular area may work without any intervention, as abuild will use the backing area to figure out tree names for trees that are backed.

Among the most significant changes to abuild for version 1.1 is the requirement that all build trees have names. In order for abuild to upgrade your trees from version 1.0 to version 1.1, it will need to know what name you wish to assign to all your build trees. You will use the abuild.upgrade-data file to provide this information to abuild.

Note that abuild's upgrade process is extremely tolerant of partially upgraded forests. It uses exactly the same logic as abuild's normal build process (it is part of abuild, after all) to internally map a forest consisting of a mixture of 1.0 and 1.1 files into an internal 1.1 structure. The main difference between the upgrade procedure and abuild's normal build process is that, when upgrading, abuild requires you to provide names of previously unnamed trees, while during the build process (in 1.0-compatibility mode only) it will generate a temporary name on the fly. So if a tree already has a name, or if abuild can figure out what its name is from a backing area, it will use that information. Otherwise, it will use the information you supply in abuild.upgrade-data.

Once it has all the required information, abuild will insert the tree-name key into the root Abuild.conf file of every tree, and it will replace any external-dirs keys with tree-deps keys. It will also remove parent-dir keys, replace this with name, upgrade Abuild.backing files including merging tree-level Abuild.backing files into a single forest-level Abuild.backing file, and remove any occurrences of deleted from root Abuild.conf files, moving the information into the deleted-items key of the new forest-wide Abuild.backing file. In addition, if you have any trees that are nested inside your existing trees, abuild will add child-dirs entries to those root items' parent Abuild.conf files to connect them into the forest. (Recall that, in abuild 1.1, nested tree roots are discovered through child-dirs just like any other build items. In 1.0, they were connected into the forest using path names in other trees' external-dirs keys instead.)

During the analysis process, abuild will find all tree roots at or below your starting directory. It will study them, examining any external-dirs or tree-deps keys to figure out which trees refer to which other trees. It will then group trees into separate, independent forests so that it can upgrade each forest separately. The list of forests is generated such that no tree in one forest refers to any tree in another forest through any of its items' Abuild.conf files. In many cases, you will find that there is only one forest. However, if you have self-contained collections of build trees nested within your primary forest, those will be recognized as separate. This could happen for several reasons, including the following:

  • Maybe you used abuild to build some self-contained, third-party software and you kept a copy of the Abuild.conf files.

  • You might have test suites that contain self-contained build trees. This is certainly true of abuild's own source tree which contains numerous self-contained build trees in its own test suite.

  • You may have stray Abuild.conf files that you never actually connected into your regular build trees. You might just be able to delete them as part of the upgrade process.

For each independent forest abuild finds, it will pick a top-level directory for that forest. This will be the lowest directory abuild can find that is a common ancestor of all the trees in the forest. This directory might, in some cases, be the root of one of the trees in the forest. If not, it might be a directory that contains no Abuild.conf file. In that case, abuild will create a Abuild.conf file containing only a child-dirs key whose value is the relative paths to all the root directories of all the build trees in the forest. You may wish to manually edit this file depending on how you intend to organize your forest. In some cases, abuild may include references to trees that are not always present. When this happens, you may wish to add the -optional flag after the directory name in the child-dirs key.

C.3.2. Editing abuild.upgrade-data

This section describes how to edit the abuild.upgrade-data file. Here's a “quick start” for the impatient or those who are already basically familiar with the process:

  • For each directory whose contents you wish to (recursively) ignore (such as nested trees you're not ready to upgrade), place the directory in the [ignored-directories] section. Place one directory per line, and specify directories relative to the one containing the abuild.upgrade-data file.

  • For each remaining detected build tree root below, replace “***” with the name you intend to give the tree.

  • Lather, rinse, repeat.

The abuild.upgrade-data file is a configuration file used to assist abuild --upgrade-trees. Every time abuild --upgrade-trees is run, it will replace this file, so any comments or formatting changes you make will be lost. Any tree names you assign will be preserved even if abuild no longer believes the directory is a tree root, so it is very unlikely that abuild will throw away work you have already done toward editing this file. If you're paranoid, make a backup copy of abuild.upgrade-data before rerunning abuild --upgrade-trees.

The abuild.upgrade-data file consists of sections Each section is opened with a line of the form

[section-name]

where section-name is replaced by one of the valid section names.

There are three sections:

[ignored-directories]

lists directories that will be skipped during the upgrade

[forest]

a repeatable section; one occurs for each group of trees that abuild finds to be in a given forest

[orphan-trees]

an optional section used to hang onto names previously assigned to any trees root at directories that no longer appear in a known forest

If there are directories below the start directory that you wish to ignore during the conversion process, list them in the [ignored-directories] section. Abuild will ignore those directories when looking for Abuild.conf files.

In each forest that abuild discovers, it will require a name for each tree. If the tree already has a name, that name will appear in the file. Otherwise, the place-holder “***” will appear. Your job is to go through and replace all occurrences of “***” with the name you wish to assign to the tree rooted at that directory.

In some cases, there may be a tree that you are not ready to upgrade, perhaps because that tree is still being used by a project that hasn't yet upgraded its version of abuild. In that case, just list the path to the root of the tree in the [ignored-directories] section. You do not need to remove it from the [forest] section in which it appears; abuild will remove it from there automatically next time it writes the file. If you subsequently change your mind and remove the path from [ignored-directories], abuild will move it back to the appropriate [forest] section. This is also where [orphan-trees] comes into play: if you had assigned a name to a tree that ended up later under an ignored directory, that path and assigned name will get moved to [orphan-trees]. if you later remove the ignored directory entry, abuild will move the path back out of [orphan-trees] so you will not lose the name you previously assigned to the tree.

Abuild is able to perform the actual upgrade when all of the following conditions are met:

  • Abuild is able to parse all Abuild.conf files at or below the current directory, excluding any ignored directories, without finding any errors.

  • Every external-dirs entry exists or can be resolved through a backing area. There is one exception, discussed below.

  • No external-dirs entries cross over any symbolic links

  • Every tree root listed in every [forest] section has a name assigned to it.

  • Every tree that abuild finds during its scan as well as every external-dirs entry referenced by those trees that points to a place at or below the start directory appears in a [forest] section and has a name assigned to it.

Abuild is usually able to upgrade forests with backing areas, but it will not do so if any externals resolve to 1.0-style trees in backing areas. (That is, the external doesn't exist relative to the tree that declares it but does exist relative to that tree's backing area.) In that case, you must either upgrade the backing area first (which is the recommended practice) or make the external resolve locally. You can make the external local by just creating a directory and populating it with an Abuild.conf and an Abuild.backing. The reason for this restriction is that abuild will not read the Abuild.backing file of an upgraded tree root that is not at the root of a forest. This means that abuild would no longer be able to resolve the external in the backing area. As discussed, it is best to upgrade your backing area first anyway since upgrades to forests with upgraded backing areas often require no manual intervention.