22.5. Dependency on a Make Variable

In the previous example, we showed how to create a code generator that generates code from a file. This works nicely because make's dependency system is based on file modification times. Sometimes, you may want to generate code based on the value of a make variable whose origin may be either Abuild.mk or, more likely, Abuild.interface. Doing this is more difficult because it requires some obscure make coding, but it is common enough to warrant an example.

The “obvious” way to pass information from a make variable into your code would be to use a preprocessor symbol based on the variable and to pass this symbol to the code with XCPPFLAGS or XCPPFLAGS_Filename. The problem with this is that there is no dependency tracking on variable values, so if you change the variable value, there is nothing that will trigger recompilation of the files that use the preprocessor symbol. To get around this problem, we use local rules to generate a file with the value of the variable. This example can be found in doc/example/auto-from-variable.

First, look at the file-provider build item in library. This build item automatically generates a header file based on the value of a make variable. The variable itself is defined in the Abuild.interface file:

auto-from-variable/library/Abuild.interface

# Add $(ABUILD_OUTPUT_DIR) to includes since that's where the
# generated header is located.
INCLUDES = . $(ABUILD_OUTPUT_DIR)
LIBDIRS = $(ABUILD_OUTPUT_DIR)
LIBS = file-provider

# Provide a variable for the location of the file that we are
# providing
declare file-provider-filename filename
file-provider-filename = interesting-file

We define the variable file-provider-filename to point to a local file. By making this a filename variable, we tell abuild to translate its location to the path to this file as resolved relative to the Abuild.interface file's directory. Note that we use the build item name in the variable name to reduce the likelihood of clashing with other interface variables. In the Abuild.mk file we use the LOCAL_RULES variable to declare the local rules file generate.mk. This is where we will actually generate the header file. Otherwise, this is an ordinary Abuild.mk:

auto-from-variable/library/Abuild.mk

TARGETS_lib := file-provider
SRCS_lib_file-provider := FileProvider.cc
RULES := ccxx
LOCAL_RULES := generate.mk

Here is generate.mk:

auto-from-variable/library/generate.mk

# Write the value to a temporary file and replace the real file if the
# value has changed or the real file doesn't exist.
DUMMY := $(shell echo > variable-value.tmp $(file-provider-filename))
DUMMY := $(shell diff >/dev/null 2>&1 variable-value.tmp variable-value || \
           mv variable-value.tmp variable-value)

# Write the header file based on the variable value.  We can just use
# the variable directly here instead of catting the "variable-value"
# file since we know that the contents of the file always match the
# variable name.

abs_filename := $(abspath $(file-provider-filename))
# If this is cygwin supporting Windows, we need to convert this into a
# Windows path.  Convert \ to / as well to avoid quoting issues.
ifeq ($(ABUILD_PLATFORM_TOOLSET),nt5-cygwin)
 abs_filename := $(subst \,/,$(shell cygpath -w $(abs_filename)))
endif

FileProvider_file.hh: variable-value
        echo '#ifndef __FILEPROVIDER_FILE_HH__' > $@
        echo '#define __FILEPROVIDER_FILE_HH__' >> $@
        echo '#define FILE_LOCATION "$(abs_filename)"' >> $@
        echo '#endif' >> $@

# Make sure our automatically generated file gets generated before we
# compile FileProvider.cc.  Unfortunately, the only way to do this
# that will work reliably in a parallel build is to create an explicit
# dependency.  We use the LOBJ variable to get the object file suffix
# because FileProvider.cc is part of a library.  One way to avoid this
# issue entirely would be to automatically generate a source file
# instead of a header file, but as it is often more convenient to
# generate a header file, we illustrate how to do so in this example.
FileProvider.$(LOBJ): FileProvider_file.hh

There is a lot going on here, so we'll go through line by line. GNU Make is essentially a functional programming environment. Makefiles are not executed sequentially; they are evaluated based on dependencies instead. Sometimes you need to force make to run steps sequentially. You can trick make into doing this by making the operations side effects of a variable assignment using the := operator since these are evaluated when they are read. Our goal here is to translate a variable value, which can't be tracked by the dependency system, into a file modification time, which can. To do this, we create a file whose value and modification time get updated whenever the variable value changes. We do this in two steps: first, we write the value of the variable to a temporary file (the first DUMMY assignment), and then we overwrite another file with the temporary file if the other file either doesn't exist or has a different value (the second assignment). In this way, whenever the variable changes, the file called variable-value gets updated. Although the variable-value.tmp file gets updated every time when run abuild, we don't care since that file is not used as a dependency. Next, we provide the rules to actually generate the header file. The header file depends on the file variable-value so it will get regenerated whenever the variable changes. Here we just use echo to write the header file. Note that we have to call make's abspath function to translate the value of file-provider-filename to an absolute path. This is because abuild writes filename variables as relative paths when it passes them to make. Note also that didn't actually have to use the value of the variable-value file. We know that its contents are identical to the value of the variable, so we can just use the variable's value directly. Finally, we want to make sure that FileProvider_file.hh exists before we start compiling any of the files that reference it. We have a little bit of a bootstrapping problem here: although abuild ordinarily generates dependency information of object files on header files automatically, this generation step is performed during the compilation itself. In order to force the header file to be generated before the compile starts, we have to create an explicit dependency. We do this by creating an explicit dependency from the object file to the header file. Notice that we use the make variable LOBJ to get the object file suffix rather than hard-coding it. All compiler support files are required to set the variable LOBJ to the suffix of object files that are going into libraries and OBJ for object files that are not going into libraries. Although they are often the same, they don't have to be. [46]

We have two files that use the header file. The first one is the library implementation itself:

auto-from-variable/library/FileProvider.cc

#include <FileProvider.hh>
#include <FileProvider_file.hh>
#include <fstream>
#include <iostream>
#include <stdlib.h>

FileProvider::FileProvider() :
    filename(FILE_LOCATION)
{
}

void
FileProvider::showFileContents() const
{
    std::ifstream in(this->filename);
    if (! in.is_open())
    {
        std::cerr << "Can't open file " << this->filename << std::endl;
        exit(2);
    }
    char c;
    while (in.get(c))
    {
        std::cout << c;
    }
}

The other is the main program from the other build item:

auto-from-variable/program/main.cc

#include <FileProvider.hh>
#include <FileProvider_file.hh>
#include <iostream>

int main()
{
    FileProvider fp;
    std::cout << "Showing contents of " << FILE_LOCATION << ":" << std::endl;
    fp.showFileContents();
    return 0;
}

There are a few additional points to be made about this example. We have taken an approach here that can be tailored for a wide variety of situations. In this example, the interface variable is accessible to other build items. If we didn't want this to be the case, we could have used an Abuild.mk variable instead or we could have made this variable visible conditionally upon an interface flag. We have also made the header file available to other build items by adding the output directory to INCLUDES in Abuild.interface. If you didn't want these to have such high visibility, you could protect them just as you would protect any private interfaces. In other words, this example is a little bit of an overkill for the exact case that it implements, but it shows a pattern that can be used when this type of functionality is required. The main thing to take away here is the use of a make trick to translate a variable value into a file modification time, thus making it trackable with make's ordinarily dependency tracking mechanism.



[46] It would be nice to be able to avoid this issue entirely. One way to avoid it would be generate a source file instead of a header file. In that case, make would definitely try to generate the source file before building, so no explicit dependency would be required. This approach would certainly work for this example. One option that would definitely not work would be to create a generate target, analogous to the generate target in abuild's Groovy/ant support, and making it a prerequisite for the all target. Although this would work for strictly serial builds, it wouldn't necessarily work for parallel builds as make is free to build all the prerequisites for a given target in any order as long as they don't have dependencies on each other. In fact, the reason this trick works in Groovy is that the Groovy framework never runs targets in parallel, and ant only runs tasks within a target in parallel when you explicitly tell it that it can. So the bottom line is that whatever we are automatically generating, at the file level, must appear as a dependency somewhere. Source files automatically appear as dependencies of their object files, but header files don't appear as dependencies anywhere until the compile has already been run at least one time. Therefore, a solution that works for parallel builds and generates header files has to create an explicit dependency such as in this example.