<!--  vim: set sw=2 sts=2 et ft=docbk:

  Part of the A-A-P recipe executive: Dependencies, Rules and Actions

  Copyright (C) 2002-2003 Stichting NLnet Labs
  Permission to copy and use this file is specified in the file COPYING.
  If this file is missing you can find it here: http://www.a-a-p.org/COPYING

-->

<bridgehead>Build Commands</bridgehead>

<para>
There are several methods to specify build commands to update a target:
<orderedlist>
<listitem><para>
A dependency</para><para>
This is more or less the same as how this is used in a Makefile:
One or more targets, a colon and any number of sources.
This specifies that the target(s) depends on the source(s).
When build commands are given these are the commands to build the
target(s) from the source(s).
Without build commands the dependency is only used to check if the target is
outdated and needs to be build.
</para></listitem>
<listitem><para>
A rule</para><para>
Specified with a <literal>:rule</literal> command.  A "%" in the target(s) and
source(s) stands for any string.  This is used to specify a dependency that is
to be used for files that match the pattern.
</para></listitem>
<listitem><para>
An action</para><para>
Specified with a <literal>:action</literal> command.
Unlike dependencies and rules an action does not specify a build dependency.
It must be invoked by other build commands with the <literal>:do</literal>
command.
</para></listitem>
</orderedlist>
</para>

<para>
Nearly all recipe commands can be used in the build commands.  But these are
not allowed, they can only be used at the recipe level:
<simplelist>
        <member> a dependency specification </member>
        <member> <literal>:rule</literal> </member>
        <member> <literal>:route</literal> </member>
        <member> <literal>:totype</literal> </member>
        <member> <literal>:clearrules</literal> </member>
        <member> <literal>:delrule</literal> </member>
        <member> <literal>:program</literal> </member>
        <member> <literal>:dll</literal> </member>
        <member> <literal>:lib</literal> </member>
        <member> <literal>:recipe</literal> </member>
        <member> <literal>:variant</literal> </member>
</simplelist>
In short: all commands that define dependencies cannot be used in build
commands.  But don't forget you can use <literal>:execute</literal> to do just about
anything.
</para>

<bridgehead id="user-produce">The Production Commands</bridgehead>

<para>
The commands
<link linkend="cmd-program">:program</link>,
<link linkend="cmd-lib">:lib</link>,
<link linkend="cmd-dll">:dll</link>
and
<link linkend="cmd-ltlib">:ltlib</link>
are called <emphasis>production commands</emphasis>
because they explicitly state what things &Aap; should
produce and what
sources are involved.
Everything the production commands can do,
can be done by hand with dependencies as well,
but the automation the production commands provide
is quite useful.
This section discusses how the production commands can
be used and the variables that affect them.
</para>

<para>
The form of each of the production commands is
<literal>:<replaceable>command</replaceable>
<replaceable>targets</replaceable> :
<replaceable>sources</replaceable></literal>.
It is unusual to have more than one target,
since both targets would be built from the
same sources,
but it is allowed.
The list of sources should list the actual, original
sources, i.e.&nbsp;only files that are actually written
by the programmer and that exist on disk.
It is these sources that will be packaged together
for distributing the program or library in source form.
</para>

<para>
Each production command transforms all of the
sources into objects using <!-- how to markup? -->compile
actions. The sources are transformed into object
files of a particular type &mdash;
e.g.&nbsp;libraries use files with type "libobject".
Once all of the sources have been compiled,
a build action is invoked to turn the object files
into the target.
The table below lists the production commands
and the actions used.
</para>

<para>
Some of the production commands can use different
programs to produce the final product, depending on
settings in the recipe.
In particular, you may need to chose to
link a program with the compiler or
through libtool, depending on whether your program
links to any libtool libraries or not.
The alternatives are listed in the table below as well.
To select an alternative form to build the final product,
set the filetype of the target to a specific value,
e.g.&nbsp;
<programlisting>
    :program myProgram { filetype=ltprogram } : source.c
</programlisting>
This example uses the <literal>ltprogram</literal>
alternative build command to build the program "myProgram."
</para>

<informaltable frame="none">
<!-- title>Actions and Variables for Production Commands</title -->
<tgroup cols="4">
  <thead>
  <row>
    <entry>Command</entry>
    <entry>Object Type</entry>
    <entry>Build Command</entry>
    <entry>Build Alternatives</entry>
  </row>
  </thead>
  <tbody>
  <row>
    <entry>:program</entry>
    <entry>object</entry>
    <entry>build</entry>
    <entry><variablelist>
    <varlistentry><term>(normal)</term>
      <listitem><para>Uses the C compiler to link all the
      objects into a program. Uses $LIBS and $LDFLAGS.</para>
      </listitem>
    </varlistentry>
    <varlistentry><term><literal>ltprogram</literal></term>
      <listitem><para>
      Uses libtool to link all the objects into a program.
      Uses $LIBS and $LDFLAGS, but also adds
      $LTLIBS and $LT_RPATH if defined.
      </para>
      </listitem>
    </varlistentry>
    </variablelist></entry>
  </row>
  <row>
    <entry>:lib</entry>
    <entry>libobject</entry>
    <entry>buildlib</entry>
    <entry>
    (normal) Uses the <command>ar</command> utility
    to link all the
    objects into a static library.
    Uses $ARFLAGS.
    </entry>
  </row>
  <row>
    <entry>:dll</entry>
    <entry>dllobject</entry>
    <entry>builddll</entry>
    <entry>
    (normal) Uses the C compiler to link the
    objects into a dynamic (shared) library.
    The object files are different from regular
    library objects, and use a different extension.
    Uses $SHLINK, and $LDFLAGS, as well as
    $SHLINKFLAGS.
    </entry>
  </row>
  <row>
    <entry>:ltlib</entry>
    <entry>ltobject</entry>
    <entry>buildltlib</entry>
    <entry>(normal)
    Uses the <command>libtool</command> utility
    to link the objects together.
    Uses $LDFLAGS.
    <!-- default.aap contains a partly wrong invocation.
         Needs to use LTLIBS and LIBS as well. -->
    </entry>
  </row>
  </tbody>
</tgroup>
</informaltable>

<para>
  In case you do want to have &Aap; figure out how to turn source files in to
  objects and then combine them into a target, but the target is not one of
  the types mentioned above, you can use the
  <link linkend="cmd-produce">:produce</link> command.
</para>

<bridgehead>Attributes for the Production Commands</bridgehead>

<para>
The production commands understand  a wide variety of attributes.
Let us return to the generic form
of a production command:
<programlisting>
    <literal>:<replaceable>command</replaceable> <replaceable>targets</replaceable> : <replaceable>sources</replaceable></literal>
</programlisting>
There are four places attributes can be inserted in this
command, as follows (we have split the command across several
lines for clarity):
<programlisting>
1   <literal>:<replaceable>command</replaceable> { <replaceable>command-attributes</replaceable> }
2       <replaceable>targets</replaceable> { <replaceable>per-target-attributes</replaceable> } : 
3       { <replaceable>source-global-attributes</replaceable> }
4       <replaceable>sources</replaceable> { <replaceable>per-source-attributes</replaceable> } </literal>
</programlisting>
</para>

<para>
There is one commonly-used command-attribute:
<literal>installvar</literal>.
The production commands add their targets to
the variable named in this attribute.
This defaults to the "normal" variable, as listed in 
<xref linkend="installvar"/>.
Assigning an empty value, through <literal>{ installvar = }</literal>,
prevents a target from being installed at all.
This is useful for internal helper programs and
libraries used during the build process.
</para>

<note>
You should use <literal>{ installvar = INSTALL_LTLIB }</literal>
for program targets that have filetype ltprogram,
since they need to be installed as if they are
libtool libraries, not programs.
</note>

<warning>
It is a bad idea (excepting ltprograms, which belong
in INSTALL_LTLIB) to add targets
to the wrong install variable, since the install
action that gets called for it will be wrong then as well.
</warning>

<para>
A slightly less-used command-attribute is
<literal>objecttype</literal>, which changes
the object file type from the default,
(<literal>dllobject</literal>, for instance,
for shared libraries), to something else.
For programs that need to be linked by libtool,
you may also want to force the object files used in the
program to be compiled with libtool, since mixing
non-libtool objects and libtool libraries can
cause problems.
For this, use the <literal>{ objecttype = ltobject }</literal>
as well, so that libtool programs will usually
have build commands like:
<programlisting>
:program { filetype = ltprogram } { objecttype = ltobject } ...
</programlisting>
</para>


<para>
The attributes assigned in the per-target-attributes
are used in the build and install actions of the target.
Typical attributes assigned here are
<literal>installdir</literal>
and <literal>keepdir</literal>.
Variables that affect the build step can be assigned
too, such as
<literal>var_LIBS</literal>
and <literal>var_LDFLAGS</literal>.
</para>


<para>
The attributes for sources are used for the compile steps
of the build process, and useful attributes
here are <literal>var_INCLUDE</literal>
(if one source file needs special include files)
and
<literal>filetype</literal>.
The attributes in the 
source-global-attributes position apply to <emphasis>all</emphasis>
the sources in the list, and 
per-source-attributes apply only to the source file
immediately preceding the attribute.
</para>

<para>
An example that uses all of these settings is:
<programlisting>
1 :ltlib { installvar = } conduit_knotes.la
2     { add_LIBS = -lkdeui } :
3     { add_INCLUDE = -I$BDIR/knotes }
4     knotes/KNotesIface.h { filetype=stub } { var_LTOBJSUF=_stub.lo }
5     knotes/knotes-factory.cc
</programlisting>
Here we see a libtool library that is not installed (line 1),
which must be linked with an additional library (line 2).
All of the sources are compiled with an extra include directory (line 3).
The first source file (line 4) has additional complications and
uses a different compile action due to its filetype.
The last source file (line 5) is compiled
with normal flags extended only by the source-global-attribute
on line 3.
</para>


<bridgehead>Rules And Dependencies</bridgehead>

<para>
  When a target is to be build &Aap; first searches for an explicit dependency
  with build commands that produces the target.  This dependency may come from
  a high level build command such as
  <link linkend="cmd-program">:program</link>.  When such a dependency is
  not found then the rules defined with
  <link linkend="cmd-rule">:rule</link> are checked:
<orderedlist>
<listitem><para>
All the matching rules without commands are used, but only if the source
already exists.  Thus this cannot be used to depend on a file that is still 
to be created.
</para></listitem>
<listitem><para>
One rule with commands will be selected, in this order of preference:
<simplelist>
  <member>A rule for which the sources exist.</member>
  <member>A rule for which one of the sources does not exist and was not
    defined with the {sourceexists} option.</member>
</simplelist>
   If there are multiple matches, the rule with the longest pattern is used.
   Thus if you have these two rules:
   <programlisting>
        :rule test/%.html : test/%.in
              :do something
        :rule %.html : %.in
              :do something-else
</programlisting>
   The first one will be used for a file "test/foo.html", the second one for a
   file "./foo.html".
   If there are two with an equally long pattern, this is an error.
</para></listitem>
</orderedlist>

TRICK: When the source and target in a rule are equal, it is skipped.  This
avoids that a rule like this becomes cyclic:
   <programlisting>
        :rule %.jpg : path/%.jpg
                :copy $source $target
</programlisting>
</para>

<bridgehead id="build-sections">Command block sections</bridgehead>

<para>
  Sometimes it is useful to execute commands when a target does NOT require
  updating.  For example, to give a message.  And sometimes commands need to
  be executed no matter if the target is outdated.  For example to add an
  attribute to the target.
</para>

<para>
  You can add sections to build commands of rules and dependencies.  Three
  kinds of sections are possible:
<simplelist>
  <member>&gt;always - always executed.</member>
  <member>&gt;build - executed when building the target.</member>
  <member>&gt;nobuild - executed when NOT building the target.</member>
</simplelist>
The sections can be used in arbitrary order and may appear multiple times.
All section headers must have the same amount of indent.
The commands in the sections must have more indent than the section headers.
</para>
<para>
  Example:
</para>
   <programlisting>
        foo.out : foo.in
            &gt;always
                # Always attach an attribute to the target
                :attr {output = yes} $target
            &gt;build
                # Only when $target requires updating
                  :copy $source $target
            &gt;nobuild
                # Only when $target does not require updating
               :print $target is up-to-date
</programlisting>

<bridgehead>Multiple targets</bridgehead>

<para>
When a dependency with build commands has more than one target, this means
that the build commands will produce all these targets.  This makes it
possible to specify build commands that produce several files at the same
time.  Here is an example that compiles a file and at the same time produces a
documentation file:
</para>

<programlisting>
   foo.o foo.html : foo.src
        :sys srcit $source -o $(target[0]) --html $(target[1])
</programlisting>

<para>
People used to "make" must be careful, they might expect the build commands to
be executed once for each target.  &Aap; doesn't work that way, because the
above example would be impossible.  To run commands on each target this must
be explicitly specified.  Example:
</para>

<programlisting>
   dir1 dir2 dir3 :
        @for item in target_list:
             :mkdir $item
</programlisting>

<para>
The variable "target_list" is a Python list of the target items. Another such
variable is "source_list", it is the list of source files (this excludes
virtual items; "depend_list" also has the virtual items).  An extreme example
of executing build commands for each combination of sources and targets:
</para>

<programlisting>
   $OutputFiles : $InputFiles
        @for trg in target_list:
            :print start of file  &gt;! $trg
            @for src in source_list:
                :sys foofilter -D$trg $src &gt;&gt; $trg
</programlisting>

<para>
When multiple targets are used and there are no build commands, this works as
if each target depends on the list of sources.  Thus this dependency:
<programlisting>
        t1 t2 : s1 s2 s3
</programlisting>
Is equivalent to:
<programlisting>
        t1 : s1 s2 s3
        t2 : s1 s2 s3
</programlisting>
Thus when t1 is outdated to s1, s2 or s3, this has no consequence for t2.
</para>


<bridgehead>Automatic dependency checking</bridgehead>

<para>
When a source file includes other files, the targets that depend on the source
file also depend on the included files.  Thus when "foo.c" includes "foo.h"
and "foo.h" is changed, the build commands to produce "foo.o" from "foo.c"
must be executed, even though "foo.c" itself didn't change.
</para>

<para>
&Aap; detects these implied dependencies automatically for the types it knows
about.  Currently that is C and C++.  Either by using gcc or a Python function
the "#include" statements are found in the source code and turned into a
dependency without build commands.
</para>

<para>
  This works recursively.  Thus when "foo.c" includes "foo.h" and "foo.h"
  includes "common.h", the dependency will look like this:
</para>

<programlisting>
        foo.c : foo.h common.h
</programlisting>

<para>
For other types of files than C and C++ you can add your own dependency
checker.  For example, this is how to define a checker for the "tt" filetype:
</para>

<programlisting>
      :action depend tt
          :sys tt_checker $source &gt; $target
</programlisting>

<para>
The "tt_checker" command reads the file "$source" and writes a dependency line
in the file "$target".  This is a dependency like it is used in a recipe.
In a Makefile this has the same syntax, thus tools that produce dependencies
for "make" will work.  Here is an example:
</para>

<programlisting>
   foo.o : foo.tt  foo.hh  include/common.hh
</programlisting>

<para>
This is interpreted as a dependency on "foo.hh" and "include/common.hh".
Note that "foo.o" and "foo.tt" are ignored.  Tools designed for "make" produce
these but they are irrelevant for &Aap;.
</para>

<para>
Since the build commands for ":action depend" are ordinary build commands, you
can use Python commands, system commands or a mix of both to do the dependency
checking.
</para>

<para>
  More about customizing dependency checking in <xref linkend="user-autodep"/>.
</para>


<bridgehead>Attributes Overruling Variables</bridgehead>

<para>
Most variables like $CFLAGS and $BDIR are used for all source files.
Sometimes it is useful to use a different value for a group of files.  This is
done with an attribute that starts with "var_".  What follows is the name of
the variable to be overruled.  Thus attribute "var_XYZ" overrules variable
"XYZ".
</para>

<para>
The overruling is done for:
<simplelist>
<member>dependencies</member>
<member>rules</member>
<member>actions</member>
</simplelist>
</para>

<para>
The attributes of all the sources are used.  In case the same attribute is
used twice, the last one wins.
</para>

<para>
Another method is to use an "add_" attribute.  This works like "var_", but
instead of overruling the variable value it is appended.  This is useful for
variables that are a list of items, such as $DEFINE.  Example:
<programlisting>
              :attr thefile.c {add_DEFINE = -DEXTRA=yes}
</programlisting>
The value of the attribute is only appended when it does not appear yet, to
avoid adding it two or more times.
</para>

<para>
Another method is to define a scope name.  This scope is then used to find
variables before searching other scopes, but after using the local scope.  For
example, to specify that the "s_opt" scope is to be used when compiling
"filter.c":
</para>
<programlisting>
              OPTIMIZE = 0
              DEBUG = yes
              :program myprog : main.c filter.c version.c
              :attr {scope = s_opt} filter.c
              s_opt.OPTIMIZE = 4
              s_opt.DEBUG = no
</programlisting>
<para>
  Note that you can set the values of the variables in the user scope after
  adding the scope attribute to "filter.c".
</para>


<bridgehead>Virtual Targets</bridgehead>

<para>
  A virtual target is a target that is not an actual file.  A Virtual target
  is used to trigger build commands without creating a file with the name of
  the target.  Common virtual targets are "clean", "all", "publish", etc.
</para>

<para>
When a target is virtual it is always built.  &Aap; does not remember if it was
already done a previous time.  However, it is only build once for an invocation of Aap.  Example:
<programlisting>
        clean:
                :del {r}{f} temp/*
</programlisting>
</para>

<para>
To remember the signatures for a virtual target use the "remember" attribute:
<programlisting>
        version {virtual}{remember} : version.txt.in
                :print $Version | :cat - $source >! version.txt
</programlisting>
Now "aap version" will only execute the
<link linkend="cmd-print">:print</link> command if version.txt.in has
changed since the last time this was done.
</para>

<para>
Using {remember} for one of the known virtual targets (e.g., "all" or
"fetch") is unusual, except for "publish".
</para>

<para>
When using {remember} for a virtual target without a dependency, it will only
be built once.  This can be used to remember the date of the first invocation.
<programlisting>
        all: firsttime
        firsttime {virtual}{remember}:
                :print First build on $DATESTR > firstbuild.txt
</programlisting>
The difference with a direct dependency on "firstbuild.txt" is that when this
file is deleted, it won't be built again.
</para>


<bridgehead>Source Path</bridgehead>

<para>
The sources for a dependency are searched for in the directories specified
with $SRCPATH.  The default is ". $BDIR", which means that the sources are
searched for in the current directory and in the build directory.
The current directory usually is the directory in which the recipe is located,
but a <link linkend="cmd-cd">:cd</link> command may change this.
</para>

<para>
The "srcpath" attribute overrules using $SRCPATH for an item.  Example:
</para>
<programlisting>
        :attr bar.c {srcpath = ~/src/lib}
</programlisting>

<para>
To avoid using $SRCPATH for a source, so that it is only found in the current
directory, make the "srcpath" attribute empty:
</para>

<programlisting>
        foo.o : src/foo.c {srcpath=}
</programlisting>

<para>
When setting $SRCPATH to include the value of other variables, you may want to
use "$=", so that the value of the variable is not expanded right away but
when $SRCPATH is used.  This is especially important when appending to
$SRCPATH before a
<link linkend="cmd-variant">:variant</link> command, since it changes $BDIR.
Example:
</para>
<programlisting>
        SRCPATH $+= include
</programlisting>
<para>
Warning: Using the search path means that the first encountered file will be
used.  When old files are lying around the wrong file may be picked up.  Use
the full path to avoid this.
</para>


<bridgehead>Depending On A Directory</bridgehead>

<para>
When a target depends on the existence of a directory, it can be specified
this way:
<programlisting>
        foodir/foo : foodir {directory}
                :print >$target this is foo
</programlisting>
The directory will be created if it doesn't exist.  The normal mode will be
used (0777 with umask applied).  When a different mode is required specify it
with an octal value: {directory = 0700}.  The number must start with a zero.
</para>


<bridgehead>Build Command Signature</bridgehead>

<para>
A special kind of signature is used to check if the build commands have
changed.  An example:
<programlisting>
        foo.o : {buildcheck = $CFLAGS} foo.c
                :sys $CC $CFLAGS -c $source -o $target
</programlisting>
This defines a check for the value of $CFLAGS.  When this value changes, the
target is considered outdated.  When something else in the build command
changes, e.g., $CC, this does not cause the target to become outdated.
</para>
<para>
The default buildcheck is made from the build commands themselves.  This is
with variables expanded before the commands have been executed.  Thus when one
of the commands is ":sys $CC $CFLAGS $source" and $CC or $CFLAGS changes, the
buildcheck signature changes.  The
<link linkend="cmd-do">:do</link> commands are also expanded into the
commands for the action specified.  However, this only works when the action
and filetype can be estimated.  The action must be specified plain, not with a
variable, and the filetype used is the first of:
<orderedlist>
<listitem><para>
a filetype attribute specified after action
</para></listitem>
<listitem><para>
if the first argument doesn't contain a "$", the filetype of this argument
</para></listitem>
<listitem><para>
the filetype of the first source argument of the dependency.
</para></listitem>
</orderedlist>
</para>

<para>
To add something to the default check for the build commands the $commands
variable can be used.  Example:
<programlisting>
        Version = 1.4
        foo.txt : {buildcheck = $commands $Version}
                :del {force} $target
                :print  >$target this is $target
                :print >>$target version number: $Version
</programlisting>
If you now change the value of $Version, change one of the
<link linkend="cmd-print">:print</link> commands
or add one, "foo.txt" will be rebuilt.
</para>

<para>
To simplify this, $xcommands can be used to check the build commands after
expanding variables, thus you don't need to specify $Version:
<programlisting>
        foo.txt : {buildcheck = $xcommands}
</programlisting>
However, this only works when all $VAR in the commands can be expanded and
variables used in Python commands are not expanded.
</para>

<para>
To avoid checking the build commands, use an empty buildcheck.  This is useful
when you only want the target to exist and don't care about the command used
to create it:
<programlisting>
        objects : {buildcheck = }
                :print "empty" > objects
</programlisting>
</para>

<para>
  Sometimes you might change the build commands in a recipe, which would
  normally mean the target should be updated, but you are sure that this isn't
  necessary and want to avoid executing the build commands.  You can tell
  &Aap; to ignore the buildcheck once with the
  <link linkend="option-contents">--contents</link> option.
</para>
