\input texinfo
@c %**start of header
@setfilename R-ints.info
@settitle R Internals
@setchapternewpage on
@c %**end of header

@c @documentencoding ISO-8859-1

@syncodeindex fn vr

@dircategory Programming
@direntry
* R Internals: (R-ints).      R Internals.
@end direntry

@finalout

@include R-defs.texi
@include version.texi

@ifinfo
This is a guide to R's internal structures.

@Rcopyright{1999}

@ignore
Permission is granted to process this file through TeX and print the
results, provided the printed document carries a copying permission
notice identical to this one except for the removal of this paragraph
(this paragraph not being relevant to the printed manual).

@end ignore

@permission{}
@c ---------- ^- read that
@end ifinfo

@titlepage
@title R Internals
@subtitle Version @value{VERSION}
@author R Development Core Team
@page
@vskip 0pt plus 1filll
@permission{}

@Rcopyright{1999}

@value{ISBN-ints}
@end titlepage

@ifnothtml
@contents
@end ifnothtml

@ifnottex
@node Top, R Internal Structures, (dir), (dir)
@top R Internals

This is a guide to the internal structures of @R{} and coding standards for
the core team working on @R{} itself.

The current version of this document is @value{VERSION}.

@value{ISBN-ints}
@end ifnottex

@menu
* R Internal Structures::
* .Internal vs .Primitive::
* R coding standards::
* Testing R code::
* Function and variable index::
* Concept index::
@end menu
@node R Internal Structures, .Internal vs .Primitive, Top, Top
@chapter R Internal Structures

This appendix is the beginnings of documentation about @R{} internal
structures.  It is written for the R core team and others studying the
code in the @file{src/main} directory.

It is a work-in-progress, first begun for @R{} 2.4.0, and should be
checked against the current version of the source code.

@menu
* SEXPs::
* Environments and variable lookup::
* Attributes::
* Contexts::
* Argument evaluation::
* The write barrier::
* Serialization Formats::
* Internal use of global and base environments::
* Modules::
@end menu

@node SEXPs, Environments and variable lookup, R Internal Structures, R Internal Structures
@section SEXPs

@cindex SEXP
@cindex SEXPRREC
What @R{} users think of as @emph{variables} or @emph{objects} are
symbols which are bound to a value.  The value can be thought of as
either a @code{SEXP} (a pointer), or the structure it points to, a
@code{SEXPREC} (and there are alternative forms used for vectors, namely
@code{VECSXP} pointing to @code{VECTOR_SEXPREC} structures).
So the basic building blocks of @R{} objects are often called
@emph{nodes}, meaning @code{SEXPREC}s or @code{VECTOR_SEXPREC}s.

Note that the internal structure of the @code{SEXPREC} is not made
available to R Extensions: rather @code{SEXP} is an opaque pointer, and
the internals can only be accessed by the functions provided.

@cindex node
Both types of node structure have as their first three fields a 32-bit
@code{sxpinfo} header and then three pointers (to the attributes and the
previous and next node in a doubly-linked list), and then some further
fields.  On a 32-bit platform a node@footnote{strictly, a @code{SEXPREC}
node; @code{VECTOR_SEXPREC} nodes are slightly smaller but followed by
data in the node.} occupies 28 bytes: on a 64-bit platform typically 56
bytes (depending on alignment constraints).

The first five bits of the @code{sxpinfo} header specify one of up to 32
@code{SEXPTYPE}s.

@menu
* SEXPTYPEs::
* Rest of header::
* The 'data'::
* Allocation classes::
@end menu

@node SEXPTYPEs, Rest of header, SEXPs, SEXPs
@subsection SEXPTYPEs

@cindex SEXPTYPE
Currently @code{SEXPTYPE}s 0:10 and 13:25 are in use.  Values 11 and 12 were
used for internal factors and ordered factors and have since been
withdrawn.  Note that the @code{SEXPTYPE}s are stored in @code{save}d
objects and that the ordering of the types is used, so the gap cannot
easily be reused.

@cindex SEXPTYPE table
@quotation
@multitable {no} {SPECIALSXPXXX} {S4 classes not of simple type}
@headitem no @tab  SEXPTYPE@tab  Description
@item @code{0}   @tab @code{NILSXP}      @tab @code{NULL}
@item @code{1}   @tab @code{SYMSXP}      @tab symbols
@item @code{2}   @tab @code{LISTSXP}     @tab pairlists
@item @code{3}   @tab @code{CLOSXP}      @tab closures
@item @code{4}   @tab @code{ENVSXP}      @tab environments
@item @code{5}   @tab @code{PROMSXP}     @tab promises
@item @code{6}   @tab @code{LANGSXP}     @tab language objects
@item @code{7}   @tab @code{SPECIALSXP}  @tab special functions
@item @code{8}   @tab @code{BUILTINSXP}  @tab builtin functions
@item @code{9}   @tab @code{CHARSXP}     @tab internal character strings
@item @code{10}   @tab @code{LGLSXP}     @tab logical vectors
@item @code{13}   @tab @code{INTSXP}     @tab integer vectors
@item @code{14}   @tab @code{REALSXP}    @tab numeric vectors
@item @code{15}   @tab @code{CPLXSXP}    @tab complex vectors
@item @code{16}   @tab @code{STRSXP}     @tab character vectors
@item @code{17}   @tab @code{DOTSXP}     @tab dot-dot-dot object
@item @code{18}   @tab @code{ANYSXP}     @tab make ``any'' args work
@item @code{19}   @tab @code{VECSXP}     @tab list (generic vector)
@item @code{20}   @tab @code{EXPRSXP}    @tab expression vector
@item @code{21}   @tab @code{BCODESXP}   @tab byte code
@item @code{22}   @tab @code{EXTPTRSXP}  @tab external pointer
@item @code{23}   @tab @code{WEAKREFSXP} @tab weak reference
@item @code{24}   @tab @code{RAWSXP}     @tab raw vector
@item @code{25}   @tab @code{S4SXP}      @tab S4 classes not of simple type
@end multitable
@end quotation

@cindex atomic vector type
Many of these will be familiar from @R{} level: the atomic vector types
are @code{LGLSXP}, @code{INTSXP}, @code{REALSXP}, @code{CPLXSP},
@code{STRSXP} and @code{RAWSXP}.  Lists are @code{VECSXP} and names
(also known as symbols) are @code{SYMSXP}.  Pairlists (@code{LISTSXP},
the name going back to the origins of @R{} as a Scheme-like language)
are rarely seen at @R{} level, but are for example used for argument
lists.  Character vectors are effectively lists all of whose elements
are @code{CHARSXP}, a type that is rarely visible at @R{} level.

@cindex language object
@cindex argument list
Language objects (@code{LANGSXP}) are calls (including formulae and so
on).  Internally they are pairlists with first element a
reference@footnote{a pointer to a function or a symbol to look up the
function by name, or a language object to be evaluated to give a
function.} to the function to be called with remaining elements the
actual arguments for the call (and with the tags if present giving the
specified argument names).

@cindex expression
Expressions are of type @code{EXPRSXP}: they are a vector of language
objects most often seen as the result of @code{parse()}.

@cindex function
The functions are of types @code{CLOSXP}, @code{SPECIALSXP} and
@code{BUILTINSXP}: where @code{SEXPTYPE}s are stored in an integer
these are sometimes lumped into a pseudo-type @code{FUNSXP} with code
99.  Functions defined via @code{function} are of type @code{CLOSXP} and
have formals, body and environment.

@cindex S4 type
The @code{SEXPTYPE} @code{S4SXP} was introduced in @R{} 2.4.0 for S4
classes which were previously represented as empty lists, that is
objects which do not consist solely of a simple type such as an atomic
vector or function.


@node Rest of header, The 'data', SEXPTYPEs, SEXPs
@subsection Rest of header

The @code{sxpinfo} header is defined as a 32-bit C structure by

@example
struct sxpinfo_struct @{
    SEXPTYPE type      :  5;  /* @r{discussed above} */
    unsigned int obj   :  1;  /* @r{is this an object with a class attribute?} */
    unsigned int named :  2;  /* @r{used to control copying} */
    unsigned int gp    : 16;  /* @r{general purpose, see below} */
    unsigned int mark  :  1;  /* @r{mark object as `in use' in GC} */
    unsigned int debug :  1;
    unsigned int trace :  1;
    unsigned int spare :  1;  /* @r{unused} */
    unsigned int gcgen :  1;  /* @r{generation for GC} */
    unsigned int gccls :  3;  /* @r{class of node for GC} */
@};  /*              Tot: 32 */
@end example

@findex debug bit
The @code{debug} bit is used for closures and environments.  For
closures it is set by @code{debug()} and unset by @code{undebug()}, and
indicates that evaluations of the function should be run under the
browser.  For environments it indicates whether the browsing is in
single-step mode.

@findex trace bit
The @code{trace} bit is used for functions for @code{trace()} and for
other objects when tracing duplications (see @code{tracemem}).

@findex named bit
@findex NAMED
@findex SET_NAMED
@cindex copying semantics
The @code{named} field is set and accessed by the @code{SET_NAMED} and
@code{NAMED} macros, and take values @code{0}, @code{1} and @code{2}.
@R{} has a `call by value' illusion, so an assignment like
@example
b <- a
@end example

@noindent
appears to make a copy of @code{a} and refer to it as @code{b}.
However, if neither @code{a} nor @code{b} are subsequently altered there
is no need to copy.  What really happens is that a new symbol @code{b}
is bound to the same value as @code{a} and the @code{named} field on the
value object is set (in this case to @code{2}).  When an object is about
to be altered, the @code{named} field is consulted.  A value of @code{2}
means that the object must be duplicated before being changed.  (Note
that this does not say that it is necessary to duplicate, only that it
should be duplicated whether necessary or not.)  A value of @code{0}
means that it is known that no other @code{SEXP} shares data with this
object, and so it may safely be altered.  A value of @code{1} is used
for situations like

@example
dim(a) <- c(7, 2)
@end example

@noindent
where in principle two copies of @code{a} exist for the duration of the
computation as (in principle)

@example
a <- `dim<-`(a, c(7, 2))
@end example

@noindent
but for no longer, and so some internal functions can be optimized to
avoid a copy in this case.

The @code{gp} bits are by definition `general purpose'.  As of version
2.4.0 of R, bit 4 (i.e., the fifth bit) is turned on to mark S4 objects.
Bits 0-3 and bits 14-15 have been used previously as described below
(from detective work on the sources).

@findex gp bits
@findex LEVELS
@findex SETLEVELS
The bits can be accessed and set by the @code{LEVELS} and
@code{SETLEVELS} macros, which names appear to date back to the internal
factor and ordered types and are now used in only a few places in the
code.  The @code{gp} field is serialized/unserialized for the
@code{SEXPTYPE}s other than @code{NILSXP}, @code{SYMSXP} and
@code{ENVSXP}.

If we label the bits from 0, bits 14 and 15 of @code{gp} are used for
`fancy bindings'.  Bit 14 is used to lock a binding or an environment,
and bit 15 is used to indicate an active binding.  (For the definition
of an `active binding' see the header comments in file
@file{src/main/envir.c}.)  Bit 15 is used for an environment to indicate
if it participates in the global cache.

Almost all other uses seem to be only of bits 0 and 1, although one
reserves the first four bits.

@findex ARGSUSED
@findex SET_ARGUSED
The macros @code{ARGUSED} and @code{SET_ARGUSED} are used when matching
actual and formal function arguments, and take the values 0, 1 and 2.

@findex MISSING
@findex SET_MISSING
The macros @code{MISSING} and @code{SET_MISSING} are used for pairlists
of arguments.  Four bits are reserved, but only two are used (and
exactly what for is not explained).  It seems that bit 0 is used by
@code{matchArgs} to mark missingness on the returned argument list, and
bit 1 is used to mark the use of a default value for an argument copied
to the evaluation frame of a closure.

@findex DDVAL
@findex SET_DDVAL
@cindex ... argument
Bit 0 is used by macros @code{DDVAL} and @code{SET_DDVAL}.  This
indicates that a @code{SYMSXP} is one of the symbols @code{..n} which
are implicitly created when @code{...} is processed, and so indicates
that it may need to be looked up in a @code{DOTSXP}.

@findex PRSEEN
@cindex promise
Bit 0 is used for @code{PRSEEN}, a flag to indicate if a promise has
already been seen during the evaluation of the promise (and so to avoid
recursive loops).

Bit 0 is used for @code{HASHASH}, on the @code{PRINTNAME} of the
@code{TAG} of the frame of an environment.

Bits 0 and 1 are used for weak references (to indicate 'ready to
finalize', 'finalize on exit').

Bit 0 is used by the condition handling system (on a @code{VECSXP}) to
indicate a calling handler.

@c Finally, @code{SETLEVELS} and @code{LEVELS} are used by that name for
@c the internal code for @code{terms.formula} to compute the @code{order}
@c attribute of the result.  This is computed on an internal pairlist, and
@c marks the order of the interaction.  This is in principle unlimited
@c (although no test is done) and could in principle exceed 15.  (This
@c usage could easily be replaced by one not making use of @code{gp}.)


@node The 'data', Allocation classes, Rest of header, SEXPs
@subsection The `data'

A @code{SEXPREC} is a C structure containing the 32-bit header as
described above, three pointers (to the attributes, previous and next
node) and the node data, a union

@example
union @{
    struct primsxp_struct primsxp;
    struct symsxp_struct symsxp;
    struct listsxp_struct listsxp;
    struct envsxp_struct envsxp;
    struct closxp_struct closxp;
    struct promsxp_struct promsxp;
@} u;
@end example

@noindent
All of these alternatives apart from the first (an @code{int}) are three
pointers, so the union occupies three words.

@cindex vector type
The vector types are @code{RAWSXP}, @code{CHARSXP}, @code{LGLSXP},
@code{INTSXP}, @code{REALSXP}, @code{CPLXSXP}, @code{STRSXP},
@code{VECSXP}, @code{EXPRSXP} and @code{WEAKREFSXP}.  Remember that such
types are a @code{VECTOR_SEXPREC}, which again consists of the header
and the same three pointers, but followed by two integers giving the
length and `true length'@footnote{This is almost unused.  The only
current use is for hash tables of environments (@code{VECSXP}s), where
@code{length} is the size of the table and @code{truelength} is the
number of primary slots in use, and for the reference hash tables in
serialization(@code{VECSXP}s), where @code{truelength} is the number of
slots in use.} of the vector, and then followed by the data (aligned as
required: on most 32-bit systems with a 24-byte @code{VECTOR_SEXPREC}
node the data can follow immediately after the node).  The data is a
block of memory of the appropriate length to store `true length'
elements (rounded up to a multiple of 8 bytes, with the 8-byte blocks
being the `Vcells' referred in the documentation for @code{gc()}).

The `data' for the various types are given in the table below.  A lot of
this is interpretation, i.e. the types are not checked.

@table @code
@item NILSXP

There is only one object of type @code{NILSXP}, @code{R_NilValue}, with
no data.

@item SYMSXP
Pointers to three nodes, the name, value and internal, accessed by
@code{PRINTNAME} (a @code{CHARSXP}), @code{SYMVALUE} and
@code{INTERNAL}.  (If the symbol's value is a @code{.Internal} function,
the last is a pointer to the appropriate @code{SEXPREC}.)  Many symbols
have @code{SYMVALUE} @code{R_UnboundValue}.

@item LISTSXP
Pointers to the CAR, CDR (usually a @code{LISTSXP} or @code{NULL}) and TAG
(usually a @code{SYMSXP}).

@item CLOSXP
Pointers to the formals (a pairlist), the body and the environment.

@item ENVSXP
Pointers to the frame, enclosing environment and hash table (@code{NULL} or a
@code{VECSXP}).  A frame is a tagged pairlist with tag the symbol and
CAR the bound value.

@item PROMSXP
Pointers to the value, expression and environment (in which to evaluate
the expression).  Once an promise has been evaluated, the environment is
set to @code{NULL}.

@item LANGSXP
A special type of @code{LISTSXP} used for function calls.  (The CAR
references the function (perhaps via a symbol or language object), and
the CDR the argument list with tags for named arguments.)  @R{}-level
documentation references to `expressions' / `language objects' are
mainly @code{LANGSXP}s, but can be symbols (@code{SYMSXP}s) or
expression vectors (@code{EXPRSXP}s).

@item SPECIALSXP
@itemx BUILTINSXP
An integer giving the offset into the table of
primitives/@code{.Internal}s.

@item CHARSXP
@code{length}, @code{truelength} followed by a block of bytes (allowing
for the @code{nul} terminator).

@item LGLSXP
@itemx INTSXP
@code{length}, @code{truelength} followed by a block of C @code{int}s
(which are 32 bits on all @R{} platforms).

@item REALSXP
@code{length}, @code{truelength} followed by a block of C @code{double}s

@item CPLXSXP
@code{length}, @code{truelength} followed by a block of C99
@code{double complex}s, or equivalent structures.

@item STRSXP
@code{length}, @code{truelength} followed by a block of pointers
(@code{SEXP}s pointing to @code{CHARSXP}s).

@item DOTSXP
A special type of @code{LISTSXP} for the value bound to a @code{...}
symbol: a pairlist of promises.

@item ANYSXP
This is used as a place holder for any type: there are no actual objects
of this type.

@item VECSXP
@itemx EXPRSXP
@code{length}, @code{truelength} followed by a block of pointers.  These
are internally identical (and identical to @code{STRSXP}) but differ in
the interpretations placed on the elements.

@item BCODESXP
For the future byte-code compiler.

@item EXTPTRSXP
Has three pointers, to the pointer, the protection value (an @R{} object
which if alive protects this object) and a tag (a @code{SYMSXP}?).

@item WEAKREFSXP
A @code{WEAKREFSXP} is a special @code{VECSXP} of length 4, with
elements @samp{key}, @samp{value}, @samp{finalizer} and @samp{next}.
The @samp{key} is @code{NULL}, an environment or an external pointer,
and the @samp{finalizer} is a function or @code{NULL}.

@item RAWSXP
@code{length}, @code{truelength} followed by a block of bytes.

@item S4SXP
two unused pointers and a tag.
@end table

@node Allocation classes,  , The 'data', SEXPs
@subsection Allocation classes

@cindex allocation classes
As we have seen, the field @code{gccls} in the header is three bits to
label up to 8 classes of nodes.  Non-vector nodes are of class 0, and
`small' vector nodes are of classes 1 to 6, with `large' vector nodes
being of class 7.  The `small' vector nodes are able to store vector
data of up to 8, 16, 32, 48, 64 and 128 bytes: larger vectors are
@code{malloc}-ed individually whereas the `small' nodes are allocated
from pages of about 2000 bytes.


@node Environments and variable lookup, Attributes, SEXPs, R Internal Structures
@section Environments and variable lookup

@cindex environment
@cindex variable lookup
What users think of as `variables' are symbols which are bound to
objects in `environments'.  The word `environment' is used ambiguously
in @R{} to mean @emph{either} the frame of an @code{ENVSXP} (a pairlist
of symbol-value pairs) @emph{or} an @code{ENVSXP}, a frame plus an
enclosure.

@cindex user databases
There are additional places that `variables' can be looked up, called
`user databases' in comments in the code.  These seem undocumented in
the @R{} sources, but apparently refer to the @pkg{RObjectTable} package
at @uref{http://www.omegahat.org/RObjectTables/}.

@cindex base environment
@cindex environment, base
The base environment is special.  There is an @code{ENVSXP} environment
with enclosure the empty environment @code{R_EmptyEnv}, but the frame of
that environment is not used.  Rather its bindings are part of the
global symbol table, being those symbols in the global symbol table
whose values are not @code{R_UnboundValue}.  When @R{} is started the
internal functions are installed (by C code) in the symbol table, with
primitive functions having values and @code{.Internal} functions having
what would be their values in the field accessed by the @code{INTERNAL}
macro.  Then @code{.Platform} and @code{.Machine} are computed and the
base package is loaded into the base environment followed by the system
profile.

The frames of environments (and the symbol table) are normally hashed
for faster access (including insertion and deletion).

By default @R{} maintains a (hashed) global cache of `variables' (that
is symbols and their bindings) which have been found, and this refers
only to environments which have been marked to participate, which
consists of the global environment (aka the user workspace), the base
environment plus environments@footnote{Remember that attaching a list or
a saved image actually creates and populates an environment and attaches
that.} which have been @code{attach}ed.  When an environment is either
@code{attach}ed or @code{detach}ed, the names of its symbols are flushed
from the cache.  The cache is used whenever searching for variables from
the global environment (possibly as part of a recursive search).

@menu
* Search paths::
* Name spaces::
@end menu

@node Search paths, Name spaces, Environments and variable lookup, Environments and variable lookup
@subsection Search paths

@cindex search path
@Sl{} has the notion of a `search path': the lookup for a `variable'
leads (possibly through a series of frames) to the `session frame' the
`working directory' and then along the search path.  The search path is
a series of databases (as returned by @code{search()}) which contain the
system functions (but not necessarily at the end of the path, as by
default the equivalent of packages are added at the end).

@R{} has a variant on the @Sl{} model.  There is a search path (also
returned by @code{search()}) which consists of the global environment
(aka user workspace) followed by environments which have been attached
and finally the base environment.  Note that unlike @Sl{} it is not
possible to attach environments before the workspace nor after the base
environment.

However, the notion of variable lookup is more general in @R{}, hence
the plural in the title of this subsection.  Since environments have
enclosures, from any environment there is a search path found by looking
in the frame, then the frame of its enclosure and so on.  Since loops
are not allowed, this process will eventually terminate: until @R{}
2.2.0 it always terminated at the base environment, but nowadays it can
terminate at either the base environment or the empty environment.  (It
can be conceptually simpler to think of the search always terminating at
the empty environment, but with an optimization to stop at the base
environment.)  So the `search path' describes the chain of environments
which is taken once the search reaches the global environment.

@node Name spaces,  , Search paths, Environments and variable lookup
@subsection Name spaces

@cindex name space
Name spaces are environments associated with packages (and once again
the base package is special and will be considered separately).  A
package @code{@var{pkg}} with a name space defines two environments
@code{namespace:@var{pkg}} and @code{package:@var{pkg}}: it is
@code{package:@var{pkg}} that can be @code{attach}ed and form part of
the search path.

The objects defined by the @R{} code in the package are symbols with
bindings in the @code{namespace:@var{pkg}} environment.  The
@code{package:@var{pkg}} environment is populated by selected symbols
fron the @code{namespace:@var{pkg}} environment (the exports).  The
enclosure of this environment is an environment populated with the
explicit imports from other name spaces, and the enclosure of
@emph{that} environment is the base name space.  (So the illusion of the
imports being in the name space environment is created via the
environment tree.)  The enclosure of the base name space is the global
environment, so the search from a package name space goes via the
(explicit and implicit) imports to the standard `search path'.

@cindex base name space
@cindex name space, base
@findex R_BaseNamespace
The base name space environment @code{R_BaseNamespace} is another
@code{ENVSXP} that is special-cased.  It is effectively the same thing
as the base environment @code{R_BaseEnv} @emph{except} that its
enclosure is the global environment rather than the empty environment:
the internal code diverts lookups in its frame to the global symbol
table.


@node Attributes, Contexts, Environments and variable lookup, R Internal Structures
@section Attributes

@cindex attributes
@findex ATTRIB
@findex SET_ATTRIB
As we have seen, every @code{SEXPREC} has a pointer to the attributes of
the node (default @code{R_NilValue}).  The attributes can be accessed/set
by the macros/functions @code{ATTRIB} and @code{SET_ATTRIB}, but such
direct access is normally@footnote{An exception is the internal code for
@code{terms.formula} which directly manipulates the attributes.} only
used to check if the attributes are @code{NULL} or to reset them.
Otherwise access goes through the functions @code{getAttrib} and
@code{setAttrib} which impose restrictions on the attributes.  One thing
to watch is that if you copy attributes from one object to another you
may (un)set the @code{"class"} attribute and so need to copy the object
bit as well.

The code assumes that the attributes of a node are either
@code{R_NilValue} or a pairlist of non-zero length (and this is checked
by @code{SET_ATTRIB}).  The attributes are named (via tags on the
pairlist).  The replacement function @code{attributes<-} ensures that
@code{"dim"} precedes @code{"dimnames"} in the pairlist.  Attribute
@code{"dim"} is one of several that is treated specially: the values are
checked, and any @code{"names"} and @code{"dimnames"} attributes are
removed.  Similarly, you cannot set @code{"dimnames"} without having set
@code{"dim"}, and the value assigned must be a list of the correct
length and with elements of the correct lengths (and all zero-length
elements are replaced by @code{NULL}).

The other attributes which are given special treatment are
@code{"names"}, @code{"class"}, @code{"tsp"}, @code{"comment"} and
@code{"row.names"}.  For pairlist-like objects the names are not stored
as an attribute but (as symbols) as the tags: however the @R{} interface
makes them look like conventional attributes (one reason why the
@code{"names"} attribute has to be treated specially).  The C code
ensures that the @code{"tsp"} attribute is an @code{REALSXP}, the
frequency is positive and the implied length agrees with the number of
rows of the object being assigned to.  Classes and comments are
restricted to character vectors, and assigning a zero-length comment or
class removes the attribute.  Setting or removing a @code{"class"}
attribute sets the object bit appropriately.  Integer row names are
converted to and from the internal compact representation.

@cindex copying semantics
Care needs to be taken when adding attributes to objects of the types
with non-standard copying semantics.  There is only one object of type
@code{NILSXP}, @code{R_NilValue}, and that should never have attributes
(and this is enforced in @code{installAttrib}).  For environments,
external pointers and weak references, the attributes should be relevant
to all uses of the object: it is for example reasonable to have a name
for an environment, and also a @code{"path"} attribute for those
environments populated from @R{} code in a package.

@cindex attributes, preserving
@cindex preserving attributes
When should attributes be preserved under operations on an object?
Becker, Chambers & Wilks (1988, pp. 144--6) give some guidance.  Scalar
functions (those which operate element-by-element on a vector and whose
output is similar to the input) should preserve attributes (except
perhaps class, and if they do preserve class they need to preserve the
@code{OBJECT} bit).  Binary operations normally call
@findex copyMostAttributes
@code{copyMostAttributes} to copy most attributes from the longer
argument (and if they are of the same length from both, preferring the
values on the first).  Here `most' means all except the @code{names},
@code{dim} and @code{dimnames} which are set appropriately by the code
for the operator.

Subsetting (other than by an empty index) generally drops all attributes
except @code{names}, @code{dim} and @code{dimnames} which are reset as
appropriate.  On the other hand, subassignment generally preserves such
attributes even if the length is changed.  Coercion drops all
attributes. For example:

@example
> x <- structure(1:8, names=letters[1:8], comm="a comment")
> x[]
a b c d e f g h
1 2 3 4 5 6 7 8
attr(,"comm")
[1] "a comment"
> x[1:3]
a b c
1 2 3
> x[3] <- 3
> x
a b c d e f g h
1 2 3 4 5 6 7 8
attr(,"comm")
[1] "a comment"
> x[9] <- 9
> x
a b c d e f g h
1 2 3 4 5 6 7 8 9
attr(,"comm")
[1] "a comment"
@end example


@node Contexts, Argument evaluation, Attributes, R Internal Structures
@section Contexts

@cindex context
@emph{Contexts} are the internal mechanism used to keep track of where a
computation has got to (and from where), so that control-flow constructs
can work and reasonable information can be produced on error conditions,
(such as traceback) and otherwise (the @code{sys.@var{xxx}} functions).

Execution contexts are a stack of C @code{structs}:

@example
typedef struct RCNTXT @{
    struct RCNTXT *nextcontext; /* @r{The next context up the chain} */
    int callflag;               /* @r{The context `type'} */
    JMP_BUF cjmpbuf;            /* @r{C stack and register information} */
    int cstacktop;              /* @r{Top of the pointer protection stack} */
    int evaldepth;              /* @r{Evaluation depth at inception} */
    SEXP promargs;              /* @r{Promises supplied to closure} */
    SEXP callfun;               /* @r{The closure called} */
    SEXP sysparent;             /* @r{Environment the closure was called from} */
    SEXP call;                  /* @r{The call that effected this context} */
    SEXP cloenv;                /* @r{The environment} */
    SEXP conexit;               /* @r{Interpreted @code{on.exit} code} */
    void (*cend)(void *);       /* @r{C @code{on.exit} thunk} */
    void *cenddata;             /* @r{Data for C @code{on.exit} thunk} */
    char *vmax;                 /* @r{Top of the @code{R_alloc} stack} */
    int intsusp;                /* @r{Interrupts are suspended} */
    SEXP handlerstack;          /* @r{Condition handler stack} */
    SEXP restartstack;          /* @r{Stack of available restarts} */
@} RCNTXT, *context;
@end example

@noindent
plus additional fields for the future byte-code compiler.  The `types'
are from

@example
enum @{
    CTXT_TOPLEVEL = 0,  /* @r{toplevel context} */
    CTXT_NEXT     = 1,  /* @r{target for @code{next}} */
    CTXT_BREAK    = 2,  /* @r{target for @code{break}} */
    CTXT_LOOP     = 3,  /* @r{@code{break} or @code{next} target} */
    CTXT_FUNCTION = 4,  /* @r{function closure} */
    CTXT_CCODE    = 8,  /* @r{other functions that need error cleanup} */
    CTXT_RETURN   = 12, /* @r{@code{return()} from a closure} */
    CTXT_BROWSER  = 16, /* @r{return target on exit from browser} */
    CTXT_GENERIC  = 20, /* @r{rather, running an S3 method} */
    CTXT_RESTART  = 32, /* @r{a call to @code{restart} was made from a closure} */
    CTXT_BUILTIN  = 64  /* @r{builtin internal function} */
@};
@end example

@noindent
where the @code{CTXT_FUNCTION} bit is on wherever function closures are
involved.

Contexts are created by a call to @code{begincontext} and ended by a
call to @code{endcontext}: code can search up the stack for a
particular type of context via @code{findcontext} (and jump there) or
jump to a specific context via @code{R_JumpToContext}.
@code{R_ToplevelContext} is the `idle' state (normally the command
prompt), and @code{R_GlobalContext} is the top of the stack.

Note that whilst all calls to closures set a context, those to special
internal functions never do, and those to builtin internal functions
have done so only recently (and prior to that only when profiling).

@findex UseMethod
@cindex method dispatch
Dispatching from a S3 generic (via @code{UseMethod} or its internal
equivalent) or calling @code{NextMethod} sets the context type to
@code{CTXT_GENERIC}.  This is used to set the @code{sysparent} of the
method call to that of the @code{generic}, so the method appears to have
been called in place of the generic rather than from the generic.

The @R{} @code{sys.frame} and @code{sys.call} work by counting calls to
closures (type @code{CTXT_FUNCTION}) from either end of the context
stack.

Note that the @code{sysparent} element of the structure is not the same
thing as @code{sys.parent()}.  Element @code{sysparent} is primarily
used in managing changes of the function being evaluated, i.e. by
@code{Recall} and method dispatch.

@code{CTXT_CCODE} contexts are currently used in @code{cat()},
@code{load()}, @code{scan()} and @code{write.table()} (to close the
connection on error), by @code{PROTECT}, serialization (to recover from
errors, e.g.@: free buffers) and within the error handling code (to
raise the C stack limit and reset some variables).


@node Argument evaluation, The write barrier, Contexts, R Internal Structures
@section Argument evaluation

@cindex argument evaluation
As we have seen, functions in @R{} come in three types, closures
(@code{SEXPTYPE} @code{CLOSXP}), specials (@code{SPECIALSXP}) and
builtins (@code{BUILTINSXP}).  In this section we consider when (and if)
the actual arguments of function calls are evaluated.  The rules are
different for the internal (special/builtin) and R-level functions
(closures).

For a call to a closure, the actual and formal arguments are matched and
a matched call (another @code{LANGSXP}) is constructed.  This process
first replaces the actual argument list by a list of promises to the
values supplied.  It then constructs a new environment which contains
the names of the formal parameters matched to actual or default values:
all the matched values are promises, the defaults as promises to be
evaluated in the environment just created.  That environment is then
used for the evaluation of the body of the function, and promises will
be forced (and hence actual or default arguments evaluated) when they
are encountered.

If the closure is an S3 generic (that is, contains a call to
@code{UseMethod}) the evaluation process is the same until the
@code{UseMethod} call is encountered.  At that point the argument on
which to do dispatch (normally the first) will be evaluated if it has
not been already.  If a method has been found which is a closure, a new
evaluation environment is created for it containing the matched
arguments of the method plus any new variables defined so far during the
evaluation of the body of the generic.  (Note that this means changes to
the values of the formal arguments in the body of the generic are
discarded when calling the method, but @emph{actual} argument promises
which have been forced retain the values found when they were forced.
On the other hand, missing arguments have values which are promises to
use the default supplied by the method and not the generic.)  If the
method found is a special or builtin it is called with the matched
argument list of promises (possibly already forced) used for the generic.

@cindex builtin function
@cindex special function
@cindex primitive function
@cindex .Internal function
The essential difference@footnote{There is currently one other
difference: when profiling builtin functions are counted as function
calls but specials are not.} between special and builtin functions is
that the arguments of specials are not evaluated before the C code is
called, and those of builtins are.  In each case positional matching of
arguments is used.  Note that being a special/builtin is separate from
being primitive or @code{.Internal}: @code{function} is a special
primitive, @code{+} is a builtin primiitve, @code{switch} is a special
@code{.Internal} and @code{grep} is a builtin @code{.Internal}.

@cindex generic, internal
@findex DispatchOrEval
Many of the internal functions are internal generics, which for specials
means that they do not evaluate their arguments on call, but the C code
starts with a call to @code{DispatchOrEval}.  The latter evaluates the
first argument, and looks for a method based on its class.  If it finds
a method, it dispatches to that method with a call based on promises to
evaluate the remaining arguments.  If no method is found, the remaining
arguments are evaluated before return to the internal generic.

@menu
* Missingness::
* Dot-dot-dot arguments::
@end menu

@node Missingness, Dot-dot-dot arguments, Argument evaluation, Argument evaluation
@subsection Missingness

@cindex missingness
Actual arguments to (non-internal) @R{} functions can be fewer than are
required to match the formal arguments of the function.  Having
unmatched formal arguments will not matter if the argument is never used
(by lazy evaluation),  but when the argument is evaluated, either its
default value is evaluated (within the evaluation environment of the
function) or an error is thrown with a message along the lines of

@example
argument "foobar" is missing, with no default
@end example

@findex MISSING
@findex R_MissingArg
Internally missingness is handled by two mechanisms. The object
@code{R_MissingArg} is used to indicate that a formal argument has no
(default) value.  When matching the actual arguments to the formal
arguments, a new argument list is constructed from the formals all of
whose values are @code{R_MissingArg} with the first @code{MISSING} bit
set.  Then whenever a formal argument is matched to an actual argument,
the corresponding member of the new argument list has its value set to
that of the matched actual argument, and if that is not
@code{R_MissingArg} the missing bit is unset.

This new argument list is used to form the evaluation frame for the
function, and if named arguments are subsequently given a new value
(before they are evaluated) the missing bit is cleared.

Missingness of arguments can be interrogated via the @code{missing()}
function.  An argument is clearly missing if its missing bit is set or
if the value is @code{R_MissingArg}.  However, missingness can be passed
on from function to function, for using a formal argument as an actual
argument in a function call does not count as evaluation.  So
@code{missing()} has to examine the value (a promise) of a
non-yet-evaluated formal argument to see if it might be missing, which
might involve investigating a promise and so on @dots{}.

@node Dot-dot-dot arguments,  , Missingness, Argument evaluation
@subsection Dot-dot-dot arguments

@cindex ... argument
Dot-dot-dot arguments are convenient when writing functions, but
complicate the internal code for argument evaluation.

The formals of a function with a @code{...} argument represent that as a
single argument like any other argument, with tag the symbol
@code{R_DotsSymbol}.  When the actual arguments are matched to the
formals, the value of the @code{...} argument is of @code{SEXPTYPE}
@code{DOTSXP}, a pairlist of promises (as used for matched arguments)
but distinguished by the @code{SEXPTYPE}.

Recall that the evaluation frame for a function initially contains the
@code{@var{name}=@var{value}} pairs from the matched call, and hence
this will be true for @code{...} as well.  The value of @code{...} is a
(special) pairlist whose elements are referred to by the special symbols
@code{..1}, @code{..2}, @dots{} which have the @code{DDVAL} bit set:
when one of these is encountered it is looked up (via @code{ddfndVar})
in the value of the @code{...} symbol in the evaluation frame.

Values of arguments matched to a @code{...} argument can be missing.


@node The write barrier, Serialization Formats, Argument evaluation, R Internal Structures
@section The write barrier and the garbage collector

@cindex write barrier
@cindex garbage collector
@R{} has since version 1.2.0 had a generational garbage collector, and
bit @code{gcgen} in the @code{sxpinfo} header is used in the
implementation of this.  This is used in conjunction with the
@code{mark} bit to identify two previous generations.

There are three levels of collections.  Level 0 collects only the
youngest generation, level 1 collects the two youngest generations and
level 2 collects all generations.  After 20 level-0 collections the next
collection is at level 1, and after 5 level-1 collections at level 2.
Further, if a level-@var{n} collection fails to provide 20% free space
(for each of nodes and the vector heap), the next collection will be at
level @var{n+1}.  (The R-level function @code{gc()} performs a level-2
collection.)

A generational collector needs to efficiently `age' the objects,
especially list-like objects (including @code{STRSXP}s).  This is done
by ensuring that the elements of a list are regarded as at least as old
as the list @emph{when they are assigned}.  This is handled by the
functions @code{SET_VECTOR_ELT} and @code{SET_STRING_ELT}, which is why
they are functions and not macros.  Ensuring the integrity of such
operations is termed the @dfn{write barrier} and is done by making the
@code{SEXP} opaque and only providing access via functions (which cannot
be used as lvalues in assignments in C).

All code in @R{} extensions is by default behind the write barrier.  The
only way to obtain direct access to the internals of the @code{SEXPREC}s
is to define @samp{USE_RINTERNALS} before including @file{Rinternals.h},
which is normally defined in @file{Defn.h}.  To enable a check on the
way that the access is used, @R{} can be compiled with flag
@option{--enable-strict-barrier} which ensures that @file{Defn.h} does
not define @samp{USE_RINTERNALS} and hence that @code{SEXP} is opaque in
most of @R{} itself.  (There are some necessary exceptions: foremost
@file{memory.c} where the accessor functions are defined and also
@file{size.c} which needs access to the sizes of the internal
structures.)

For background papers see
@url{http://www.stat.uiowa.edu/~luke/R/barrier.html} and
@url{http://www.stat.uiowa.edu/~luke/R/gengcnotes.html}.

@node Serialization Formats, Internal use of global and base environments, The write barrier, R Internal Structures
@section Serialization Formats

@cindex serialization
Serialized versions of @R{} objects are used by @code{load}/@code{save}
and also at a lower level by @code{.saveRDS}/@code{.readRDS} and
@code{serialize}/@code{unserialize}.  These differ in what they
serialize to (a file, a connection, a raw vector) and whether they are
intended to serialize a single object or a collection of objects
(typically a workspace).  @code{save} writes a header indicating the
format at the beginning of the file (a single LF-terninated line) which
the lower-level versions do not.

@R{} has used the same serialization format since @R{} 1.4.0 in December
2001.  Reading of earlier formats is still supported via @code{load},
but they are not described here.  (Files of most of these formats can
still be found in @file{data} directories of packages.) The current
serialization format is called `version 2', and has been expanded in
back-compatible ways since @R{} 1.4.0, for example to support additional
@code{SEXPTYPE}s.

@code{save()} works by first creating a tagged pairlist of objects to be
saved, and then saving that single object preceded by a single-line
header (typically @code{RDX2\n} for a binary save).  @code{load()} reads
the header line, unserializes a single object (a pairlist or a vector
list) and assigns the elements of the list in the appropriate
environment.

Serialization in @R{} needs to take into account that objects may
contain references to environments, which then have enclosing
environments and so on.  (Environments recognized as package or name
space environments are saved by name.)  Further, there are `reference
objects' which are not duplicated on copy and should should remain
shared on unserialization.  These are weak references, external pointers
and environments other than those associated with packages, name spaces
and the global environment.  These are handled via a hash table, and
references after the first are written out as a reference marker indexed
by the table entry.

Serialization first writes a header indicating the format (normally
@samp{X\n} for an XDR format binary save) and the version number of the
format and of two @R{} versions (as integers).  (Unserialization
interprets the two versions as the version of @R{} which wrote the file
followed by the minimal version of @R{} needed to read the format.)
Serialization then writes out the object recursively using function
@code{WriteItem} in file @file{src/main/serialize.c}.

Some objects are written as if they were @code{SEXPTYPE}s: such
pseudo-@code{SEXPTYPE}s cover @code{R_NilValue}, @code{R_EmptyEnv},
@code{R_BaseEnv}, @code{R_GlobalEnv}, @code{R_UnboundValue},
@code{R_MissingArg} and @code{R_BaseNamespace}.

For all @code{SEXPTYPE}s except @code{NILSXP}, @code{SYMSXP} and
@code{ENVSXP} serialization starts with a integer with the
@code{SEXPTYPE} in bits 0:7@footnote{only 0:4 will currently be used for
@code{SEXPTYPE}s but values 241:255 are used for pseudo-@code{SEXPTYPE}s.}
followed by the object bit, two bits indicating if there are any
attributes and if there is a tag (for the pairlist types), an unused bit
and then the @code{gp} field@footnote{Currently the only relevant bits
are 0:1, 4, 14:15.} in bits 12:27.  Pairlist-like objects write their
attributes (if any), tag (if any), CAR and then CDR (using tail
recursion): other objects write their attributes after themselves.
Atomic vector objects write their length followed by the data: generic
vector-list objects write the length followed by a call to
@code{WriteItem} for each element.  The code for @code{CHARSXP}s
special-cases @code{NA_STRING} and writes it as length @code{-1} with no
data.

Environments are treated in several ways: as we have seen, some are
written as specific pseudo-@code{SEXPTYPE}s.  Package and name space
environments are written with pseudo-@code{SEXPTYPE}s followed by the
name.  `Normal' environments are written out as @code{ENVSXP}s with an
integer indicating if the environment is locked followed by the
enclosure, frame, tag and attributes.


@node Internal use of global and base environments, Modules, Serialization Formats, R Internal Structures
@section Internal use of global and base environments

This section notes known use by the system of these environments: the
intention is to minimize or eliminate them.

@menu
* Base environment::
* Global environment::
@end menu

@node Base environment, Global environment, Internal use of global and base environments, Internal use of global and base environments
@subsection Base environment

@cindex base environment
@cindex environment, base
@findex .Device
@findex .Devices
The graphics devices system maintains two variables @code{.Device} and
@code{.Devices} in the base environment: both are always set.  The
variable @code{.Devices} gives a list of character vectors of the names
of open devices, and @code{.Device} is the element corresponding to the
currently active device.  The null device will always be open.

@findex .Options
There appears to be a variable @code{.Options}, a pairlist giving the
current options settings.  But in fact this is just a symbol with a
value assigned, and so shows up as a base variable.

@findex .Last.value
Similarly, the evaluator creates a symbol @code{.Last.value} which
appears as a variable in the base environment.

@findex .Traceback
@findex last.warning
Errors can give rise to objects @code{.Traceback} and
@code{last.warning} in the base environment.

@node Global environment,  , Base environment, Internal use of global and base environments
@subsection Global environment

@cindex global environment
@cindex environment, global
@findex .Random.seed
The seed for the random number generator is stored in object
@code{.Random.seed} in the global environment.

@findex dump.frames
Some error handlers may give rise to objects in the global environment:
for example @code{dump.frames} by default produces @code{last.dump}.

@findex .SavedPlots
The @code{windows()} device makes use of a variable @code{.SavedPlots}
to store display lists of saved plots for later display.  This is
regarded as a variable created by the user.


@node Modules,  , Internal use of global and base environments, R Internal Structures
@section Modules

@cindex modules
@R{} makes use of a number of shared objects/DLLs stored in the
@file{modules} directory.  These are parts of the code which have been
chosen to be loaded `on demand' rather than linked as dynamic libraries
or incorporated into the main executable/dynamic library.

For a few of these (e.g.@: @code{vfonts}) the issue is size: the
database for the Hershey fonts is included in the C code of the module
and was at one time an appreciable part of the codebase for a rarely
used feature.  However, for most of the modules the motivation has been
the amount of (often optional) code they will bring in via libraries to
which they are linked.

@table @asis

@item @code{internet}
The internal HTTP and FTP clients and socket support, which link to
system-specific support libraries.

@item @code{lapack}
The code which makes use of the LAPACK library, and is linked to
@file{libRlapack} or an external LAPACK library.

@item @code{vfonts}
The Hershey font databases and the code to draw from them.

@item @code{X11}
(Unix-alikes only.)
The @code{X11()}, @code{jpeg()} and @code{png()} devices. These are
optional, and link to the @code{X11}, @code{jpeg} and @code{libpng}
libraries.

@item @file{Rbitmap.dll}
(Windows only.)  The code for the BMP, JPEG and PNG devices and for
saving on-screen graphs to those formats. This is technically optional,
and needs source code not in the tarball.

@item @file{Rchmtl.dll}
(Windows only.)  A link to an ActiveX control that displays Compiled
@HTML{} help.  This is optional, and only compiled if CHTML is specified.

@item @file{iconv.dll}
(Windows only.)  A DLL compiled via Visual C++ which contains the
routines to convert between character sets.

@item @file{internet2.dll}
(Windows only.)  An alternative version of the internet access routines,
compiled against Internet Explorer internals (and so loads
@file{wininet.dll} and @file{wsock32.dll}).
@end table

@node .Internal vs .Primitive, R coding standards, R Internal Structures, Top
@chapter @code{.Internal} vs @code{.Primitive}

@findex .Internal
@findex .Primitive
C code compiled into @R{} at build time can be called ``directly'' or
via the @code{.Internal} interface, which is very similar to the
@code{.External} interface except in syntax.  More precisely, @R{}
maintains a table of R function names and corresponding C functions to
call, which by convention all start with @samp{do_} and return a SEXP.
Via this table (@code{R_FunTab} in file @file{src/main/names.c}) one can
also specify how many arguments to a function are required or allowed,
whether the arguments are to be evaluated before calling or not, and
whether the function is ``internal'' in the sense that it must be
accessed via the @code{.Internal} interface, or directly accessible in
which case it is printed in @R{} as @code{.Primitive}.

@R{}'s functionality can also be extended by providing corresponding C
code and adding to this function table.

In general, all such functions use @code{.Internal()} as this is safer
and in particular allows for transparent handling of named and default
arguments.  For example, @code{axis} is defined as

@example
@group
axis <- function(side, at = NULL, labels = NULL, ...)
    .Internal(axis(side, at, labels, ...))
@end group
@end example

However, for reasons of convenience and also efficiency (as there is
some overhead in using the @code{.Internal} interface wrapped in a
function closure), there are exceptions which can be accessed directly.
Note that these functions make no use of @R{} code, and hence are very
different from the usual interpreted functions.  In particular,
@code{args}, @code{formals} and @code{body} return @code{NULL} for such
objects, and argument matching is purely positional (with empty
positions being dropped).

The list of these ``primitive'' functions is subject to change:
currently, it includes the following.

@enumerate

@item
``Special functions'' which really are @emph{language} elements, however
exist as ``primitive'' functions in @R{}:

@example
@group
@{       (         if     for      while  repeat  break  next
return  function  quote  on.exit
@end group
@end example

@item
Basic @emph{operator}s (i.e., functions usually @emph{not} called as
@code{foo(a, b, ...)}) for subsetting, assignment, arithmetic and logic.
These are the following 1-, 2-, and @math{N}-argument functions:

@example
@group
               [    [[    $    @
<-   <<-  =    [<-  [[<-  $<-

+    -    *    /     ^    %%   %*%  %/%
<    <=   ==   !=    >=   >
|    ||   &    &&    !
@end group
@end example

@item
``Low level'' 0- and 1-argument functions which belong to one of the
following groups of functions:

@enumerate a
@item
Basic mathematical functions with a single argument, i.e.,

@example
@group
sign    abs
floor   ceiling  trunc
@end group

@group
sqrt    exp
cos     sin      tan
acos    asin     atan
cosh    sinh     tanh
acosh   asinh    atanh
@end group

@group
cumsum  cumprod
cummax  cummin
@end group

@group
Im      Re
Arg     Conj     Mod
@end group
@end example

@noindent Note however that the @R{} function @code{log} has an optional
named argument @code{base}, and therefore is defined as

@example
@group
log <- function(x, base = exp(1)) @{
  if(missing(base))
    .Internal(log(x))
  else
    .Internal(log(x, base))
@}
@end group
@end example

@noindent in order to ensure that @code{log(x = pi, base = 2)} is
identical to @code{log(base = 2, x = pi)}.

@item
Functions rarely used outside of ``programming'' (i.e., mostly used
inside other functions), such as

@example
@group
nargs        missing
interactive  is.@emph{xxx}
.Primitive   .Internal
symbol.C     symbol.For
globalenv    baseenv     emptyenv     pos.to.env
unclass
seq_along    seq_len
@end group
@end example

@noindent
(where @emph{xxx} stands for almost 30 different notions, such as
@code{function}, @code{vector}, @code{numeric}, and so forth, but not
@code{is.loaded}).

@item
The programming and session management utilities

@example
@group
debug    undebug    trace  untrace
browser  proc.time  gc.time
@end group
@end example

@end enumerate

@item
The following basic assignment and extractor functions

@example
@group
length      length<-
class       class<-
oldClass    oldCLass<-
attr        attr<-
attributes  attributes<-
dim         dim<-
dimnames    dimnames<-
            environment<-
@end group
@end example

@item
The following few @math{N}-argument functions are ``primitive'' for
efficiency reasons:

@example
@group
:          ~          c           list
call       as.call    as.character
expression substitute as.environment
UseMethod  invisible  standardGeneric
.C         .Fortran   .Call       .External
.Call.graphics        .External.graphics
.subset    .subset2   .primTrace  .primUntrace
rep        seq.int
@end group
@end example

@noindent
@code{rep} and @code{seq.int} manage their own argument matching and so
do work in the standard way.

@end enumerate

@node R coding standards, Testing R code, .Internal vs .Primitive, Top
@chapter R coding standards

@cindex coding standards
@R{} is meant to run on a wide variety of platforms, including Linux and
most variants of Unix as well as 32-bit Windows versions and on MacOS X.
Therefore, when extending @R{} by either adding to the @R{} base
distribution or by providing an add-on package, one should not rely on
features specific to only a few supported platforms, if this can be
avoided.  In particular, although most @R{} developers use @acronym{GNU}
tools, they should not employ the @acronym{GNU} extensions to standard
tools.  Whereas some other software packages explicitly rely on e.g.@:
@acronym{GNU} make or the @acronym{GNU} C++ compiler, @R{} does not.
Nevertheless, @R{} is a @acronym{GNU} project, and the spirit of the
@cite{@acronym{GNU} Coding Standards} should be followed if possible.

The following tools can ``safely be assumed'' for @R{} extensions.

@itemize @bullet
@item
As from @R{} 2.3.0 an ISO C99 C compiler.  Note that extensions such as
@acronym{POSIX} 1003.1 must be tested for, typically using Autoconf
unless you are sure they are supported on all mainstream @R{} platforms
(including Windows and MacOS X).  Packages will be more portable if
written assuming only C89, but this should not be done where using C99
features will make for cleaner or more robust code.

@item
A FORTRAN 77 compiler (but not Fortran 9x).

@item
A simple @command{make}, considering the features of @command{make} in
4.2 @acronym{BSD} systems as a baseline.
@findex make

@acronym{GNU} or other extensions, including pattern rules using
@samp{%}, the automatic variable @samp{$^}, the @samp{+=} syntax to
append to the value of a variable, the (``safe'') inclusion of makefiles
with no error, conditional execution, and many more, must not be used
(see Chapter ``Features'' in the @cite{@acronym{GNU} Make Manual} for
more information).  On the other hand, building @R{} in a separate
directory (not containing the sources) should work provided that
@command{make} supports the @code{VPATH} mechanism.

Windows-specific makefiles can assume @acronym{GNU} @command{make} 3.75
or later, as no other @command{make} is viable on that platform.

@item
A Bourne shell and the ``traditional'' Unix programming tools, including
@command{grep}, @command{sed}, and @command{awk}.

There are @acronym{POSIX} standards for these tools, but these may not
fully be supported.  Baseline features could be determined from a book
such as @cite{The UNIX Programming Environment} by Brian W. Kernighan &
Rob Pike.  Note in particular that @samp{|} in a regexp is an extended
regexp, and is not supported by all versions of @command{grep} or
@command{sed}.  The Open Group Base Specifications, Issue 6, which is
technicall identical to ISO/IEC 9945 and IEEE Std 1003.1 (POSIX), 2004,
are available at
@url{http://www.opengroup.org/onlinepubs/009695399/mindex.html}.
@end itemize

Under Windows, most users will not have these tools installed, and you
should not require their presence for the operation of your package.
However, users who install your package from source will have them, as
they can be assumed to have followed the instructions in ``the Windows
toolset'' appendix of the ``R Installation and Administration'' manual
to obtain them.  Redirection cannot be assumed to be available via
@command{system} as this does not use a standard shell (let alone a
Bourne shell).

@noindent
In addition, the following tools are needed for certain tasks.

@itemize @bullet
@item
Perl version 5 is needed for converting documentation written in Rd
format to plain text, @HTML{}, @LaTeX{}, and to extract the examples.
In addition, several other tools, in particular @command{check} and
@command{build} require Perl.
@findex Perl

The @R{} Core Team has decided that Perl (version 5) can safely be
assumed for building @R{} from source, building and checking add-on
packages, and for installing add-on packages from source.  On the other
hand, Perl cannot be assumed at all for installing @emph{binary}
(pre-built) versions of add-on packages, or at run time.

@item
Makeinfo version 4.7 is needed to build the Info files for the @R{}
manuals written in the @acronym{GNU} Texinfo system.  (Future
distributions of @R{} may contain the Info files.)
@findex makeinfo
@end itemize

It is also important that code is written in a way that allows others to
understand it.  This is particularly helpful for fixing problems, and
includes using self-descriptive variable names, commenting the code, and
also formatting it properly.  The @R{} Core Team recommends to use a
basic indentation of 4 for @R{} and C (and most likely also Perl) code,
and 2 for documentation in Rd format.  Emacs users can implement this
indentation style by putting the following in one of their startup
files.  (For @acronym{GNU} Emacs 20: for @acronym{GNU} Emacs 21 or later
use customization to set the @code{c-default-style} to @code{"bsd"} and
@code{c-basic-offset} to @code{4}.)
@findex emacs

@smallexample
@group
;;; C
(add-hook 'c-mode-hook
          (lambda () (c-set-style "bsd")))
@end group
@group
;;; ESS
(add-hook 'ess-mode-hook
          (lambda ()
            (ess-set-style 'C++)
            ;; Because
            ;;                                 DEF GNU BSD K&R C++
            ;; ess-indent-level                  2   2   8   5   4
            ;; ess-continued-statement-offset    2   2   8   5   4
            ;; ess-brace-offset                  0   0  -8  -5  -4
            ;; ess-arg-function-offset           2   4   0   0   0
            ;; ess-expression-offset             4   2   8   5   4
            ;; ess-else-offset                   0   0   0   0   0
            ;; ess-close-brace-offset            0   0   0   0   0
            (add-hook 'local-write-file-hooks
                      (lambda ()
                        (ess-nuke-trailing-whitespace)))))
(setq ess-nuke-trailing-whitespace-p 'ask)
;; or even
;; (setq ess-nuke-trailing-whitespace-p t)
@end group
@group
;;; Perl
(add-hook 'perl-mode-hook
          (lambda () (setq perl-indent-level 4)))
@end group
@end smallexample

@noindent
(The `GNU' styles for Emacs' C and R modes use a basic indentation of 2,
which has been determined not to display the structure clearly enough
when using narrow fonts.)

@node Testing R code, Function and variable index, R coding standards, Top
@chapter Testing R code

When you (as @R{} developer) add new functions to the R base (all the
packages distributed with @R{}), be careful to check if @kbd{make
test-Specific} or particularly, @kbd{cd tests; make no-segfault.Rout}
still works (without interactive user intervention, and on a standalone
computer).  If the new function, for example, accesses the Internet, or
requires @acronym{GUI} interaction, please add its name to the ``stop
list'' in @file{tests/no-segfault.Rin}.

[To be revised: use @command{make check-devel}, check the write barrier
if you change internal structures.]

@node Function and variable index, Concept index, Testing R code, Top
@unnumbered Function and variable index

@printindex vr

@node Concept index,  , Function and variable index, Top
@unnumbered Concept index

@printindex cp

@bye

@c Local Variables: ***
@c mode: TeXinfo ***
@c End: ***
