=head1 NAME

zoiddevel - Development documentation for zoid

=head1 DESCRIPTION

=head2 Debugging

If you switch on the global debug bit and the verbose bit both your input
and debug information are echoed to STDERR. Thus by piping both STDOUT and
STDERR to a file you get a detailed log of what is happening.

To set both bits type:

 zoid$ set debug
 zoid$ set verbose

or start zoid with:

 $ zoid -vD

You can also get debug information from just one module by setting a global
variable $DEBUG in the module's namespace.

If you set the debug variable to a number higher than 1, you get a stack trace
of that number of frames for each exception thrown using the C<error()> method.

=head2 Structure

=head3 class diagram

   Zoidberg::Shell
         /|\            Zoidberg::Contractor
          |                 /|\   /|\
          |__________________|     |
               |                   |
               |            Zoidberg::Job
            Zoidberg              /|\
                                   |
                                   |
                            Zoidberg::Job::Builtin

All other classes stand on their own 
(except perhaps for some internal helper classes).

Zoidberg::Shell is an interface class to be used in perl scripts like F<zoidrc>.
Zoidberg inheriting from it in order to make this interface also available through
the C<$shell> object in the eval scope and to plugins.

The Zoidberg::Contractor class contains the code to manage Jobs.
Because the Job class also inherits from it sub-jobs are possible.

=head3 object structure

   main object  class Zoidberg
     |
     |___ {eval}          Zoidberg::Eval object
     |___ {stringparser}  Zoidberg::StringParser object
     |
     |___ {jobs}          Array with Job objects
     |      |_ items of class Zoidberg::Job
     |
     |___ {settings}      Settings and config hashes
     |___ {objects}       Plugin objects       - tied Zoidberg::PluginHash
     |___ {parser}        Custom parsing       - tied Zoidberg::DispatchTable
     |___ {events}        Event code refs      - tied Zoidberg::DispatchTable
     |___ {commands}      Command code refs    - tied Zoidberg::DispatchTable
     |___ {aliases}       Alias definitions

FIXME explanation

=head3 Process flow

       start
         |
         |      incomplete input - readmore
         |    .--------------------.
         |    |                    |
         |    |                    |
         V    V   string           |           tree
   input loop     -------->  parse string      ------>  execute tree
   Z:main_loop()             Z:shell_string()           Z:C:shell_list()
   readline                                               |
         A                                   reincarnate  |
         |                                       ,------> |
  new    |                           next chunk  |        |
  input  |                           of blocks   |        | logic chunk
         |                                       |        |
         |                                       |        V
         '---------------------------------------+---  execute job
                                                       Z:J:exec()
                                                          
  Z:   = Zoidberg::
  Z:C: = Zoidberg::Contractor::
  Z:J: = Zoidberg::Job::

The parse tree going to C<shell_list()> is a simple parse tree consisting of scalars
and scalar references (see FIXME below). This tree is processed in logic chunks,
corresponding to one job each.
The blocks references in a logic chunk are parsed by C<parse_block()> just before 
executing. This is where the context is determined and things like expansions and
subtitutions are done.

=head2 Parsing

FIXME put devel opts in scheme

--------------------------------------------------------------------------------

  string  --> shell_string()
  
            split script_gram
                   
                  |
                  V
  
  list    --> shell_list()   ... FIXME tell bout this format below
         
                  |
                  V
   
   ref   -->  parse_block()   ... FIXME tell bout possible block forms
                  
                  |
                  V                 ------------.
                                                |
             parse_env()                        | if word block
                                                |
             split word_gram                    |
             split redir_gram                   |
           strip env declaration     A          |
                                     |          |
                 |           <------------------'
                 V                   |
           alias expansion    -------' if multiple blocks result
           custom filtering
           decide context unless already known
           - context on whole string like perl
           - custom word lists
           - default word context
               
                 |----> custom parser
                 |
                 |----> parse_perl()  for perl context
                 |
                 | for word context   ---------.  if 'FIXME'
                 V                             |
                                               |
             parse_words()                     V
  
             expansion   ... FIXME list them in zoiduser
             (globs)

--------------------------------------------------------------------------------

A block that originates from C<shell()> or C<Zoidberg::Shell::AUTOLOAD> is recognised
as having already been split into words. This means the parsing of words, redirections
and environment declarations is skipped. Only alias are checked so this interface
is consistent in command names etc. with the commandline interface.

If you use C<builtin()> or C<command()>, these are understood as simple commands
with parsed words and effectively all parsing is skipped.

=head2 Parse tree

FIXME tell about the blocks used in lots of hooks

FIXME renice this stuff

This part describes the form of a parse tree as used between the various Zoidberg objects.

=head3 Example

 # Commandline input:
 
 $ ls -al | perl{ while (<STDIN>) { print $_ }  }abc && echo done
 
 # Would be parsed to:
 
 [ 
   [{context => 'CMD'}, qw/ls -al/],
   [{context => 'PERL', opts => 'abc'}, q{ while (<STDIN>) { print $_ } } ],
   'AND',
   [{context => 'CMD'}, qw/echo done/]
 ]
 
 
 
 # Commandline input:
 
 $ cd .. && ls -al ; cp dus ~/tmp/ &
 
 # Would be parsed to:
 
 [
   [{context => 'CMD'}, qw/cd ../],
   'AND',
   [{context => 'CMD'}, qw/ls -al/],
   'EOS',    # End Of Statement
   [{context => 'CMD'}, qw{cp dus ~/tmp/}],
   'EOS_BG'  # background statement
 ]
  
 # FIXME an example with redirections

There are a lot more meta fields like 'fork_job', FIXME where to read bout them ?

=head3 Basics

A parse tree is an array consisting of blocks and tokens. A block can be any kind of code 
and is stored in a nested array.
Blocks directly following each other are supposed to be a pipeline. 
A token is a delimiter between blocks.

The first field of a block is a hash which contains information about the block, 
all other field in a block make up the content.
The most important information about a block is the context, 
which tells the parser how to execute the block.
You are  free to store all kinds of specific information in this first field, but some
key names are reserved.

FIXME reserved meta fields

=head3 Pseudo parse trees

These are forms that can be used with the C<shell()> function provided by L<Zoidberg::Shell>.
Just as by the real parse tree blocks of code are references and tokens are plain scalars. 
A block that is a scalar reference will be split into words and parsed completely 
(although still expected to be one block).
A block that is an array reference will be considered to be completely parsed if the first element
is a hash reference and the context defined, else it is considered a word list, possibly with
meta data defined in the first element.

  # for example "ls -al | perl{ while (<STDIN>) { print $_ }  }abc && echo done"
  # can be executed by calling :
  shell(
    [qw/ls -al/],
    \'perl{ while (<STDIN>) { print $_ }  }abc',
    'AND'
    [{context => 'CMD'}, qw/echo done/]
  );

Using this kind of pseudo trees only makes sense if you are lazy or you don't know exactly what
the command is but you have some clues.

=head3 Built-in contexts

The contexts 'PERL', 'CMD' and 'SUBZ' are used by the built-in parser.
'PERL' flags code as perl code, 'CMD' as a system or built-in command.
The 'SUBZ' context is used for sub-shells.

=head2 Settings

Notice that every plugin has it's own config hash in the settings hash,
it is suggested that settings only affecting one plugin are placed in this sub-hash.
This rule is broken for some settings required by the posix specification of
the C<set> built-in.

FIXME list these posix settings somwhere

Below are listed advanced settings only needed for development.
Common settings can be found in L<zoiduser>.

The parser settings can be set globally, but also per block.
FIXME they should also be settable per mode/context

=over 4

=item split_script

Split a command according to the script grammar.

=item split_words

Split a command into words.

=item parse_fd

Parse redirections.

=item parse_env

Parse environment declerations.

=item parse_aliases

Check for aliases.

=item parse_def_contexts

Use the default contexts PERL and CMD.

=item expand_comm

Expand commands in "word" contexts.

=item expand_param

Expand parameters in "word" contexts.

=item expand_path

Expand globs in "word" contexts.

=item debug

Turn on _all_ debug messages.

=item plain_words

This is not a global setting, but if used for a single block it is identical to turning off
all parser settings except for 'parse_aliases'.

=back

=head2 Events

The hash called 'events' in the main object is a tied hash of the class L<Zoidberg::DispatchTable>. 
This hash contains stacks of code refs to be called when certain events happen.

Some events used:

FIXME more events

=over 4

=item beat

Called every idle second when using L<Term::ReadLine::Zoid>. Used for asynchronous stuff.

=item newpwd

Called if C<$ENV{PWD}> has changed. The old pwd can be found in C<$ENV{OLDPWD}>.

=item prompt

This event is called from the main loop just before respawning the prompt.
It can be used to update routines.

=item cmd

This event is called after the input prompt with the commandline as the first argument.

=item envupdate

This event is called after the spawning of every job. Like 'prompt' it can be used for
update routines; since 'envupdate' is called more often then 'prompt', use 'prompt' if it
makes no difference, 'envupdate' should only be used for atomic updates.

=item loadrc

Called after all the plugins are loaded, it triggers reading the rcfiles.
Can be used by plugins to set hooks that depend on other plugins being loaded.

=item exit

Called just before C<round_up()>, used as replacement for F<.bash_logout>.

=item set_*

Whenever a setting is changed a event is called which is the name of
the setting prefixed by 'set_' . Arguments are the new value and the old value
of the setting. When the new value is 'undef' the setting was deleted.

These events are generated by the Zoidberg::SettingHash class, which is used
to tie the settings hash of the main Zoidberg object.

=item plug_*

=item unplug_*

Called when a plugin is loaded or deleted.

=back

=head2 Plugins

Zoid makes extensive use of plugins to perform tasks that are not considered
the core functionality of a shell (and very little is). A plugin has three
ways of communicating with the the shell and other plugins:

=over 4

=item builtins

These are commands implemented in the shell to offer some function to both
the user and other plugins.

=item events

A plugin can both trigger events ('broadcast') or listen to certain events.
When an event is triggered all code 'listening' to it is executed, this
is a one-to-many dispatcher.

=item contexts

Used to extend the code types zoid knows. By default only command syntax and perl
are recognized, but a plugin might want to add for example SQL syntax.

A context is typically used when it is unconvenient to define all commands
the plugin can handle as builtins, or when a plugin want to handle the whole
code block instead of the individual (possibly expanded) words.

Also some methods are provided to hook into certain parser stages.

=back

Some general tips for writing plugins:

=over 4

=item *

See the L<Zoidberg::Utils> namespace for common functions. Using these functions
will make your plugin more coherent with the rest of the shell.

=item *

Try to use api's, not data structures; unless the structures are specified in the
documentation they can change without notice. Api's are in general more stable.

=item *

Try to use L<AutoLoader> if you have a lot of optional functionality.

( When using AutoLoader make sure your subs have names that
are case-insensitive unique, else you'll run into problems on OS X. )

=item *

When you need hooks between two plugins, use a builtin command. Make sure to
find a name for it that doesn't overload a normal system command. When
you need various hooks, see if they can be combined in one builtin command
using options (see L<Zoidberg::Utils::GetOpt>). Also check if other shell
implementations (like bash, tcsh or zsh) have builtins for similar
funtions and try to make your interface consistent.

When calling a builtin as a hook the most efficient way is to use the 
C<< $shell->builtin($cmd, @args) >> method in a list context.
Using this method will cause a lot of parser levels to be skipped, also
the command won't be subject to things like the current command mode.
Using list context makes the method return directly without bothering 
with overloaded C<Zoidberg::Shell::scalar> objects.

=back

=head3 PluginConf

A plugin configuration file can use the following keys:

=over 4

=item module

The class to bless the plugin object in.

=item config

Hash with plugin specific config stuff. For plugins that inherit from
Zoidberg::Fish this will automatically become the C<{config}> attribute.

=item commands

Hash linking command names to subroutines. 
See L<Zoidberg::DispatchTable> for the string format used in this hash.

=item export

Array with commands automatically linking to a subroutine of the same name 
in the plugin object.

=item events

Like C<commands> but used for events.

=item import

Like C<export> but used for events.

=item settings

Supply some default settings. Used for global settings, plugin settings
should be in the 'config' hash.

=item aliases

Supply some default aliases.

=item context

Add context hooks.

=back

If you want your plugin to be loaded as soon as possible, you can use
the L<loadrc> event to load your plugin. For plugins inheriting from
L<Zoidberg::Fish> C<< loadrc => 'plug' >> will work.

=head3 Custom contexts

Typically you'll want to set a custom context from a plugin.
To do this you either provide some context hooks in the L<PluginConf>
or you set a L<loadrc> event to load your plugin and then 
setup the context parser and handlers routines from the plugin's C<init()> routine,
using the C<add_context()> method in C<Zoidberg::Fish>.
If you defined a L<word_list> or L<filter> hook it makes sense to use the
second method. The plugin would be loaded anyway when the first command is parsed.

A context config hash can contain the following routines:

FIXME more verbose description

=over 4

=item word_list BLOCK

If you want your context to work with words you should have this hook to recognize
and list commands.
On wantarray it should return a list of possible completions for the word in C<$$block[1]>;
else it  should check whether the word is a recognized command and return true on success.
In both cases it is also allowed to return a block ref, this is for the more advanced options.

=item parser BLOCK

Here you can set options like for example C<fork_job> or C<no_words> for your context.

=item handler BLOCK

This is the part where your command gets executed.

=item completion_function WORD, LINE, START, END

Hook used by the Intel plugin for custom completion.

=item intel BLOCK

Hook used by the Intel plugin for custom completion.

=item filter BLOCK

Hook to filter blocks before they are parsed, this can be used for example
for custom redirection code.

Be aware that the a block might be filtered twice with the same meta hash,
the second time should reset any meta fields set by the first time.

=back

=head3 "Hello World" plugin

Here follows an example to create a simple plugin with the "hello_world" builtin.

First create a dir F<~/.zoid/plugins/HelloWorld>.

Next create a module, for example :

  package HelloWorldZoid;
  
  # Zoidberg::Fish is the base class for plugins
  use Zoidberg::Fish;
  our @ISA = qw/Zoidberg::Fish/;
  
  # Zoidberg::Utils provides the output method
  use Zoidberg::Utils;
  
  # no need for a constructor, bootstrap with init
  sub init { $$_[0]{config}{string} ||= "Hello world !" }
  
  # and in this sub we actually print the string
  sub hello_world { output $$_[0]{config}{string} }
  
  # this will be called when the plugin is unloaded
  sub round_up { output "someone help me" }
  
  1; # keep require happy

Save the module as F<~/.zoid/plugins/HelloWorld/HelloWorldFish.pm>.

Then create a config file, this is just a perl script returning a config hash.

  {
    module => q/HelloWorldFish/,
    config => {
      # This hash will be both accessible as $shell->{settings}{HelloWorld}
      # and as $shell->HelloWorld->{config}
      string => q/Hello cruel world !/
    },
    export => ['hello_world'],
  }

Save the config file as F<~/.zoid/plugins/HelloWorld/PluginConf.pl>.

After restarting zoid, you should have a builtin command "hello_world" that prints the string
"Hello cruel world !", and an object called "HelloWorld".
Of course you guessed already that you can control the string that will be printed from the
config file, and also that it defaults to "Hello world !".
On run-time this string is available under C<< $shell->{settings}{HelloWorld}{string} >>,
it can be changed at any time.

=head1 ENVIRONMENT

The following environment variables are used be Zoidberg.

=over 4

=item ZOIDPID

Contains the process id of a parent zoidberg shell, intended to be used for an IPC mechanism.

=item ZOIDREF

In forked child processes this variable contains a stringyfied version of the current Zoidberg
object in charge. The parent process has a global hash C<%Zoidberg::OBJECTS> which maps these
strings to the original references.
It is intended that an IPC mechanism uses this hash to convert strings back to references.

This value should correspond to the object stored in the global C<$Zoidberg::CURRENT> at
the time of forking.

In secondary scripts it is better to use C<< Zoidberg::Shell->current() >>.

TODO this mechanism seems to have some problems

=item ZOIDCMD

Contains the string that caused the current command to be executed.
Can be used by a command to inspect its arguments as they were before path expansions etc. .

=head1 SEE ALSO

L<perl>(1), L<http://zoidberg.sourceforge.net>
