TODO for Pugs::Compiler::Rule (PCR)

* 2007-04-30

- <rule( param... )> doesn't always emit the parameter list

* 2007-03-09

- reuse kp6's AST processor
  - fix repeated capture counting problems

- remove PadWalker dependency?

- move all unicode tables (\n, \t, \v, \h, char classes) into 
  Pugs::Emitter::Rule::Perl5::CharClass

- try to compile Perl-6.0.0-STD.pm 

- finish | || & &&
  - DFA
  - [done] check precedence in Rule2.pm
  - workaround Rule2.pm problems (see the header in that file)
    (see MP6 for a workaround)

- document how to install a rule using eval $rule->source
  in order to access local variables (without PadWalker)

- implement compile-as-closure in order to allow ratchet/non-ratchet in
  the same rule

- finish **{...} implementation
- finish modifiers implementation

- check failing (infinite-loop) tests

* OLDER TODO

- integrate pure-regex into the main compiler

- fix static count for repeated captures: 
  { <x> <x>+ }  -- the first capture is not captured to array

- add rollback for [ x <!before k> ]+
  - should 'unmatch' last 'x' if there is a 'k' after it

- add 'negate' node to Regex
- add tests for 'negate'
- finish remaining metasyntax in <!...> Grammar

- static signature for '$^a' 

- add 'inlined' switch to disable PadWalker lookups

- fix the <ws> rule

Sigspace switch:

- 'doubled' <ws> doesn't work
  my $rule = Pugs::Compiler::Rule->compile('a (b) * c' );
  - but this works:  
  my $rule = Pugs::Compiler::Rule->compile('a (b )*c' );
- implement, test in non-ratchet rules

Ratchet switch:

- quantifiers
  - implement ?? +? *?
  - implement ?+ ++ *+
  - implement captures to Array inside quantifiers
- is :ratchet allowed as a runtime switch?
  - are there runtime switches?
  - is :+ the non-ratchet specifier?

Perl 6 parser:

- implement Perl 6 minilanguages (and categories).
  each language generate entries for the tokenizer hash
  <%statement_control|%prefix|%term>
- implement the main tokenizer using the minilanguages as terms

Priorities:

- features used in PGE P6 Grammar
- <after ...> in non-ratchet

- make PadWalker optional, as most distros don't hold PW 1.0

- '%var := xxx'; '@var := xxx'; %<var> := ; @<var> :=

- rule parameters 

- add placeholder 'not implemented' messages in the right places

- make Emitter::Rule::Perl5 subclassable - such that closures can be 
  parsed/compiled by a custom parser
- make the 'return' block functionally detectable (instead of regex)

Bugs:

- $variable should look into the user pad

Categories (some things belong to P::C::Perl6) :

- see "Syntactic categories" in S05 (extensibility)

- finish documenting add_op( %options )

- memoize/recompile dynamically

- special cases (test):
  <tight> <op>+ [<tight> <op>+]* <tight>?
  1;;;2  - ignore extra ;;?
  1,,2
  1, 2,
  1 Y 2 Y 
  loop ($a;;$b) { ... } - special form via macro?

Tests:

- unify ratchet/non-ratchet tests

- add categories looser/tighter than current boundaries

- backtracking control ':' 

- <'... <?... <!...

Related projects:

- implement PCR using pugs

- implement Pugs::Compiler::PILN ?
  - not a priority - p5 could load ready-made YAML from Pugs

Other:

- see 'not implemented' messages in Pugs::Emitter::Rule::Perl5

- closures should be called on the Match object

- rule documentation (shorter than S05)
- examples of use

- incomplete rule syntax (see Pugs::Compiler::Rule pod); 
- more escaped chars

- More switches (:p, :ratchet, :sigspace are implemented)

- optimization: inline code in the non-ratchet Engine; 
- see "benchmark.pl"

- cached precompiled variable rules, per S05

- bug: can only access parameters through $^a $^b...
- bug: parameters must be constants

API:

- maybe change PCR->code() to PCR->as_method()


TODO for Pugs-Compiler-Precedence

- needs to use category identifiers, like INFIX
  maybe INFIX_004 for category+precedence?

- \s{ is an expression terminator if the parser is expecting an operator

- add is-parsed to the appropriate category in op-precedence and tokenizer


= Implemented Features  -- update this!

  <ws> 
  #comment\n 
  . 
  ? * + *? +? ??
  literal
  [] 
  ()     
  |
 
  <'literal'>
  <subrule>
  <namespace::subrule>
  <$var>      
  <?subrule>
  <!subrule>
  <before ...>
  <after ...>         -- implemented in :ratchet mode only
  <subrule('param')>  -- constant parameters only
 
  %hash
 
  <@var>             -- special-cased for array-of-rule (but not Rule|Str)
  \char              -- not all chars implemented
  {code}             -- non-capturing closure
                     -- perl5 syntax inside closure
                     -- $/ doesn't work yet
  { return code }    -- capturing closure
                     -- perl5 syntax inside closure
                     -- $/ works
  $var := (capture)  -- capture aliasing
  $<> $/<>           -- special variables can't be used inside a match yet
  $/ 
  $<0> $<1>
  $0 $1
  \n \N
  $^a $^b            -- positional parameters (must start with $^a)
  ^ $
  :
  $/ $()  - global match variables 
  $var := [non-capture]
  $var := <rule>
  <unicode-class> <+unicode-class> <+unicode-class+unicode-class>

= Unimplemented or untested features

  @var := [non-capture]
  @var := <rule>
  $variable 
  @variable
  $/<0> $/<1>
  $/0 $/1
  <"literal">
  ^^ $$
  <&var> 
  <%var>
  **{n..m}
  :: :::   (commit)
  <(return-object)> <{code-returns-rule}>
  <<character-class>> <[character-class]>
  :flag :flag() :flag[]
  \x0a \o123 ...
  &    
  $1      - lvalue match variables