ENH: robuster lemon parsing

- previously simply reused the scan token, which works fine for
  non-nested tokenizations but becomes too fragile with nesting.

  Now changed to use tagged unions that can be copied about
  and still retain some rudimentary knowledge of their types,
  which can be manually triggered with a destroy() call.

- provide an 'identifier' non-terminal as an additional catch
  to avoid potential leakage on parsing failure.

- adjust lemon rules and infrastructure:

  - use %token to predefine standard tokens.
    Will reduce some noise on the generated headers by retaining the
    order on the initial token names.

  - Define BIT_NOT, internal token rename NOT -> LNOT

- handle non-terminal vector values.
  Support vector::x, vector::y and vector::z constants

- permit fieldExpr access to time().
  Probably not usable or useful for an '#eval' expression,
  but useful for a Function1.

- provisioning for hooks into function calls. Establishes token
  names for next commit(s).
This commit is contained in:
Mark Olesen
2021-11-19 12:06:32 +01:00
parent adbcd3a19f
commit e6697edb52
38 changed files with 6819 additions and 5812 deletions

File diff suppressed because it is too large Load Diff