ENH: robuster lemon parsing

- previously simply reused the scan token, which works fine for
  non-nested tokenizations but becomes too fragile with nesting.

  Now changed to use tagged unions that can be copied about
  and still retain some rudimentary knowledge of their types,
  which can be manually triggered with a destroy() call.

- provide an 'identifier' non-terminal as an additional catch
  to avoid potential leakage on parsing failure.

- adjust lemon rules and infrastructure:

  - use %token to predefine standard tokens.
    Will reduce some noise on the generated headers by retaining the
    order on the initial token names.

  - Define BIT_NOT, internal token rename NOT -> LNOT

- handle non-terminal vector values.
  Support vector::x, vector::y and vector::z constants

- permit fieldExpr access to time().
  Probably not usable or useful for an '#eval' expression,
  but useful for a Function1.

- provisioning for hooks into function calls. Establishes token
  names for next commit(s).

src/OpenFOAM/expressions/fields/fieldExprParser.H

@@ -5,7 +5,7 @@
     \\  /    A nd           | www.openfoam.com
      \\/     M anipulation  |
 -------------------------------------------------------------------------------
-    Copyright (C) 2019 OpenCFD Ltd.
+    Copyright (C) 2019-2021 OpenCFD Ltd.
 -------------------------------------------------------------------------------
 License
     This file is part of OpenFOAM.
@@ -34,6 +34,7 @@ Description
 #ifndef expressions_fieldExprParser_H
 #define expressions_fieldExprParser_H
 
+#include "exprScanToken.H"
 #include "fieldExprFwd.H"
 
 namespace Foam
@@ -97,10 +98,14 @@ public:
         //- Stop parsing, freeing the allocated parser
         void stop();
 
-        //- Push token/value to parser
-        void parse(int tokenId, scanToken* tokenVal);
+        //- Push token type to parser with default token
+        void parse(int tokenId);
+
+        //- Push token type/value to parser
+        void parse(int tokenId, scanToken tok);
 };
 
+
 // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
 
 } // End namespace fieldExpr