openfoam/doc/changes/dynamicCode.org

#                            -*- mode: org; -*-
#
#+TITLE:           =dynamicCode=: Dynamic code compilation
#+AUTHOR:                      OpenCFD Ltd.
#+DATE:                            TBA
#+LINK:                  http://www.openfoam.com
#+OPTIONS: author:nil ^:{}
# Copyright (c) 2011 OpenCFD Ltd.

* Dictionary preprocessing directive: =#codeStream=
  This is a dictionary preprocessing directive ('=functionEntry=') which
  provides a snippet of OpenFOAM C++ code which gets compiled and executed to
  provide the actual dictionary entry. The snippet gets provided as three
  sections of C++ code which just gets inserted into a template:
  - =code= section: the actual body of the code. It gets called with arguments
    =OStream& os, const dictionary& dict= and the C++ code can do a
    =dict.lookup= to find current dictionary values.
  - optional =codeInclude= section: any #include statements to include OpenFOAM
    files.
  - optional 'codeOptions' section: any extra compilation flags to be added to
    =EXE_INC= in =Make/options=

  To ease inputting mulit-line code there is the =#{ #}= syntax. Anything in
  between these two delimiters becomes a string with all newlines, quotes etc
  preserved.

  Example: Look up dictionary entries and do some calculation
  #+BEGIN_SRC c++
    startTime       0;
    endTime         100;
    ..
    writeInterval   #codeStream
    {
        code
        #{
            scalar start = readScalar(dict["startTime"]);
            scalar end = readScalar(dict["endTime"]);
            label nDumps = 5;
            os  << ((end-start)/nDumps);
        #};
    };
  #+END_SRC

* Implementation
  - the =#codeStream= entry reads the dictionary following it, extracts the
    =code=, =codeInclude=, =codeOptions= sections (these are just strings) and
    calculates the SHA1 checksum of the contents.
  - it copies a template file
    =(~OpenFOAM/codeTemplates/dynamicCode/codeStreamTemplate.C)= or
    =($FOAM_CODE_TEMPLATES/codeStreamTemplate.C)=, substituting all
    occurences of =code=, =codeInclude=, =codeOptions=.
  - it writes library source files to =dynamicCode/<SHA1>= and compiles
    it using =wmake libso=.
  - the resulting library is generated under
    =dynamicCode/platforms/$WM_OPTIONS/lib= and is loaded (=dlopen=, =dlsym=)
    and the function executed
  - the function will have written its output into the Ostream which then gets
    used to construct the entry to replace the whole =#codeStream= section.
  - using the SHA1 means that same code will only be compiled and loaded once.

* Boundary condition: =codedFixedValue=
  This uses the code from codeStream to have an in-line specialised
  =fixedValueFvPatchScalarField=. For now only for scalars:
  #+BEGIN_SRC c++
  outlet
  {
      type            codedFixedValue;
      value           uniform 0;
      redirectType    fixedValue10;

      code
      #{
          operator==(min(10, 0.1*this->db().time().value()));
      #};
  }
  #+END_SRC
  It by default always includes =fvCFD.H= and adds the =finiteVolume= library to
  the include search path.

  A special form is where the code is not supplied in-line but instead comes
  from the =codeDict= dictionary in the =system= directory. It should contain
  a =fixedValue10= entry:
  #+BEGIN_SRC c++
  fixedValue10
  {
      code
      #{
          operator==(min(10, 0.1*this->db().time().value()));
      #};
  }
  #+END_SRC
  The advantage of using this indirect way is that it supports
  runTimeModifiable so any change of the code will be picked up next iteration.

* Security
  Allowing the case to execute C++ code does introduce security risks.  A
  third-party case might have a =#codeStream{#code system("rm -rf .");};= hidden
  somewhere in a dictionary.  =#codeStream= is therefore not enabled by default
  you have to enable it by setting in the system-wide =controlDict=
  #+BEGIN_SRC c++
  InfoSwitches
  {
      // Allow case-supplied c++ code (#codeStream, codedFixedValue)
      allowSystemOperations   1;
  }
  #+END_SRC

* Field manipulation
  Fields are read in as =IOdictionary= so can be upcast to provide access to the
  mesh:
  #+BEGIN_SRC c++
  internalField  #codeStream
  {
      codeInclude
      #{
          #include "fvCFD.H"
      #};

      code
      #{
          const IOdictionary& d = dynamicCast<const IOdictionary>(dict);
          const fvMesh& mesh = refCast<const fvMesh>(d.db());
          scalarField fld(mesh.nCells(), 12.34);
          fld.writeEntry("", os);
      #};

      codeOptions
      #{
          -I$(LIB_SRC)/finiteVolume/lnInclude
      #};
  };
  #+END_SRC

  Note: above field initialisation has the problem that the boundary conditions
  are not evaluated so e.g. processor boundaries will not hold the opposite cell
  value.

* Pitfalls
  The syntax of =#codeStream= can be quite hard to get right. These are some
  common pitfalls:
  + the =code= string has to be a valid set of C++ expressions so has to end in
    a ';'
  + the C++ code upon execution has to print a valid dictionary entry. In above example it
    prints 'uniform 12.34;'. Note the ';' at the end. It is advised to use the
    =writeEntry= as above to handle this and also e.g. binary streams (=codeStream=
    inherits the stream type from the dictionary)
  + the =code=, =codeInclude=, =codeOptions= entries are just like any other
    dictionary string entry so there has to be a ';' after the string
  + the =#codeStream= entry (itself a dictionary) has to end in a ';'

* Exceptions
  There are unfortunately some exceptions to above field massaging.
  Following applications read
  the field as a dictionary, not as an =IOdictionary=:
  - =foamFormatConvert=
  - =changeDictionaryDict=
  - =foamUpgradeCyclics=
  These applications will usually switch off all '#' processing which
  just preserves the entries as strings (including all formatting).

* Other
  - parallel running not tested a lot. What about distributed data
    (i.e. non-=NFS=) parallel?
  - paraview has been patched so it will pass in RTLD_GLOBAL when loading
    the OpenFOAM reader module. This is necessary for above dictionary
    processing to work.