Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1351 - Support for user-defined kernels that operate on DoFs #2556

Merged
merged 12 commits into from
May 13, 2024
75 changes: 73 additions & 2 deletions doc/user_guide/dynamo0p3.rst
Original file line number Diff line number Diff line change
Expand Up @@ -848,7 +848,7 @@ types.
an ``integer``-valued field as an argument.

.. _lfric-no-cma-mdata-rules:

Rules specific to General-Purpose Kernels without CMA Operators
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

arporter marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -973,6 +973,45 @@ on a ``CELL_COLUMN`` without CMA Operators. Specifically:

3) Stencil accesses are not permitted.

.. _lfric-dof-kernel-rules:

Rules for all User-Supplied Kernels that Operate on DoFs (DoF Kernels)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

DoF Kernels and :ref:`LFRic Built-ins<lfric-built-ins>` overlap significantly
in their scope, and the conventions that DoF Kernels must follow are influenced
by those for built-ins as a result. This includes :ref:`metadata arguments
<dynamo0.3-api-built-ins-metadata>` and :ref:`valid data types
and access modes<lfric-built-ins-dtype-access>`. Naming conventions for DoF
Kernels should follow those for General-Purpose Kernels.

The list of rules for DoF Kernels is as follows:

1) A DoF Kernel must have at least one argument that is a field.
This rule reflects that a Kernel operates on some subset of the
whole domain and is therefore designed to be called from within
a loop that iterates over those subsets of the domain. Only fields
arporter marked this conversation as resolved.
Show resolved Hide resolved
(as opposed to e.g. field vectors or operators) are accepted for DoF
Kernels because only they have a single value at each DoF.

2) All Kernel arguments must be either fields or scalars (`real-` and/or
`integer`-valued). DoF Kernels cannot accept operators.

3) All field arguments to a given DoF Kernel must be on the same function
space so they have the same number of DoFs.

4) They must have at least one modified (i.e. writen to) field argument. Unlike
built-ins, this is not limited and more than one modified argument is
allowed.

5) A Kernel may not write to a scalar argument. (Only built-ins are permitted
to do this.) Any scalar arguments must therefore be declared in the metadata
as `GH_READ` - see :ref:`below<dynamo0-3-kernel-valid-access>`
arporter marked this conversation as resolved.
Show resolved Hide resolved

6) Kernels must be written to operate on a single DoF, such that single DoFs
can be provided to the Kernel within a loop for assignment to an indexed
arporter marked this conversation as resolved.
Show resolved Hide resolved
location in a field.

.. _dynamo0.3-api-kernel-metadata:

Metadata
Expand Down Expand Up @@ -1277,7 +1316,7 @@ modes depend upon the argument type and the function space it is on:
| | | GH_READWRITE |
+------------------------+------------------------------+--------------------+
| GH_FIELD | Continuous | GH_READ, GH_WRITE, |
| | | GH_INC, GH_READINC |
| | | GH_INC, GH_READINC |
+------------------------+------------------------------+--------------------+
| GH_OPERATOR | Any for both 'to' and 'from' | GH_READ, GH_WRITE, |
| | | GH_READWRITE |
Expand Down Expand Up @@ -2426,6 +2465,38 @@ in the halo (``ncell_2d_no_halos``), must be passed in. This is provided
as the second argument to the kernel (after ``nlayers``).
``ncell_2d_no_halos`` is an ``integer`` of kind ``i_def`` with intent ``in``.

Rules for DoF Kernels
#####################

A DoF Kernel requires the shared index of the DoF(s) in the field(s) to operate
arporter marked this conversation as resolved.
Show resolved Hide resolved
on, and the field(s) to extract the DoF(s) from. The full set of rules, along
with PSyclone's naming conventions, are:

1) Include `df`, the index of the single dof to be operated on. This is an
``integer`` of of kind ``i_def`` with intent ``in``.

2) For each scalar/field in the order specified by the meta_args metadata:

1) If the current entry is a scalar quantity then include the Fortran
variable in the argument list. The intent is determined from the
metadata (see meta_args for an explanation).

2) If the current entry is a field then include the field array. The
field array name is currently specified as being ``"field_"
<argument_position>``. A field array is a rank-1, real array with
extent equal to the number of unique degrees of freedom for the space
that the field is on. Its precision (kind) depends on how it is
defined in the algorithm layer, see the :ref:`Mixed Precision
<lfric-mixed-precision>` section for more details. This value is
passed in separately. Again, the intent is determined from the
metadata (see :ref:`meta_args <dynamo0.3-api-meta-args>`).

3) Include the unique number of degrees of freedom for the function space.
This is an ``integer`` of kind ``i_def`` with intent ``in``. The name of
this argument is simply ``undf`` without a function space suffix (as for
general purpose kernels) since all fields will be on the same function
space.

.. _dynamo0.3-kernel-arg-intents:

Argument Intents
Expand Down
Loading