-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
448 make public example #450
base: master
Are you sure you want to change the base?
Changes from all commits
c16e073
a012aed
c42f398
982ca17
de7612f
c0bb76b
f43777e
a1189f7
3fd9603
acd2da2
7e07821
4e033b1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,149 @@ | ||
.. -*- rest -*- | ||
|
||
.. | ||
Copyright (c) 2024 Science and Technology Facilities Council. | ||
|
||
All rights reserved. | ||
|
||
Modifications made as part of the fparser project are distributed | ||
under the following license: | ||
|
||
Redistribution and use in source and binary forms, with or without | ||
modification, are permitted provided that the following conditions are | ||
met: | ||
|
||
1. Redistributions of source code must retain the above copyright | ||
notice, this list of conditions and the following disclaimer. | ||
|
||
2. Redistributions in binary form must reproduce the above copyright | ||
notice, this list of conditions and the following disclaimer in the | ||
documentation and/or other materials provided with the distribution. | ||
|
||
3. Neither the name of the copyright holder nor the names of its | ||
contributors may be used to endorse or promote products derived from | ||
this software without specific prior written permission. | ||
|
||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | ||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | ||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | ||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | ||
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | ||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | ||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | ||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | ||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | ||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
|
||
|
||
.. _examples: | ||
|
||
Examples | ||
======== | ||
|
||
The distribution comes with a set of examples showing how fparser can | ||
be used. At the same time, some of these examples are actually useful tools | ||
that are used in other projects. | ||
|
||
|
||
fparser2_f2008.py | ||
^^^^^^^^^^^^^^^^^ | ||
This is a very small example code that shows how a Fortran code, given as | ||
a string, is parsed, and then converted back into Fortran. It just prints | ||
out the re-created Fortran source code. | ||
|
||
|
||
create_dependencies.py | ||
^^^^^^^^^^^^^^^^^^^^^^ | ||
This file analyses the dependencies between a set of Fortran files, based on | ||
the ``Use`` statements in each file. It assumes that the module name in the | ||
``use`` statement corresponds to the name of the file (adding one of | ||
.F90/.f90/.x90). Only files in the current directory will be tested, so | ||
external dependencies will not be listed. Its output | ||
is in a format that can be immediately used in a Makefile:: | ||
|
||
../create_dependencies.py *.f90 | ||
b.o: a.o | ||
c.o: a.o b.o | ||
|
||
|
||
Ignoring error handling, the simplified main part of this code | ||
that is related to fparser is:: | ||
|
||
reader = FortranFileReader(filename) | ||
parser = ParserFactory().create(std="f2003") | ||
parse_tree = parser(reader) | ||
|
||
# Collect all used modules in a list | ||
all_use = set() | ||
for node in walk(parse_tree, Use_Stmt): | ||
use_name = str(node.items[2]) | ||
# A more sophisticated mapping could be used here. | ||
# But for now just assume that the name in the use statement | ||
# with an added ".o" is the required object file: | ||
obj_dependency = use_name + ".o" | ||
all_use.add(use_name) | ||
# Now ``all_use`` contains all .o files that ``filename`` depends on | ||
|
||
The rest of the example is related to creating the proper format for | ||
a Makefile. | ||
|
||
.. note:: It would be straight-forward to loop over all files twice, first | ||
to collect the module names provided by each file, then use this | ||
information | ||
|
||
|
||
make_public.py | ||
^^^^^^^^^^^^^^ | ||
This example removes all ``private`` and ``protected`` attributes in any | ||
declaration. In general these attributes are important and should | ||
obviously not be removed in order to give the compiler more information | ||
about intended use of the variables. But `PSyclone | ||
<https://github.com/stfc/PSyclone>`_ offers a feature called | ||
Kernel Extraction, which automatically writes all variables read and written | ||
in a code section to a file. It also then creates a stand-alone driver program | ||
that will read this file, execute the kernel, and compare the results with | ||
the original results. | ||
|
||
Since PSyclone will follow the call tree, the code must be able to read even | ||
variables declared as ``private`` (to write them into the output file), and | ||
a driver program must be able to modify ``private`` and ``protected`` | ||
variables in modules. If the driver creation is used, the | ||
`Fab <https://github.com/MetOffice/fab>`_ based build system will | ||
remove all ``private`` and ``protected`` attributes in a separate build phase, | ||
so that the kernel extraction and driver creation works as expected. | ||
|
||
A short example, which shows how a ``Access_Stmt`` like ``private :: a`` is | ||
removed:: | ||
|
||
for node in walk(parse_tree, Access_Stmt): | ||
if node.items[0] == "PRIVATE": | ||
# Find the node in the parent, and remove it: | ||
node.parent.children.remove(node) | ||
|
||
|
||
Modifying some of the fparser data structures can be more difficult, since | ||
they are often based on Python tuples, which cannot be modified. The following | ||
example from ``make_public.py`` shows how the middle element of a three-element | ||
tuple is replaced with None:: | ||
|
||
type_decl.items = (type_decl.items[0], None, type_decl.items[2]) | ||
|
||
|
||
split_file.py | ||
~~~~~~~~~~~~~ | ||
This script splits one Fortran source file into several files, each containing | ||
one top level module, subroutine, function or program. Each file uses the name | ||
of the program unit (module-, subroutine-, function-, program name). The | ||
extension will be ``.F90`` if there are preprocessor directives in the file, | ||
and ``.f90`` otherwise. | ||
|
||
Additionally, ``split_file.py`` will create a Makefile to build either the | ||
binary (if a program is found in the file), or all object files. If any of | ||
the environment variables ``F90``, ``F90FLAGS``, and ``LDFLAGS`` are set at | ||
run time of the script, it will use these values as default values in the | ||
makefile. But by setting these environment variables when running ``make``, | ||
these defaults can always be overwritten. The Makefile also has a ``clean`` | ||
target, which will remove all ``.mod``, object, and the program file (if | ||
available). It uses the ``create_dependencies.py`` script to add the | ||
required dependencies to the Makefile. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -79,5 +79,6 @@ Welcome to fparser's documentation! | |
introduction | ||
fparser | ||
fparser2 | ||
examples | ||
developers_guide | ||
reference_guide |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -33,17 +33,25 @@ | |
# ------------------------------------------------------------------------------ | ||
# Author: J. Henrichs, Bureau of Meteorology | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Comment please, e.g. "Makefile driver to test the various examples" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||
.PHONY: test create_dependency fparser2_f2008 | ||
# A simple Makefile driver to test the various examples. | ||
|
||
test: create_dependencies split_file fparser2_f2008 | ||
|
||
.PHONY: test create_dependency fparser2_f2008 make_public split_file | ||
|
||
test: create_dependencies fparser2_f2008 make_public split_file | ||
|
||
create_dependencies: | ||
(cd test_files/create_dependencies; ../../create_dependencies.py *.f90) | ||
(cd test_files/create_dependencies; \ | ||
../../create_dependencies.py *.f90 | diff correct_dependencies -) | ||
|
||
fparser2_f2008: | ||
python ./fparser2_f2008.py | ||
|
||
make_public: | ||
./make_public.py test_files/make_public.f90 | diff test_files/make_public_correct.f90 - | ||
|
||
split_file: | ||
(cd test_files/split_file; \ | ||
rm -f Makefile func.f90 mod1.f90 sub.f90 test_prog.f90; \ | ||
../../split_file.py test.f90) | ||
|
||
fparser2_f2008: | ||
python ./fparser2_f2008.py | ||
../../split_file.py test.f90; \ | ||
cat mod1.f90 sub.f90 func.f90 test_prog.f90 | diff -B test.f90 -) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,112 @@ | ||
#!/usr/bin/env python | ||
|
||
# ----------------------------------------------------------------------------- | ||
# BSD 3-Clause License | ||
# | ||
# Copyright (c) 2024, Science and Technology Facilities Council. | ||
# All rights reserved. | ||
# | ||
# Redistribution and use in source and binary forms, with or without | ||
# modification, are permitted provided that the following conditions are met: | ||
# | ||
# * Redistributions of source code must retain the above copyright notice, this | ||
# list of conditions and the following disclaimer. | ||
# | ||
# * Redistributions in binary form must reproduce the above copyright notice, | ||
# this list of conditions and the following disclaimer in the documentation | ||
# and/or other materials provided with the distribution. | ||
# | ||
# * Neither the name of the copyright holder nor the names of its | ||
# contributors may be used to endorse or promote products derived from | ||
# this software without specific prior written permission. | ||
# | ||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | ||
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | ||
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS | ||
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE | ||
# COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, | ||
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, | ||
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | ||
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER | ||
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT | ||
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN | ||
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE | ||
# POSSIBILITY OF SUCH DAMAGE. | ||
# ------------------------------------------------------------------------------ | ||
# Author: Joerg Henrichs, Bureau of Meteorology | ||
|
||
"""This file contains a script that will remove any private or protected | ||
declaration in a Fortran file. This is used by PSyclone's kernel extraction | ||
and driver creation feature, see | ||
https://psyclone.readthedocs.io/en/latest/psyke.html. | ||
for details. | ||
|
||
Usage: make_public.py file1.f90 | ||
""" | ||
|
||
import sys | ||
|
||
from fparser.common.readfortran import FortranFileReader | ||
from fparser.two.Fortran2003 import ( | ||
Access_Stmt, | ||
Access_Spec, | ||
Attr_Spec, | ||
Binding_Private_Stmt, | ||
Private_Components_Stmt, | ||
Protected_Stmt, | ||
) | ||
from fparser.two.parser import ParserFactory | ||
from fparser.two.utils import walk | ||
|
||
|
||
# ----------------------------------------------------------------------------- | ||
def remove_private(filename): | ||
"""Simple function that removes all private and protected declarations. | ||
:param str filename: the file in which to remove private and protected | ||
""" | ||
reader = FortranFileReader(filename) | ||
parser = ParserFactory().create(std="f2008") | ||
parse_tree = parser(reader) | ||
# A useful print to see the actual rules | ||
# print(repr(parse_tree)) | ||
|
||
# Loop over all access and protected statements. Note that a | ||
# `protected_stmt` is not an access statement, so it needs to | ||
# be listed additionally: | ||
for node in walk( | ||
parse_tree, | ||
(Access_Stmt, Protected_Stmt, Private_Components_Stmt, Binding_Private_Stmt), | ||
): | ||
# A Private_Components_Stms has no items: | ||
if isinstance( | ||
node, (Binding_Private_Stmt, Private_Components_Stmt) | ||
) or node.items[0] in ["PRIVATE", "PROTECTED"]: | ||
# Find the node in the parent, and remove it: | ||
node.parent.children.remove(node) | ||
|
||
for node in walk(parse_tree, Access_Spec): | ||
if str(node) == "PRIVATE": | ||
node.string = "PUBLIC" | ||
|
||
all_nodes = list(walk(parse_tree, Attr_Spec)) | ||
for node in all_nodes: | ||
if str(node) == "PROTECTED": | ||
# This is a tuple, so we can't simply remove the attribute | ||
node.parent.items = tuple(i for i in node.parent.items if i is not node) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's neat! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure tbh - given that modifying the tree is quite common task (I'd say), it's strange that we have tuples everywhere :( |
||
# If all items in the Attr_Spec are removed, we need to replace the | ||
# Attr_Spec in the parent-parent (Type_Declaration) with None, | ||
# otherwise fparser will create e.g. `real, :: a` | ||
if len(node.parent.items) == 0: | ||
# Again all tuples, which we can't modify, so we need to | ||
# recreate the tuple but replace the attr_spec with None | ||
type_decl = node.parent.parent | ||
type_decl.items = (type_decl.items[0], None, type_decl.items[2]) | ||
|
||
return parse_tree | ||
|
||
|
||
# ----------------------------------------------------------------------------- | ||
if __name__ == "__main__": | ||
user_filename = sys.argv[1] | ||
modified_parse_tree = remove_private(user_filename) | ||
print(modified_parse_tree) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
b.o: a.o | ||
c.o: a.o b.o |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good stuff, thanks for adding this Joerg.