Fortran WHERE construct unsupported

[MarkUoLeeds]

Hello,
processing some legacy FORTRAN (UKCA glomap) that has several (many) WHERE statements; there are instances which PSyclone cannot support. i.e. when an array (1:nbox) is coded without the array index subscript...
The example shown is declared as corrh(1:nbox), is occasionally written as corrh. I wrote a toy sample to demo the situation.
Also noted that the write_stmnt comment is interleaved with the WHERE unsupported report. Line 27 is the actual troublesome statement.
corrh_where_f90.txt

==== quoted ===
21 ! PSyclone CodeBlock (unsupported code) reason:
22 ! - Unsupported statement: Write_Stmt
23 ! - Only WHERE constructs using explicit array notation (e.g. my_array(:, :)) are supported but found
24 !& '[Level_4_Expr(Name('corrh'), '<', Real_Literal_Constant('1.0', None))]'.
25 ! - Unsupported statement: Write_Stmt
26 WRITE(6, '(20F5.2,2x)') corrh(1 : 20)
27 WHERE (corrh < 1.0) corrh = 1.05

=== end quoted ===

[MarkUoLeeds]

added Fortran as TXT due to github not letting me attach .F90
I was using the recommend omp_cpu_trans.py which seems to work well...

[LonelyCat124]

Thanks Mark, I think PSyclone should probably be able to handle this case without resorting to CodeBlocks since the variable declaration is clearly visible. I'll take a look.

[LonelyCat124]

We have a TODO in the code referencing #1799 but I'm not sure why it needs that to be implementable - I'll have a try.

[arporter]

It's because, in the absence of explicit array ranges, we need to be able to query (a possibly arbitrary) mask expression to get its shape. In this particular case though it's trivial as it's a single Reference. We do now support querying the type of expressions anyway so we can definitely do better.

[LonelyCat124]

Ah ok, I see the challenge and this can't work for all cases very easily I think, I need to look up exactly what a WHERE expressions rules are, but something like WHERE ( a < b ) ... is more difficult maybe? I need to work out exactly what the fparser2 code is doing.

@arporter how is best to query the type of expressions?

[arporter]

(In fact, this is a duplicate of #717.)

how is best to query the type of expressions?

Most Nodes now have the datatype method which will do this for you. (Call doesn't have this yet and possibly something else too - this is why #1799 isn't yet completed.)

[LonelyCat124]

I have a first implementation that fixes this for the case Mark posted. I need to extend it to work for structure accesses and fix some tests that now fail (because they don't expect this behaviour I think, or hit some assert comments I put in the draft code).

I need to add more complex tests as well to ensure this is behaving correctly.

[LonelyCat124]

Ok, just running the full test suite now - I've limited the ability to handle structures for now since that is a can of worms I'd like to avoid if possible, but it should now work for cases like the above, and also discovered (and fixed) another bug in that we weren't testing for.

[sergisiso]

I've limited the ability to handle structures for now since that is a can of worms I'd like to avoid if possible

My plan was to use Reference2ArrayReferences (which I seen that you started using in your PR) with #1858 fixed. Then it is not a fparser concern as after that transformation if its an array it is an ArrayReference, if it's scalar it is a Reference, if it cannot be decided the transformation does not validate.

(The problem WHERE has is that for this we need to resolve type information which in practice often lives in separate modules otherwise it does not validate most of the time, but fparser happens before we have the chance to ask psyir to explore other modules. This could be solved by telling fparser to follow dependencies first or by having a where-codeblock to psyir transformation later.)

[MarkUoLeeds]

hello,
just curious, how should I "update" my pscylone environment if pip thinks 2.5.0 is the up to date version.
Or is the fix for this still on a branch?
Thought I could test it for you...

[sergisiso]

Hey Mark, you may need to do a pip uninstall psyclone first.

Then what we typically do is get the lastest psyclone:

git clone git@github.com:stfc/PSyclone.git

and install that one from inside the folder:

cd PSyclone
pip install -e .

Note the "-e" (editable) flag, this allows you to do changes in your repository (e.g. pulling updated upstream versions, changing branches, ...) without needing to re-install the package with pip.

The fix for the WHERE issue is still under development, we will let you know when it is ready for you to give it a go.

[MarkUoLeeds]

Thanks! (not used to using pip installer) Dr. Mark Richardson Technical Head of CEMAC ( www cemac ac uk ) Room 9.145b Earth and Environment Building School of Earth and Environment University of Leeds LS2 9JT

…

________________________________ From: Sergi Siso ***@***.***> Sent: 16 August 2024 09:47 To: stfc/PSyclone ***@***.***> Cc: Mark Richardson ***@***.***>; Author ***@***.***> Subject: Re: [stfc/PSyclone] Fortran WHERE construct unsupported (Issue #2684) CAUTION: External Message. Use caution opening links and attachments. Hey Mark, you may need to do a pip uninstall psyclone first. Then what we typically do is get the lastest psyclone: git clone ***@***.***:stfc/PSyclone.git and install that one from inside the folder: cd PSyclone pip install -e . Note the "-e" (editable) flag, this allows you to do changes in your repository (e.g. pulling updated upstream versions, changing branches, ...) without needing to re-install the package with pip. The fix for the WHERE issue is still under development, we will let you know when it is ready for you to give it a go. — Reply to this email directly, view it on GitHub<#2684 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AGHT4VW5FENYBVP264BU67LZRW4C5AVCNFSM6AAAAABMABGJBSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTGEYDANBVGY>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[sergisiso]

Hey @MarkUoLeeds , I will merge the fix that @LonelyCat124 implemented and close this issue this afternoon. We fixed the WHERE constructs that you provided in the file in your first comment, psyclone now produced the following code for that file:

❯ psyclone corrh_where_f90.txt
program corrh_where
  integer, parameter :: nc = 20
  integer :: ii
  real, dimension(20) :: corrh
  integer :: widx1
  integer :: widx1_1

  corrh = 1.5
  ! PSyclone CodeBlock (unsupported code) reason:
  !  - Unsupported statement: Write_Stmt
  WRITE(6, '(20F5.2,2x)') corrh(1 : 20)
  do ii = 10, 12, 1
    corrh(ii) = -1.0
  enddo
  ! PSyclone CodeBlock (unsupported code) reason:
  !  - Unsupported statement: Write_Stmt
  WRITE(6, '(20F5.2,2x)') corrh(1 : 20)
  do widx1 = 1, 20, 1
    if (corrh(widx1) < 1.0) then
      corrh(widx1) = 1.05
    end if
  enddo
  ! PSyclone CodeBlock (unsupported code) reason:
  !  - Unsupported statement: Write_Stmt
  WRITE(6, '(20F5.2,2x)') corrh(1 : 20)
  do widx1_1 = 1, 20, 1
    if (corrh(widx1_1) < 1.4) then
      corrh(widx1_1) = 1.6
    end if
  enddo
  ! PSyclone CodeBlock (unsupported code) reason:
  !  - Unsupported statement: Write_Stmt
  WRITE(6, '(20F5.2,2x)') corrh(1 : 20)

end program corrh_where

However, note that not all WHEREs succeed yet, particularly when the array dimension declaration comes imported from another file. There should be a comment in the generated code explaining it if that's the case. If this affects your application let us know or feel free to open another issue.