form crashes when .clear is in a procedure

[piisgaaf]

When form executes a .clear statement inside a procedure it crashes with the following output:

FORM 5.0.0-beta.1 (May 21 2024, v5.0.0-beta.1-66-g83e3d41)  Run: Sat Sep 14 10:31:36 2024
    #+
    #procedure LOOP()
    .clear;
    #endprocedure
    #call LOOP
form(87689,0x1f47b4f40) malloc: *** error for object 0x600003c200e0: pointer being freed was not allocated
form(87689,0x1f47b4f40) malloc: *** set a breakpoint in malloc_error_break to debug

The minimal program that illustrates this:

#+
#procedure LOOP()
.clear;
#endprocedure
#call LOOP
.end

[tueda]

Valgrind output

FORM 5.0.0-beta.1 (May 21 2024, v5.0.0-beta.1-66-g83e3d41)  Run: Sat Sep 14 20:10:07 2024
    #+
    #procedure LOOP()
    .clear;
    #endprocedure
    #call LOOP
==615792== Invalid free() / delete / delete[] / realloc()
==615792==    at 0x484C8BF: free (in /home/linuxbrew/.linuxbrew/Cellar/valgrind/3.23.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==615792==    by 0x239A43: M_free (tools.c:2449)
==615792==    by 0x18206D: FullCleanUp (module.c:323)
==615792==    by 0x1E03A4: PreProcessor (pre.c:1064)
==615792==    by 0x220A59: main (startup.c:1690)
==615792==  Address 0x51cf3b0 is 0 bytes inside a block of size 8 free'd
==615792==    at 0x484C8BF: free (in /home/linuxbrew/.linuxbrew/Cellar/valgrind/3.23.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==615792==    by 0x239A43: M_free (tools.c:2449)
==615792==    by 0x18206D: FullCleanUp (module.c:323)
==615792==    by 0x1E03A4: PreProcessor (pre.c:1064)
==615792==    by 0x220A59: main (startup.c:1690)
==615792==  Block was alloc'd at
==615792==    at 0x4849859: malloc (in /home/linuxbrew/.linuxbrew/Cellar/valgrind/3.23.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==615792==    by 0x2399B4: Malloc1 (tools.c:2322)
==615792==    by 0x239A04: strDup1 (tools.c:1908)
==615792==    by 0x1DBF50: DoProcedure (pre.c:4011)
==615792==    by 0x1DF598: PreProInstruction (pre.c:1233)
==615792==    by 0x1E01AC: PreProcessor (pre.c:1010)
==615792==    by 0x220A59: main (startup.c:1690)
==615792==
==615792== Invalid free() / delete / delete[] / realloc()
==615792==    at 0x484C8BF: free (in /home/linuxbrew/.linuxbrew/Cellar/valgrind/3.23.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==615792==    by 0x239A43: M_free (tools.c:2449)
==615792==    by 0x182088: FullCleanUp (module.c:324)
==615792==    by 0x1E03A4: PreProcessor (pre.c:1064)
==615792==    by 0x220A59: main (startup.c:1690)
==615792==  Address 0x51cf2e0 is 0 bytes inside a block of size 136 free'd
==615792==    at 0x484C8BF: free (in /home/linuxbrew/.linuxbrew/Cellar/valgrind/3.23.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==615792==    by 0x239A43: M_free (tools.c:2449)
==615792==    by 0x182088: FullCleanUp (module.c:324)
==615792==    by 0x1E03A4: PreProcessor (pre.c:1064)
==615792==    by 0x220A59: main (startup.c:1690)
==615792==  Block was alloc'd at
==615792==    at 0x4849859: malloc (in /home/linuxbrew/.linuxbrew/Cellar/valgrind/3.23.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==615792==    by 0x2399B4: Malloc1 (tools.c:2322)
==615792==    by 0x1DB883: PreLoad (pre.c:4282)
==615792==    by 0x1DBEAA: DoProcedure (pre.c:3998)
==615792==    by 0x1DF598: PreProInstruction (pre.c:1233)
==615792==    by 0x1E01AC: PreProcessor (pre.c:1010)
==615792==    by 0x220A59: main (startup.c:1690)
==615792==
    .end

[jodavies]

I suppose we are just after a sensible error message and a clean crash here; after all, .clear removes procedure definitions from earlier in the script.

[tueda]

If the procedure is in a .prc file, then somehow it doesn't cause this memory issue (lines after .clear inside the procedure are ignored; could be (ab)used as an "early return" from procedures).

LOOP.prc:

#procedure LOOP()
.clear
#endprocedure

main.frm:

#call LOOP
.end

[piisgaaf]

Just to give some background on how I use this now. FORM is started with the -pipe option. An external application is sending commands to FORM. When one of the commands is the .clear instruction then the FORM program terminates and the client is unable to send more commands.
An outline of the FORM program is:

#-
#setexternal `PIPE1_'
#message FORM started and waiting for input
#prompt READY
#procedure LOOP()
#fromexternal-
#call LOOP()
#endprocedure
#call LOOP()
.end

I also tried to put the #fromexternal in an indefinite loop, but the .clear statement ends the loop after which the FORM program reaches the .end and terminates.

[jodavies]

And you would like to send .clear so that you can "reset" everything and send new instructions? The problem is that .clear is also trying to erase procedure definitions. @tueda 's example works but not if you call LOOP twice in a row; then it segfaults.

To be honest, I have never used .clear in my own FORM scripts. I am not sure what the intended behaviour is here.

[piisgaaf]

Hi, I also never used it before ;-) until this weekend ;-)

I have created a simple FORM kernel for use in jupyter notebooks. In a notebook I have 2 chapters. Both chapters share the same FORM setup file. After performing FORM statements in the first chapter, the second chapter starts with a .clear statement to start fresh, without having to restart and reconnect to the FORM program.

As a workaround, I now just restart the FORM program. The solution with the clear statement would be a cleaner solution than to just restart it.