Added the `llvm_sym` backend for testing symbolic operations through the llvm backend #2192

anutosh491 · 2023-07-20T05:40:35Z

This Pr does the following .

Introduces the llvm_sym backend . Currently we are testing symbolics_07.py and we get

(lf) anutosh491@spbhat68:~/lpython/lpython/integration_tests$ ./run_tests.py -b llvm_sym
-- Configuring done
-- Generating done
-- Build files have been written to: /home/anutosh491/lpython/lpython/integration_tests/_lpython-tmp-test-llvm_sym
+ make -j8
[ 50%] Generating symbolics_07.o
[100%] Linking C executable symbolics_07
[100%] Built target symbolics_07
+ ctest -j8 --output-on-failure
Test project /home/anutosh491/lpython/lpython/integration_tests/_lpython-tmp-test-llvm_sym
    Start 1: symbolics_07
1/1 Test #1: symbolics_07 .....................   Passed    0.01 sec

100% tests passed, 0 tests failed out of 1

Label Time Summary:
c_sym          =   0.01 sec*proc (1 test)
cpython_sym    =   0.01 sec*proc (1 test)
llvm_sym       =   0.01 sec*proc (1 test)

Adds the --enable-symengine flag support for the llvm backend (as we have the for C backend)

(lf) anutosh491@spbhat68:~/lpython/lpython$ lpython --enable-symengine --backend=c integration_tests/symbolics_07.py
pi
(lf) anutosh491@spbhat68:~/lpython/lpython$ lpython --enable-symengine --backend=llvm integration_tests/symbolics_07.py
pi

anutosh491 · 2023-07-20T16:40:52Z

This PR would need rebasing after #2195 & #2196 would be merged. Then this would be ready to go !

certik · 2023-07-20T23:41:30Z

Not sure what is going on, but: https://github.com/lcompilers/lpython/actions/runs/5616901797/job/15220061785

-- Build files have been written to: /home/runner/work/lpython/lpython/integration_tests/_lpython-tmp-test-llvm_sym
Test project /home/runner/work/lpython/lpython/integration_tests/_lpython-tmp-test-llvm_sym
No tests were found!!!

it seems the LLVM symbolic tests are not being run.

Also the Release test failed somehow.

anutosh491 · 2023-07-21T02:38:52Z

Hello @certik . So the symbolic tests are running

-- Configuring done (1.4s)
-- Generating done (0.0s)
-- Build files have been written to: /home/runner/work/lpython/lpython/integration_tests/_lpython-tmp-test-llvm_sym
[ 50%] Generating symbolics_07.o
[100%] Linking C executable symbolics_07
[100%] Built target symbolics_07
Test project /home/runner/work/lpython/lpython/integration_tests/_lpython-tmp-test-llvm_sym
    Start 1: symbolics_07
1/1 Test #1: symbolics_07 .....................   Passed    0.01 sec

But while using the basic_new_heap() the test with the fast flag returns a SegFault

[ 50%] Generating symbolics_07_FAST.o
[100%] Linking C executable symbolics_07_FAST
[100%] Built target symbolics_07_FAST
+ ctest -j8 --output-on-failure
Test project /home/anutosh491/lpython/lpython/integration_tests/_lpython-tmp-test-llvm_sym
    Start 1: symbolics_07_FAST
1/1 Test #1: symbolics_07_FAST ................***Exception: SegFault  0.05 sec

As I was discussing this with @Shaikh-Ubaid (as we were not sure why this might be happening), we decided to disable the FAST flag for this as of now
RUN(NAME symbolics_07 LABELS cpython_sym c_sym llvm_sym NOFAST)

anutosh491 · 2023-07-21T02:42:18Z

Though the counterpart with basic_new_stack doesn't give us any issues (even with the fast flag)

def main0():
    y: i64
    x: CPtr
    p_c_pointer(pointer(y), x)
    basic_new_stack(x)
    basic_const_pi(x)
    s: str = basic_str(x)
    print(s)
    assert s == "pi"

main0()

I would be adding this example in the next pull request !

Also not sure why the Release test would fail here .

Shaikh-Ubaid · 2023-07-21T02:48:17Z

FAST version of the test case case be supported later. It does not seem to be a blocker. So, I think it is fine.

Shaikh-Ubaid · 2023-07-21T02:50:19Z

Also not sure why the Release test would fail here .

It seems falsely fail sometimes (and seems to fail more often now). See #2162. If you think it failed falsely, just rerun it for now.

Shaikh-Ubaid · 2023-07-21T02:52:37Z

By the way, with respect to this #2192 (comment), please could you share the ASR, C and possibly LLVM Outputs generated for FAST and non-FAST executions?

We can fix the FAST version later, but I am just curious to see the difference.

anutosh491 · 2023-07-21T03:07:28Z

Yes , would be good to have reference of ASR, C, LLVM.

ASR (same with and without fast flag)
https://gist.github.com/anutosh491/3b296181895d46afa89b070156f948a4
C (same with and without fast flag)

#include <inttypes.h>
#include "symengine/cwrapper.h"

#include <stdlib.h>
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
#include <lfortran_intrinsics.h>

void main0();
void __main____global_statements();



// Implementations
void main0()
{
    char * s = NULL;
    void* x;
    x = basic_new_heap();
    printf("%p\n", x);
    basic_const_pi(x);
    _lfortran_strcpy(&s, basic_str(x), 1);
    printf("%s\n", s);
    ASSERT(strcmp(s, "pi")  ==  0);
}

void __main____global_statements()
{
    main0();
}

int32_t O_RDONLY;
void _lpython_close(int64_t fd);

int64_t _lpython_open(char * path, char * flag);

char* _lpython_read(int64_t fd, int64_t n);

int main(int argc, char* argv[])
{
    _lpython_set_argv(argc, argv);
    __main____global_statements();
    return 0;
}

anutosh491 · 2023-07-21T03:08:51Z

LLVM ( is different with and without fast flag)

(lf) anutosh491@spbhat68:~/lpython/lpython$ lpython --show-llvm integration_tests/symbolics_07.py
; ModuleID = 'LFortran'
source_filename = "LFortran"

@0 = private unnamed_addr constant [2 x i8] c" \00", align 1
@1 = private unnamed_addr constant [2 x i8] c"\0A\00", align 1
@2 = private unnamed_addr constant [7 x i8] c"%lld%s\00", align 1
@3 = private unnamed_addr constant [2 x i8] c" \00", align 1
@4 = private unnamed_addr constant [2 x i8] c"\0A\00", align 1
@5 = private unnamed_addr constant [5 x i8] c"%s%s\00", align 1
@6 = private unnamed_addr constant [3 x i8] c"pi\00", align 1
@7 = private unnamed_addr constant [16 x i8] c"AssertionError\0A\00", align 1
@O_RDONLY = global i32 0

define void @__module___main_____main____global_statements() {
.entry:
  call void @__module___main___main0()
  br label %return

return:                                           ; preds = %.entry
  ret void
}

declare void @basic_const_pi(void*)

declare void* @basic_new_heap()

declare i8* @basic_str(void*)

define void @__module___main___main0() {
.entry:
  %s = alloca i8*, align 8
  store i8* null, i8** %s, align 8
  %x = alloca void*, align 8
  %0 = call void* @basic_new_heap()
  store void* %0, void** %x, align 8
  %1 = load void*, void** %x, align 8
  %2 = ptrtoint void* %1 to i64
  call void (i8*, ...) @_lfortran_printf(i8* getelementptr inbounds ([7 x i8], [7 x i8]* @2, i32 0, i32 0), i64 %2, i8* getelementptr inbounds ([2 x i8], [2 x i8]* @1, i32 0, i32 0))
  %3 = load void*, void** %x, align 8
  call void @basic_const_pi(void* %3)
  %4 = load void*, void** %x, align 8
  %5 = call i8* @basic_str(void* %4)
  call void @_lfortran_strcpy(i8** %s, i8* %5, i8 0)
  %6 = load i8*, i8** %s, align 8
  call void (i8*, ...) @_lfortran_printf(i8* getelementptr inbounds ([5 x i8], [5 x i8]* @5, i32 0, i32 0), i8* %6, i8* getelementptr inbounds ([2 x i8], [2 x i8]* @4, i32 0, i32 0))
  %7 = load i8*, i8** %s, align 8
  %8 = alloca i8*, align 8
  store i8* %7, i8** %8, align 8
  %9 = alloca i8*, align 8
  store i8* getelementptr inbounds ([3 x i8], [3 x i8]* @6, i32 0, i32 0), i8** %9, align 8
  %10 = call i1 @_lpython_str_compare_eq(i8** %8, i8** %9)
  br i1 %10, label %then, label %else

then:                                             ; preds = %.entry
  br label %ifcont

else:                                             ; preds = %.entry
  call void (i8*, ...) @_lcompilers_print_error(i8* getelementptr inbounds ([16 x i8], [16 x i8]* @7, i32 0, i32 0))
  call void @exit(i32 1)
  br label %ifcont

ifcont:                                           ; preds = %else, %then
  br label %return

return:                                           ; preds = %ifcont
  ret void
}

declare void @_lfortran_printf(i8*, ...)

declare void @_lfortran_strcpy(i8**, i8*, i8)

declare i1 @_lpython_str_compare_eq(i8**, i8**)

declare void @_lcompilers_print_error(i8*, ...)

declare void @exit(i32)

declare void @_lpython_close(i64)

declare i64 @_lpython_open(i8*, i8*)

declare i8* @_lpython_read(i64, i64)

define i32 @main(i32 %0, i8** %1) {
.entry:
  call void @_lpython_set_argv(i32 %0, i8** %1)
  call void @__module___main_____main____global_statements()
  ret i32 0
}

declare void @_lpython_set_argv(i32, i8**)

(lf) anutosh491@spbhat68:~/lpython/lpython$ lpython --show-llvm --fast integration_tests/symbolics_07.py
; ModuleID = 'LFortran'
source_filename = "LFortran"
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

@0 = private unnamed_addr constant [7 x i8] c"%lld%s\00", align 1
@1 = private unnamed_addr constant [2 x i8] c"\0A\00", align 1
@2 = private unnamed_addr constant [5 x i8] c"%s%s\00", align 1
@3 = private unnamed_addr constant [3 x i8] c"pi\00", align 1
@4 = private unnamed_addr constant [16 x i8] c"AssertionError\0A\00", align 1
@O_RDONLY = local_unnamed_addr global i32 0

define void @__module___main_____main____global_statements() local_unnamed_addr {
.entry:
  tail call void @__module___main___main0()
  ret void
}

declare void @basic_const_pi(void*) local_unnamed_addr

declare void* @basic_new_heap() local_unnamed_addr

declare i8* @basic_str(void*) local_unnamed_addr

define void @__module___main___main0() local_unnamed_addr {
.entry:
  %s = alloca i8*, align 8
  store i8* null, i8** %s, align 8
  %0 = tail call void* @basic_new_heap()
  tail call void (i8*, ...) @_lfortran_printf(i8* getelementptr inbounds ([7 x i8], [7 x i8]* @0, i64 0, i64 0), i64 undef, i8* getelementptr inbounds ([2 x i8], [2 x i8]* @1, i64 0, i64 0))
  tail call void @basic_const_pi(void* undef)
  %1 = tail call i8* @basic_str(void* undef)
  call void @_lfortran_strcpy(i8** nonnull %s, i8* %1, i8 0)
  %2 = load i8*, i8** %s, align 8
  call void (i8*, ...) @_lfortran_printf(i8* getelementptr inbounds ([5 x i8], [5 x i8]* @2, i64 0, i64 0), i8* %2, i8* getelementptr inbounds ([2 x i8], [2 x i8]* @1, i64 0, i64 0))
  %3 = bitcast i8** %s to i64*
  %4 = load i64, i64* %3, align 8
  %5 = alloca i8*, align 8
  %6 = bitcast i8** %5 to i64*
  store i64 %4, i64* %6, align 8
  %7 = alloca i8*, align 8
  store i8* getelementptr inbounds ([3 x i8], [3 x i8]* @3, i64 0, i64 0), i8** %7, align 8
  %8 = call i1 @_lpython_str_compare_eq(i8** nonnull %5, i8** nonnull %7)
  br i1 %8, label %return, label %else

else:                                             ; preds = %.entry
  call void (i8*, ...) @_lcompilers_print_error(i8* getelementptr inbounds ([16 x i8], [16 x i8]* @4, i64 0, i64 0))
  call void @exit(i32 1)
  br label %return

return:                                           ; preds = %else, %.entry
  ret void
}

declare void @_lfortran_printf(i8*, ...) local_unnamed_addr

declare void @_lfortran_strcpy(i8**, i8*, i8) local_unnamed_addr

declare i1 @_lpython_str_compare_eq(i8**, i8**) local_unnamed_addr

declare void @_lcompilers_print_error(i8*, ...) local_unnamed_addr

declare void @exit(i32) local_unnamed_addr

define i32 @main(i32 %0, i8** %1) local_unnamed_addr {
.entry:
  tail call void @_lpython_set_argv(i32 %0, i8** %1)
  tail call void @__module___main___main0()
  ret i32 0
}

declare void @_lpython_set_argv(i32, i8**) local_unnamed_addr

anutosh491 · 2023-07-21T03:16:12Z

Well I think what's happening in the LLVM with the fast flag enabled is the following . I think the CPtr object is being deallocated in intermediately in the program . A fix for this might be making x_global a global variable, the CPtr object will not be deallocated until the script terminates, and the segmentation fault issue should be resolved.

from lpython import ccall, CPtr
import os

x_global: CPtr

@ccall(header="symengine/cwrapper.h", c_shared_lib="symengine", c_shared_lib_path=f"{os.environ['CONDA_PREFIX']}/lib")
def basic_new_heap() -> CPtr:
    pass

@ccall(header="symengine/cwrapper.h", c_shared_lib="symengine", c_shared_lib_path=f"{os.environ['CONDA_PREFIX']}/lib")
def basic_const_pi(x: CPtr) -> None:
    pass

@ccall(header="symengine/cwrapper.h", c_shared_lib="symengine", c_shared_lib_path=f"{os.environ['CONDA_PREFIX']}/lib")
def basic_str(x: CPtr) -> str:
    pass

def main0():
    global x_global
    x_global = basic_new_heap()
    basic_const_pi(x_global)
    s: str = basic_str(x_global)
    print(s)
    assert s == "pi"

main0()

(lf) anutosh491@spbhat68:~/lpython/lpython$ lpython --enable-symengine integration_tests/symbolics_07.py
pi
(lf) anutosh491@spbhat68:~/lpython/lpython$ lpython --enable-symengine --fast integration_tests/symbolics_07.py
pi

certik force-pushed the GSoC_PR_8 branch 2 times, most recently from 018b3c2 to a5be540 Compare July 20, 2023 23:33

certik marked this pull request as draft July 20, 2023 23:42

certik marked this pull request as ready for review July 21, 2023 16:42

anutosh491 added 2 commits July 21, 2023 10:44

Added the llvm_sym backend

6967f8e

disabled fast flag for now

6dd1229

certik force-pushed the GSoC_PR_8 branch from a5be540 to 6dd1229 Compare July 21, 2023 16:44

certik enabled auto-merge July 21, 2023 16:46

certik merged commit 9a84918 into lcompilers:main Jul 21, 2023
9 checks passed

anutosh491 deleted the GSoC_PR_8 branch July 21, 2023 17:28

anutosh491 added the symbolic label Aug 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added the `llvm_sym` backend for testing symbolic operations through the llvm backend #2192

Added the `llvm_sym` backend for testing symbolic operations through the llvm backend #2192

anutosh491 commented Jul 20, 2023

anutosh491 commented Jul 20, 2023

certik commented Jul 20, 2023 •

edited

Loading

anutosh491 commented Jul 21, 2023

anutosh491 commented Jul 21, 2023 •

edited

Loading

Shaikh-Ubaid commented Jul 21, 2023

Shaikh-Ubaid commented Jul 21, 2023

Shaikh-Ubaid commented Jul 21, 2023

anutosh491 commented Jul 21, 2023 •

edited

Loading

anutosh491 commented Jul 21, 2023 •

edited

Loading

anutosh491 commented Jul 21, 2023

Added the llvm_sym backend for testing symbolic operations through the llvm backend #2192

Added the llvm_sym backend for testing symbolic operations through the llvm backend #2192

Conversation

anutosh491 commented Jul 20, 2023

anutosh491 commented Jul 20, 2023

certik commented Jul 20, 2023 • edited Loading

anutosh491 commented Jul 21, 2023

anutosh491 commented Jul 21, 2023 • edited Loading

Shaikh-Ubaid commented Jul 21, 2023

Shaikh-Ubaid commented Jul 21, 2023

Shaikh-Ubaid commented Jul 21, 2023

anutosh491 commented Jul 21, 2023 • edited Loading

anutosh491 commented Jul 21, 2023 • edited Loading

anutosh491 commented Jul 21, 2023

Added the `llvm_sym` backend for testing symbolic operations through the llvm backend #2192

Added the `llvm_sym` backend for testing symbolic operations through the llvm backend #2192

certik commented Jul 20, 2023 •

edited

Loading

anutosh491 commented Jul 21, 2023 •

edited

Loading

anutosh491 commented Jul 21, 2023 •

edited

Loading

anutosh491 commented Jul 21, 2023 •

edited

Loading