Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCaml backend #148

Draft
wants to merge 55 commits into
base: master
Choose a base branch
from
Draft

OCaml backend #148

wants to merge 55 commits into from

Conversation

mdurero
Copy link
Collaborator

@mdurero mdurero commented May 25, 2022

No description provided.

match line_list with
| [] -> List.rev (List.map List.rev line_list_list)
| line :: tail ->
if String.starts_with ~prefix:"#" line then
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

String.starts_with comes with ocaml 4.13, Mlang aims to be compatible with 4.11.2 (cf. the .opam file).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be fixed by using the parser from the test_framework in the main mlang codebase, instead of doing the same job badly and buggyly.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The backend is now usable with compiler 4.11.2.

I have copied selected parts of Mlang internal parser and test parser in order to obtain a independent parser for DGFiP test format. It is incomplete (as it was in Mlang test), but I think it would be a good idea to extend it.

@mdurero
Copy link
Collaborator Author

mdurero commented Sep 8, 2022

Native compilation of the generated code file causes a stack overflow, due to multiple huge dynamic data structures for entries and outputs. As a first approach, we could modify generated code to build the corresponding static arrays as the first step, then apply relevant transformation to compute the dynamic parts and convert to the target data structure.

Update : building purely static array of the variable positions then transforming it to values array or to a position map seems to be enough to solve the overflow.
The initialisation of the whole variable table tgv at undefined values line by line was useless and also resulting in stack overflow, this is solved by PR #184.

else floor (x.value +. 0.50005));
}

let m_null = m_not
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Habile

if bound >= 1 then
multimax variable_array (position + 1) (position + bound)
(get_position_value position)
else get_position_value position
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if would not be needed if the exit condition above was more robust.
Hint: you can write the whole thing more concisely with Array.sub and Array.fold.

I don't doubt the current implementation is correct, mind.

m_max reference (get_position_value current_index)
in
if current_index = max_index then new_max
else multimax variable_array (current_index + 1) max_index new_max
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine until you call multimax with a start index greater than the max index.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically multimax only initial call was after the bound ≥ 1 check, so start index being postion+1 and max index position+bound, having start index greater than max index shouldn't have been possible. But I'm trying an Array.fold_left based implementation.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's my point above: you shouldn't need the >= 1 check with a stronger exit condition. But all this won't matter with a fold anyway.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. Well I don't know how, but I manage to break it with the fold…

@Keryan-dev
Copy link
Collaborator

I'd like to have #184 merged to have a more clean diff on this PR for reviewing. @denismerigoux ?

As an early thought: parts of it rely a lot on makefiles whereas we could use dune for a less tedious build system.
On a related note, we could have the test parser exposed as a library for both the compiler and the harness, avoiding code duplication (and possibly more extensive features and tooling in the future).

But it works as it is, so we can just keep that in mind for later patches.

@Keryan-dev
Copy link
Collaborator

Also could you add this backend to the CI ?

@mdurero
Copy link
Collaborator Author

mdurero commented Oct 3, 2022

Making the test parser an independent library and fleshing it out to support all our test files is precisely my plan, if I'm not asked to prioritise something else.
I would like to go farther but at the moment I don't clearly see the external uses we could have for this tool (for instance a test-validator to automate syntax check when test files are delivered by our business owners).

As for adding the backend to the CI, I have this in my Git stash since I don't know when, applying it when I need it. At that time I was squeamish about modifying Mlang code base so I never committed the changes on the main Makefile. Just have to check if I use custom test files because I had to tailored some of them to test M errors when I added them.

@denismerigoux
Copy link
Contributor

#184 merged

Copy link
Collaborator

@Keryan-dev Keryan-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments, but it seems good to merge.

There are a few points of improvement to make which I already discussed a while back with @MdrditArthur. But it's not important enough to delay the merge any further IMO.

let bindings_list = Mir.IndexMap.bindings es in
Format.pp_print_list ~pp_sep:pp_statement_separator
(fun fmt (i, v) ->
generate_one_var (get_var_pos variable |> ( + ) i) fmt v)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it certainly is one way to write this 😀

let cond_name =
Format.asprintf "cond_%s_%d_%d_%d_%d" fname (Pos.get_start_line pos)
(Pos.get_start_column pos) (Pos.get_end_line pos)
(Pos.get_end_column pos)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this for debugging purposes ? Seems a bit overkill for a shadowable name.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say: "probably close to how it was done in other backends" and the only use I picture for it in the foreseeable future is helping generated code readability.

{undefined = false ; value = entry_var.value} in@,\
List.iter init_tgv_var entry_list" pp_print_position_map input_vars
(* Prévoir les cas : variable manquante, variable en trop dans entry_list,
variable définie n fois*)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it done ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. No it isn't. Seems like all tests are kindly providing well built entry lists. Not so obvious a check to add, though.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't bother, just put a comment stating we don't check those cases.

mdurero added 28 commits July 6, 2023 13:50
…ingle FIP file or all files from a test directory.
…n the output variable list, generally because it's an input variable, so it is pointless to mark it as output.
… calculate_tax function. Test command "raw" now writes also the list of triggered M errors.
…pec file, .mpp file and main MPP function, to run test suite with bytecode and native code.
…variable position instead of computing values or map line by line
…vant code from Mlang main and interpreter parsers
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants