Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation/tutorial for ortac-stm plugin #141

Merged
merged 1 commit into from
Oct 19, 2023

Conversation

n-osborne
Copy link
Collaborator

@n-osborne n-osborne commented Sep 20, 2023

This PR is based on #152 in order to have to latest warning messages.

Copy link
Contributor

@shym shym left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice first draft, well done! I’m looking forward to seeing the complete version 😄

Comment on lines 16 to 17
(with-accepted-exit-codes
0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that required? It’s the default behaviour, I think.

Now that [qcheck-stm] is able to generate an initial value for the type under
test and its model, we can turn our attention to the functions we will want to
test. Here is the example of the classic [set] function. You can see again the
[checks] clause. Here again, it will be ignored by the tool. Then there is the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, the checks clause will be tested to check the function did raise an exception.

@n-osborne n-osborne force-pushed the stm-plugin-documentation branch 8 times, most recently from 24858e3 to 5677f4f Compare October 11, 2023 09:50
@jmid
Copy link

jmid commented Oct 11, 2023

Thanks for doing this documentation. Writing nice documentation is a lot of work - so this is much appreciated! 🙏

I suggest toning the following a bit down:

{2 QCheck-STM own limitations}

Finally, some limitations come directly from the QCheck-STM. There is not much
to do here except from testing the skipped function with another tool.

First of all, no other tools have been released so this suggestion is not helpfui.

In

a function should have exactly one type [SUT] in its signature, and it should be as argument.

one could support 0 or 1 arguments without hitting any QCheck-STM limitations AFAIK.

More broadly, one can encode "multiple ts" in QCheck-STM, by, say letting the SUT type be a stack of actual ts.
That way

  • init_sut should initialize the stack
  • t-consuming operations pop ts from the stack (a precond can ensure that there is enough available)
  • t-producing operations push their result to the stack.

So this isn't a limitation of QCheck-STM as it is reasonably customizable.

The second one is that higher order functions are not supported yet.

In multicoretests src/lazy/stm_tests.ml contains an STM example of doing so.
It is thus a matter of emitting something along these lines from the plugin.

Let me stress again that putting in limitations to get a first version working is only natural. 💯 😃
I furthermore applaud this effort to document the limitations! 🎉

I object to listing these under QCheck-STM own limitations and I challenge the claim that There is not much to do.

To make this constructive, a suggestion could be to rewrite this to just say

Please note, the current release of the plugin has the following limitations. We expect to address these in the future.

@n-osborne
Copy link
Collaborator Author

Thanks for the review :-)

I was not completely happy with this part, now I have an idea on how to modify it!

@n-osborne n-osborne force-pushed the stm-plugin-documentation branch 3 times, most recently from 7649205 to 89b969d Compare October 13, 2023 08:36
@n-osborne n-osborne marked this pull request as ready for review October 13, 2023 08:52
@n-osborne n-osborne requested a review from shym October 13, 2023 08:55
@n-osborne n-osborne changed the title Documentaion/tutorial for ortac-stm plugin Documentation/tutorial for ortac-stm plugin Oct 13, 2023
@n-osborne n-osborne added this to the 0.1 milestone Oct 18, 2023
Copy link
Contributor

@shym shym left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads very nicely, thank you!

I’ve added 4 commits containing fixes and suggestions in my stm-plugin-documentation-plus branch. They are separate commits to help you review my proposed changes (the one for typos should really be reviewed with --color-words!) but they would be better squashed in after.
I’ve also noticed that example_ghost doesn’t contain what it is supposed to, I think.

Comment on lines 13 to 14
val incompatible_type : char -> string t -> bool
(*@ b = incompatible_type c t *)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example is called example_ghost.mli but this is not what it illustrates, so I’m a bit confused here. It was overwritten with the content of example_incompatible_type, it seems.

@n-osborne n-osborne force-pushed the stm-plugin-documentation branch 7 times, most recently from b668468 to 932262d Compare October 18, 2023 18:19
Documentation will always be up to date regarding error messages, as
long as tests are run before the documentation is build.
@shym shym merged commit 5ae8426 into ocaml-gospel:main Oct 19, 2023
2 checks passed
@shym
Copy link
Contributor

shym commented Oct 19, 2023

Thank you for the updates, LGTM. CI is happy, so I merged.

@n-osborne n-osborne deleted the stm-plugin-documentation branch October 19, 2023 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants