Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Juvix testing framework #3033

Open
lukaszcz opened this issue Sep 13, 2024 · 2 comments
Open

Juvix testing framework #3033

lukaszcz opened this issue Sep 13, 2024 · 2 comments
Assignees
Milestone

Comments

@lukaszcz
Copy link
Collaborator

lukaszcz commented Sep 13, 2024

Add a first version of the testing framework discussed at HHH2024.Q3.

Add testing pragmas which specify that a function is a test and how the test should be executed. For now, let's have three possibilities.

  1. test: eval indicates that the zero-argument boolean property function should be evaluated with the Core evaluator and the result compared with true.
    Example:
module Tests;
  {-# test: eval #-}
  prop-reverseReverseIsIdentity : Bool :=
    let xs := [1; 2; 3; 4; 5] in
    xs == reverse (reverse xs);
end;
  1. test-cases: ident indicates that the property function should be tested for the argument values specified by Juvix identifier ident.
    Example:
module Tests;
  cases-reverseReverseIsIdentity : List (List Int)) := [
    [1; 2; 3; 4; 5;];
    [2; 4; 9; 8; 7];
    [1; 90; 9];
    []
  ];

  {-# test-cases: cases-reverseReverseIsIdentity #-}
  prop-reverseReverseIsIdentity (xs : List Int) : Bool :=
    xs == reverse (reverse xs);
end;
  1. test: rand specifies that the juvix-quickcheck library should be used to generate random test cases
    Example:
module Tests;
  {-# test: rand #-}
  prop-reverseReverseIsIdentity (xs : List Int) : Bool :=
    xs == reverse (reverse xs);
end;

The command juvix test would collect all tests declared in all modules in the project and run them according to the test pragmas. It would take care under the hood of all details like setting the random seed and using juvix-quickcheck correctly.

@lukaszcz lukaszcz added this to the 0.6.8 milestone Sep 13, 2024
@lukaszcz lukaszcz self-assigned this Sep 13, 2024
@janmasrovira
Copy link
Collaborator

I think we should replace (if possible) the pragmas by Juvix code. Even if the available options in this proposal are minimal, they are bound to grow. So we should take advantage of all the Juvix facilities that we have already implemented, such as default arguments, go to definition, documentation on mouse over, etc.

@lukaszcz
Copy link
Collaborator Author

Better alternative: have builtin data structure(s) that represent tests, e.g., similar to how Tasty does it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants