Add support for async fn in proptest!

[johnchildren]

Adds cases for asynchronous functions with the proptest macros and a
test case using async-std.

Should work for both tokio::test and async_std::test attributes.

Support for async closures not added as they are not available
in the latest stable rust (I think?), but they are not supported
by the test attributes currently regardless.

---

As this PR adds an extra dev dependency I'm willing to look at alternative
ways to test this change as the test is more for syntax than for functionality.

Should resolve #179

[johnchildren]

Ah unfortunately this obviously wouldn't work on 1.33, potentially could be feature gated but that seems undesirable.

[dirvine]

Is there any update on this? It seems essential to use proptest in async code.

[johnchildren]

I think there were some problems with a dependency, but now the PR should hopefully pass CI. I wonder if this should be gated behind a feature flag to continue to allow people with pre-1.39 rust versions to use this crate? I could make that change if necessary.

[johnchildren]

XAMPPRocky/remove_dir_all#26 looks like this is the reason it can't compile, I can bump rustc to 1.40, which should work, but it seems like quite a big departure for the project.

edit: alternatively I can just not include the test?

[danburkert]

IIUC this PR doesn't actually bump the MSRV for users of proptest, it only bumps it for tests. If retaining the MSRV is a requirement there are workarounds available for enabling tests when the rust version is high enough (although they are all hacks). Regardless, would love to see this land!

[danburkert]

In other words - all of the non-test changes are in macros, so this new functionality is 'opt-in' for downstream users of proptest.

[danburkert]

I think it should be possible to do this without the duplication by matching an optional async keyword via $( async )?, then outputting it in the same way.

[danburkert]

nevermind that's not a thing

[danburkert]

If you add an await call into the body this will fail to compile, even something like async { }.await.

[johnchildren]

From expanding the macro it looks like this is not actually adding the async keyword in front of the function, but I'm not sure why. Annoying that I didn't spot this before.

[danburkert]

Yeah I spent a bit of time trying to figure out how to fix, but I'm not familiar with that code so couldn't puzzle it out. seems like it should be straightforward, but macros are always harder than you think they'll be :)

[fmorency]

I'm curious to know if there are any updates on this?

Thanks a lot for the crate and for this PR!

[johnchildren]

I've been looking at this PR again and I'm not sure it's possible to do without writing a lot of new capabilities for the runner to be able to handle async tests; I think my original idea was a bit naive and not really viable.

I think if someone wanted to implement this feature the best way would be to add an async feature in a similar way to the fork feature.

Possibly this PR is just worth closing?

[bhoudebert]

To me it is fairly one deal breaker of proptest to not having async.

You can obviously write in each test an ugly block async but it is inconvenient and breaks partially RLS ...

[hcsch]

I'm not sure that adding async support to the runner would be the ideal solution. With tokio at least, users might want to start a test with the time paused or a different runtime flavour. Async support could be added somewhat easily by adding something like the following to proptest_helper!

(@_ASYNC_BODY $config:ident ($($parm:pat in $strategy:expr),+) $(tokio_test_args ($($tokio_test_arg:tt)*))? [$($mod:tt)*] $body:expr) => {{
    #[tokio::main$(($($tokio_test_arg)*))?]
    async fn test_fn() -> Result<(), $crate::test_runner::TestCaseError> {
        let _: () = $body;
        Ok(())
    }

    $config.source_file = Some(file!());
    let mut runner = $crate::test_runner::TestRunner::new($config);
    let names = $crate::proptest_helper!(@_WRAPSTR ($($parm),*));
    match runner.run(
        &$crate::strategy::Strategy::prop_map(
            $crate::proptest_helper!(@_WRAP ($($strategy)*)),
            |values| $crate::sugar::NamedArguments(names, values),
        ),
        $($mod)* |
            $crate::sugar::NamedArguments(
            _,
            $crate::proptest_helper!(@_WRAPPAT ($($parm),*)))
        | {
            test_fn()
        },
    )
    {
        Ok(_) => (),
        Err(e) => panic!("{}\n{}", e, runner),
    }
}};

This should work for #[tokio::test], because #[tokio::main] happens to take the exact same parameters (e.g. including start_paused). This rightfully feels a bit hacky though and might break if that ever changes.
Another issue with this is that it's pretty painful to impossible at the moment to extract a particular attribute from a sequence of attributes in proptest!, because of rust decl macros' inability to backtrack causing a local ambiguity. Maybe we can get something working for the case in which the #[tokio::test] is the very first attribute, but I haven't even been able to get that working. That issue could probably be fixed by using a proc macro for proptest! instead though.

For the record: The reason just keeping the #[tokio::test] on the final test function didn't work in the current PR code is that we need the async body to run in the regular closure given to the test runner. The outer function the test runner is run in doesn't magically make our closure passed to the runner async too, alas. Which is why in the snippet above, we have another intermediary function that spawns the runtime and and manages the async stuff in the regular closure (in this case with #[tokio::main] creating the runtime and making a regular function out of an async one for us).

[LAC-Tech]

For anyone else waiting on this to be merged, below package seems to work to combine tokio with proptest:

https://github.com/frozenlib/test-strategy

[matthew-russo]

This got closed when i switched branch from master to main. I've added a link to our tracking issue for async support to incorporate any details from this