Common problematic values are often unlikely to be generated

[shelbyd]

In the Readme, the "Limitations of Property Testing" section details that the any::<i64>() strategy is unlikely to produce i64::MIN. This, and other i64 constants, are usually far more likely to cause problems than an arbitrary i64.

I propose, at least for the built in Strategies, that they proactively generate common problematic values before generating arbitrary ones.

Some possibilities for the different types:

• integer: 0, -1, 1, MAX, MIN, MAX - 1, MIN + 1
• range: start, end, start + 1, start - 1
• float: 0, -0, INFINITY, NEG_INFINITY, MAX, MIN, MIN_POSITIVE, NAN
• string: "", common control characters, rtl unicode, others from the Big List of Naughty Strings
• slice: fewest elements, most elements

I'm happy to work on this. I'll just need a bit of direction in how to go about it.

[AltSysrq]

I'd certainly welcome any contributions!

integer, range

These all sound reasonable to me. Integers are defined in terms of ranges so instead of MIN and MAX, the range boundaries are probably the best option. This would just be a matter of changing how the starting value is generated (ANY is special but might not need to be anymore).

float

Some of these already happen (zeroes, infinites, NaNs), though any::<$float>() by design only includes real numbers since (in my experience at least) the majority of code handling floats isn't expected to do anything reasonable if handed infinity or NaN. Floats are otherwise handled similarly to integers and only the starting value would need to be changed.

slice

Similar to integer range handling, all that needs to change is the starting value

string ""

This would actually happen automatically with the slice change since regex repititions are defined in terms of the collection strategies.

common control characters

This should already be the case

BLNS

This is harder, since strings are generated in terms of regexes. Just setting the starting point won't work since some way would be needed to represent that starting point in terms of all the machinery underneath. It might be possible to extend the string strategy to have a chance of picking a BLNS string, check whether it matches the regex, and start from there, and then shrink to whatever regex-based string it would have generated otherwise.

[peterjoel]

I actually came here to report this exact issue. I could be wrong, but I was under the impression that Haskell's Quickcheck generates all "edge" cases for each type.

Another thing to consider is permutations of those edge cases. e.g., when generating values of a type:

Pair {
   a: i32,
   b: Option<f32>
}

This now has requires "minimal" edge cases of:

• a = 0
• a = 1
• a = -1
• a = i32::MIN
• a = i32::MAX
• a = MAX - 1
• a = MIN + 1

multiplied by:

• b = None
• b = Some(NaN)
• b = Some(NaN)
• b = Some(MIN)
• b = Some(MAX)
• b = Some(INFINITY)
• b = Some(NEG_INFINITY)
• b = Some(+0.0)
• b = Some(-0.0)
• b = Some(1.0)
• b = Some(-1.0)

For 7 * 11 = 77 "edge" combinations.

This can explode pretty fast!

[peterjoel]

float

... only includes real numbers since (in my experience at least) the majority of code handling floats isn't expected to do anything reasonable if handed infinity or NaN.

Even if code isn't expecting these values, it shouldn't panic or exhibit UB. Sometimes, you just forget that these values are possible. It would be much better to have these included by default but provide a convenient way to exclude them. That way, the first time a test fails due to a NEG_INFINITY you can consciously decide that this isn't a problem and switch to a more restrictive input.

[brendanzab]

I was doing roundtrip tests for a binary parsing library testing and thought that proptest would check NaNs early on in the sample space, but was surprised that it didn't fail on the naive proptest_assert! that I was initially using (because NaN != NaN).

I had a hard time tracking down what proptest was doing for NaNs - and it seems like it is only documented deep in the code - which doesn't show up to user-facing docs. It's be nice if the default state spaces were documented somewhere more prominently at least!

Thanks for the great library at any rate! It quickly found a copy-paste error I had in some of my low level parsing code, so it did its job well! 😍