You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I just want to provide a little bit more context to explain why it can be an important issue:
I have benchmark suite which works about 20 hours per platform, it measures all individual aspects of solution and we run it before we commit each refactoring/change to ensure we didn't introduce any performance degradation.
Normally, after every coding day we run subset of this suite, and it then works for about 10 hours, overnight.
As per normal development process some bugs are time to time introduced and some benchmarks can fail.
And if this is the case - then you simply lost 1 day and should continue development without knowing impact of your last changes.
In situations when you literally hunting microseconds and bytes of allocations it really hurts.
In the near future I won't have time to investigate the problem, but feel free to do it and send a PR with a fix. I'll definitely find some time to review the PR.
Somewhere between version 0.12.1 and 0.13.1, when a Benchmark throws an exception the entire run is now aborted. Where I used to receive the message:
"No more Benchmark runs will be launched as NO measurements were obtained from the previous run!"
and the rest of the benchmarks continued to run, the entire suite now stops and reports the exception.
Edit The work around is to use 0.12.1.
The text was updated successfully, but these errors were encountered: