Bench: Actually create different limits for each scenario

[alexsnaps]

So this is going to hurt:

We always created the bench "wrong", or rather we created dups of one identical Limit in each namespace, when it was more than 1 (i.e. 10 & 50). Sadly this meant that Limitador would only ever do one limit (with more variables and conditions tho)... So this fixes that, and as such exposes the O(n) problem I mentioned in our refinement call...

Bench result, compared as to when it was 1 limit, whereas this now are 10 limits to be tested against:

10 namespaces with 10 limits each with 10 conditions and 10 variables

In memory:

On disk:

With Redis:

As expected, Memory is the worst: O(n), while Redis, which does an MGET is the least impacted. While the operation is still O(n) within Redis, the network overhead ans (de)serialization dominates the latency here…

[eguzki]

That hurts, indeed.

[eguzki]

Does it make sense to document those perf benchmarks and upload to Github?

[eguzki]

I am still surprised with the disk performance. I was expecting much worse performance. I guess actual numbers depend on the disk properties.

[alexsnaps]

Does it make sense to document those perf benchmarks and upload to Github?

Eventually maybe, yes... but I'd rather fix that issue first tho

I am still surprised with the disk performance. I was expecting much worse performance. I guess actual numbers depend on the disk properties.

And... this! If we want to publish these, I think we need to have a well controlled environment. These are all from runs on my MacBook Pro M1, with Redis running within Docker. We probably want something more reliable than that to use as a bench.