Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disk storage using RocksDB #178

Merged
merged 5 commits into from
May 17, 2023
Merged

Disk storage using RocksDB #178

merged 5 commits into from
May 17, 2023

Conversation

alexsnaps
Copy link
Member

@alexsnaps alexsnaps commented May 12, 2023

This supersedes #146, #167 and #170 - while keeping this broken down in a few distinct steps, including the Sled storage "for the record":

use limitador::storage::CounterStorage;
use limitador::RateLimiter;
use rand::SeedableRng;
use std::collections::HashMap;
use std::fmt::{Display, Formatter};
use tempdir::TempDir;

const SEED: u64 = 42;

#[cfg(not(feature = "redis"))]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we have the same for disk_storage ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean here…

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh you mean add it as a feature? 🤔
Unsure… what do you think the value would be? disk is "self sufficient" in the sense that it doesn't require an additional service... Now, we could do it to keep compile time and size of project actually using it as a dependency (crate) - but those don't exist as of now…
Anyways, absolute willing to add it as one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's a feature already, because of wasm 🤦
So, yeah, confused… Do you mean skip the bench_disk when disk_storage is not enabled? Cause yeah, right now to run cargo bench. I had added redis to be skipped, because of the external dependency… Which in this case isn't a thing… I'm on a fence, as I'd rather have it tested and any regression being caught, as again, it's self sufficient. But … why not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I misunderstood the meaning of

#[cfg(not(feature = "redis"))]                                   
criterion_group!(benches, bench_in_mem, bench_disk);             
#[cfg(feature = "redis")]                                        
criterion_group!(benches, bench_in_mem, bench_disk, bench_redis);

When doing benchmarks, I expect by default to have redis, memory and disk benchmarks. Thus, if limitador was configured to disable redis_storage, then benchmarks with redis would not be done. Then, I would expect something similar for the disk storage. If limitador is configured to disable disk_persistence, then benchmarks on disk would not run.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 2b635e2

@alexsnaps alexsnaps merged commit dd76789 into main May 17, 2023
@alexsnaps alexsnaps deleted the rocksdb-full branch May 17, 2023 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants