Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compressed blobs caching #354

Open
jimmymathews opened this issue Sep 20, 2024 · 0 comments
Open

Compressed blobs caching #354

jimmymathews opened this issue Sep 20, 2024 · 0 comments
Assignees

Comments

@jimmymathews
Copy link
Collaborator

For this issue, let's decide on a compression system and caching system for the binary blobs with phenotype data etc.

We need to decide when compression happens and where/when the results are stored, what is served by the API, etc.

@franciscouzo did some experiments with compression ratios and algorithms, and mentioned https://github.com/google/ngx_brotli .

This is an nginx plugin that supports brotli compression (and caching, as usual?). We could insert this between the FastAPI service and clients. In the current architecture, an off-the-shelf ingress nginx is used as a load balancer, so we probably shouldn't alter the configuration of that nginx component.

But we could just add a new service/pod to the helm configuration?

Needs some research.

@franciscouzo franciscouzo self-assigned this Sep 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants