Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce ObjectStoreManager trait for the ObjectStoreRegistry to provide polymorphism for get_by_url #5541

Closed
yahoNanJing opened this issue Mar 10, 2023 · 0 comments · Fixed by #5543
Labels
enhancement New feature or request

Comments

@yahoNanJing
Copy link
Contributor

yahoNanJing commented Mar 10, 2023

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

The context is we are working on introducing a data source cache layer for Ballista & DataFusion apache/datafusion-ballista#645. To make the cache layer transparent for the ObjectStore caller, we plan to introduce a special cached based ObjectStore, which will search local cache first and then the original remote source ObjectStore. A sample implementation is like https://github.com/yahoNanJing/arrow-ballista/blob/dev-20230302/ballista/core/src/cache_layer/object_store/file.rs. Then we hope the ObjectStoreRegistry be able to provide polymorphism for the get_by_url method. If cached enabled, that function will always return a cached-based ObjectStore.

Describe the solution you'd like

To introduce an object trait as the attribute of ObjectStoreRegistry.

Describe alternatives you've considered

Additional context

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant