Allow ballista_scheduler
to be embedded in the process of other applications
#568
Labels
enhancement
New feature or request
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Recently, I am working with Ballista as a distributed query engine in our existing scheduling service systems.
I'm trying to embed
ballista_scheduler
into our already existing scheduling system due to the following cases:TableProvider
that cannot be serialized intoballista_scheduler
through theballista_client
using protobuf.ballista_scheduler
and our existing scheduling system into one process.I found that although
ballista_scheduler
has exposed thescheduler_server
struct through thepub mod scheduler_server
, but it still cannot be integrated and embedded.Because the
state
andsubmit_job
in thescheduler_server
only arepub(crate)
:https://github.com/apache/arrow-ballista/blob/20891ae0a740c03b5a3a909ca033f45d59fcfc83/ballista/scheduler/src/scheduler_server/mod.rs#L59-L66
https://github.com/apache/arrow-ballista/blob/20891ae0a740c03b5a3a909ca033f45d59fcfc83/ballista/scheduler/src/scheduler_server/mod.rs#L150-L167
Describe the solution you'd like
Expose
state
andsubmit_job
in thescheduler_server
to pub.Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered: