-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use smile format instead of json for internal Druid process communication. #7798
Comments
Adding to |
This doesn't need to be done in 0.15.0, since HTTP supports negotiated response types: the client sends an Accept header with a preference-ranked list of content types that it supports, and the server picks one and responds with it, setting the Content-Type header to the one it chose. IMO, the best way to switch to smile is to adjust things at a deeper layer so all Druid JSON endpoints support this kind of negotiation, and could return either JSON or Smile based on what the client supports and prefers. |
I don't think all breaking changes in internal API should be a release blocker. And as @gianm said, it can be compatible. |
Just curious what is the rationale behind choosing Smile amongst myriads of binary encoding formats JSON binary encoding formats like MessagePack, BSON, BJSON etc or others like Protocol Buffers, Thrift, Avro etc. |
For internal communication between Druid processes, switching to a binary format would provide reduction in bytes transferred over the network and better performance.
This should be done for transferring published segments data between coordinator and broker and possibly at other places.
The text was updated successfully, but these errors were encountered: