Bump grpc stream flag default values #2051
Conversation
WalkthroughThe update boosts the gRPC streaming configuration by increasing the maximum batch size and channel buffer size from 2000 to 1000000. This adjustment aims to enhance performance and accommodate larger data volumes during streaming operations, while preserving the existing application logic and control flow. Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
dydxwill
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- protocol/app/flags/flags.go (1 hunks)
Additional comments not posted (2)
protocol/app/flags/flags.go (2)
67-67: Significant increase inDefaultGrpcStreamingMaxBatchSize.The maximum batch size for gRPC streaming has been increased from 2000 to 100000. This could improve performance by allowing larger batches, but it could also increase memory usage. Monitor the impact on memory usage and performance under load.
68-68: Significant increase inDefaultGrpcStreamingMaxChannelBufferSize.The channel buffer size for gRPC streaming has been increased from 2000 to 100000. This could improve performance by allowing more messages to be buffered, but it could also increase memory usage. Monitor the impact on memory usage and performance under load.
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- protocol/app/flags/flags.go (1 hunks)
- protocol/app/flags/flags_test.go (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- protocol/app/flags/flags.go
Additional comments not posted (1)
protocol/app/flags/flags_test.go (1)
83-84: Ensure the new buffer size values are appropriate.The
GrpcStreamingMaxBatchSizeandGrpcStreamingMaxChannelBufferSizehave been increased to 1000000. Ensure these values are appropriate for your use case and do not cause memory issues.
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- protocol/app/flags/flags_test.go (4 hunks)
Files skipped from review as they are similar to previous changes (1)
- protocol/app/flags/flags_test.go
This reverts commit ac64d4a.
Changelist
Previously for orderbook updates, we were counting # of clob pair ids but now we are counting number of messages. This PR bumps up the default grpc stream buffer size.
Test Plan
Tested config in dev
Author/Reviewer Checklist
state-breakinglabel.indexer-postgres-breakinglabel.PrepareProposalorProcessProposal, manually add the labelproposal-breaking.feature:[feature-name].backport/[branch-name].refactor,chore,bug.Summary by CodeRabbit
New Features
Performance Improvements