GO-3757 graceful shutdown for long objectstore queries #1388
Conversation
|
Warning Rate limit exceeded@requilence has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 18 minutes and 44 seconds before requesting another review. How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. WalkthroughThe changes introduce concurrency management and graceful shutdown capabilities to the Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant dsObjectStore
participant DB
participant Error
Note over dsObjectStore: Store is Running
Client->>dsObjectStore: BatchProcessFullTextQueue
dsObjectStore->>DB: fetch IDs for processing
DB-->>dsObjectStore: return IDs
dsObjectStore->>Client: process IDs
Note over dsObjectStore: Store is Closing
Client->>dsObjectStore: BatchProcessFullTextQueue
dsObjectStore->>Error: return ErrStoreIsClosing
Error-->>Client: Store is closing
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (4)
- pkg/lib/localstore/objectstore/indexer_store.go (2 hunks)
- pkg/lib/localstore/objectstore/links.go (3 hunks)
- pkg/lib/localstore/objectstore/objects.go (5 hunks)
- pkg/lib/localstore/objectstore/queries.go (4 hunks)
Additional comments not posted (12)
pkg/lib/localstore/objectstore/links.go (3)
20-22: LGTM! Tracking running queries withrunningQueriesWG.The changes correctly add the query to the
WaitGroupand ensure it is decremented when the query is done.
68-69: LGTM! Tracking running queries withrunningQueriesWG.The changes correctly add the query to the
WaitGroupand ensure it is decremented when the query is done.
80-81: LGTM! Tracking running queries withrunningQueriesWG.The changes correctly add the query to the
WaitGroupand ensure it is decremented when the query is done.pkg/lib/localstore/objectstore/indexer_store.go (2)
22-23: LGTM! Tracking running queries withrunningQueriesWGand handling store closing.The changes correctly add the query to the
WaitGroup, handle the closing state, and ensure the query is properly managed.Also applies to: 30-31
47-48: LGTM! Tracking running queries withrunningQueriesWG.The changes correctly add the query to the
WaitGroupand ensure it is decremented when the query is done.pkg/lib/localstore/objectstore/objects.go (4)
195-196: LGTM! Adding fields for managing running queries and closing state.The added fields
runningQueriesWGandisClosingChare correctly introduced to manage running queries and handle the closing state.
64-64: LGTM! Initializing theisClosingChchannel.The
isClosingChchannel is correctly initialized in theNewmethod.
213-216: LGTM! Handling store closing by closingisClosingChand waiting for running queries.The
Closemethod correctly handles the closing state by closing theisClosingChchannel and waiting for all running queries to complete usingrunningQueriesWG.
354-355: LGTM! Tracking running queries withrunningQueriesWG.The changes correctly add the query to the
WaitGroupand ensure it is decremented when the query is done.pkg/lib/localstore/objectstore/queries.go (3)
101-108: LGTM! AddingisClosingmethod to check store closing state.The
isClosingmethod is correctly implemented to check if the store is closing by using theisClosingChchannel.
122-123: LGTM! Tracking running queries withrunningQueriesWGand handling store closing.The changes correctly add the query to the
WaitGroup, handle the closing state, and ensure the query is properly managed.Also applies to: 131-133, 144-146
186-188: LGTM! Handling store closing during full-text queries.The changes correctly handle the store's closing state during full-text queries by checking the
isClosingmethod and returningErrStoreIsClosingif the store is closing.
Coverage provided by https://github.com/seriousben/go-patch-cover-action |
isClosingChfor the objectstoreisClosingChis closed. (not possible for all queries, requires refactoring)