Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(batch): abort task #2757

Merged
merged 3 commits into from
May 25, 2022
Merged

feat(batch): abort task #2757

merged 3 commits into from
May 25, 2022

Conversation

lmatz
Copy link
Contributor

@lmatz lmatz commented May 24, 2022

What's changed and what's your intention?

Abort batch tasks on CN. The RPC just sends the message to notify the task that it should abort, but it does not wait for the task to abort successfully.

Part of handling task execution failure, i.e. when one task fails, the scheduler should be able to notify other batch tasks of the same query to abort and thus release the resources.

Introduce a new channel in the TaskExeuction to notify the spawned execution that its task needs to be aborted.
The spawned execution select on both the stream of a data chunk and the stream of this new channel's receiver, so that whenever the abort message is sent, the task will immediately be aborted.

Turn try_execute into a member function by making self a Arc<Self> so that the task's state can be set more easily.

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

#1977

@codecov
Copy link

codecov bot commented May 24, 2022

Codecov Report

Merging #2757 (c1cbc14) into main (b1888df) will increase coverage by 0.05%.
The diff coverage is 70.06%.

@@            Coverage Diff             @@
##             main    #2757      +/-   ##
==========================================
+ Coverage   72.61%   72.67%   +0.05%     
==========================================
  Files         700      700              
  Lines       91145    91259     +114     
==========================================
+ Hits        66182    66318     +136     
+ Misses      24963    24941      -22     
Flag Coverage Δ
rust 72.67% <70.06%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/batch/src/rpc/service/task_service.rs 0.00% <0.00%> (ø)
src/expr/src/expr/expr_field.rs 94.05% <ø> (-1.07%) ⬇️
src/expr/src/expr/mod.rs 48.83% <ø> (ø)
src/batch/src/task/task_.rs 52.09% <60.00%> (+13.97%) ⬆️
src/batch/src/task/task_manager.rs 82.12% <86.15%> (+1.95%) ⬆️
src/expr/src/expr/test_utils.rs 100.00% <100.00%> (ø)
src/stream/src/executor/managed_state/join/mod.rs 73.56% <100.00%> (+0.46%) ⬆️
src/meta/src/model/barrier.rs 78.57% <0.00%> (-3.58%) ⬇️
src/meta/src/hummock/compaction/mod.rs 82.87% <0.00%> (-0.47%) ⬇️
src/meta/src/barrier/mod.rs 65.49% <0.00%> (-0.32%) ⬇️
... and 6 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@lmatz lmatz force-pushed the lz/abort_task branch 4 times, most recently from 87a00b4 to 3055a2a Compare May 24, 2022 13:38
@lmatz lmatz marked this pull request as ready for review May 24, 2022 14:04
@lmatz lmatz requested a review from liurenjie1024 May 24, 2022 15:47
src/expr/src/expr/expr_field.rs Show resolved Hide resolved
src/batch/src/task/task_.rs Outdated Show resolved Hide resolved
src/batch/src/task/task_manager.rs Show resolved Hide resolved
@lmatz lmatz merged commit 1348deb into main May 25, 2022
@lmatz lmatz deleted the lz/abort_task branch May 25, 2022 06:06
Enter-tainer pushed a commit that referenced this pull request May 26, 2022
* feat(batch): abort task

* revision

* make wait_until_task_aborted private and test-only
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants