[ML] Allow force stopping failed and stopping DF analytics #54650
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Pinging @elastic/ml-core (:ml) |
droberts195
reviewed
Apr 2, 2020
| failed.add(analyticsId); | ||
| break; | ||
| default: | ||
| break; |
There was a problem hiding this comment.
I know you've only moved this in this PR, but maybe it's still best to add an assert in the default case just in case someone adds a new state but doesn't add it here?
dimitris-athanasiou
force-pushed
the
fix-force-stopping-df-analytics
branch
from
8f2fe6d
to
fe7745e
Compare
Force stopping a failed job used to work but it now puts the job in `stopping` state and hangs. In addition, force stopping a `stopping` job is not handled. This commit addresses those issues with force stopping data frame analytics. It inlines the approach with that followed for anomaly detection jobs.
dimitris-athanasiou
added a commit
to dimitris-athanasiou/elasticsearch
that referenced
this pull request
Apr 3, 2020
…tic#54650) Force stopping a failed job used to work but it now puts the job in `stopping` state and hangs. In addition, force stopping a `stopping` job is not handled. This commit addresses those issues with force stopping data frame analytics. It inlines the approach with that followed for anomaly detection jobs. Backport of elastic#54650
dimitris-athanasiou
added a commit
that referenced
this pull request
Apr 3, 2020
…) (#54712) Force stopping a failed job used to work but it now puts the job in `stopping` state and hangs. In addition, force stopping a `stopping` job is not handled. This commit addresses those issues with force stopping data frame analytics. It inlines the approach with that followed for anomaly detection jobs. Backport of #54650
dimitris-athanasiou
added a commit
that referenced
this pull request
Apr 3, 2020
…) (#54715) Force stopping a failed job used to work but it now puts the job in `stopping` state and hangs. In addition, force stopping a `stopping` job is not handled. This commit addresses those issues with force stopping data frame analytics. It inlines the approach with that followed for anomaly detection jobs. Backport of #54650
dimitris-athanasiou
added a commit
to dimitris-athanasiou/elasticsearch
that referenced
this pull request
Apr 16, 2020
After elastic#54650 we catch `TaskCancelledException` when we wait for reindexing to complete as it may be thrown. However, when that happens we do not mark the task as completed. This results in the stop request never returning and the failures we saw in elastic#55068. Closes elastic#55068
dimitris-athanasiou
added a commit
to dimitris-athanasiou/elasticsearch
that referenced
this pull request
Apr 16, 2020
…xing (elastic#55286) After elastic#54650 we catch `TaskCancelledException` when we wait for reindexing to complete as it may be thrown. However, when that happens we do not mark the task as completed. This results in the stop request never returning and the failures we saw in elastic#55068. Closes elastic#55068 Backport of elastic#55286
dimitris-athanasiou
added a commit
that referenced
this pull request
Apr 16, 2020
…xing (#55286) (#55290) After #54650 we catch `TaskCancelledException` when we wait for reindexing to complete as it may be thrown. However, when that happens we do not mark the task as completed. This results in the stop request never returning and the failures we saw in #55068. Closes #55068 Backport of #55286
dimitris-athanasiou
added a commit
that referenced
this pull request
Apr 16, 2020
…xing (#55286) (#55295) After #54650 we catch `TaskCancelledException` when we wait for reindexing to complete as it may be thrown. However, when that happens we do not mark the task as completed. This results in the stop request never returning and the failures we saw in #55068. Closes #55068 Backport of #55286
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Force stopping a failed job used to work but it
now puts the job in
stoppingstate and hangs.In addition, force stopping a
stoppingjob isnot handled.
This commit addresses those issues with force
stopping data frame analytics. It inlines the
approach with that followed for anomaly detection
jobs.