Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove support for delaying state recovery pending master nodes #51806

Closed
DaveCTurner opened this issue Feb 3, 2020 · 8 comments · Fixed by #53845
Closed

Remove support for delaying state recovery pending master nodes #51806

DaveCTurner opened this issue Feb 3, 2020 · 8 comments · Fixed by #53845
Assignees
@DaveCTurner
Labels
:Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. Team:Distributed Meta label for distributed team

Comments

@DaveCTurner
Copy link
Contributor

It is useful to be able to delay state recovery until enough data nodes have joined the cluster, since this gives the shard allocator a decent opportunity to re-use as much existing data as possible. However we also have the option to delay state recovery until a certain number of master-eligible nodes have joined, and this seems unnecessary: we require a majority of master-eligible nodes for state recovery, but there is no advantage in waiting for more.

We should deprecate the following settings in 7.x and remove them in 8.0:

  • gateway.expected_nodes
  • gateway.expected_master_nodes
  • gateway.recover_after_nodes
  • gateway.recover_after_master_nodes
@DaveCTurner DaveCTurner added :Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. team-discuss labels Feb 3, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Cluster Coordination)

@DaveCTurner DaveCTurner added help wanted adoptme good first issue low hanging fruit and removed team-discuss labels Feb 12, 2020
@DaveCTurner
Copy link
Contributor Author

We discussed this today and agreed that it will help to simplify this area, but this isn't on the roadmap yet so I'm marking it as help wanted to invite contributions from the community.

@aldenpeffer
Copy link

@DaveCTurner I'm going to take a crack at this.

@DaveCTurner
Copy link
Contributor Author

Thanks @aldenpeffer, go ahead. Note that this'll take two PRs in sequence: the first to deprecate these settings and the second to remove them, and both should be opened against the master branch. We will take care of backporting the first.

@ShawnLi1014 ShawnLi1014 mentioned this issue Mar 17, 2020
Merged
DaveCTurner pushed a commit that referenced this issue Mar 18, 2020
@ShawnLi1014
Deprecate delaying state recovery for master nodes (#53646)
fe5092a 
It is useful to be able to delay state recovery until enough data nodes have
joined the cluster, since this gives the shard allocator a decent opportunity
to re-use as much existing data as possible. However we also have the option to
delay state recovery until a certain number of master-eligible nodes have
joined, and this is unnecessary: we require a majority of master-eligible nodes
for state recovery, and there is no advantage in waiting for more.

This commit deprecates the unnecessary settings in preparation for their
removal.

Relates #51806
DaveCTurner pushed a commit that referenced this issue Mar 18, 2020
@ShawnLi1014 @DaveCTurner
Deprecate delaying state recovery for master nodes (#53646)
e7ae9ae 
It is useful to be able to delay state recovery until enough data nodes have
joined the cluster, since this gives the shard allocator a decent opportunity
to re-use as much existing data as possible. However we also have the option to
delay state recovery until a certain number of master-eligible nodes have
joined, and this is unnecessary: we require a majority of master-eligible nodes
for state recovery, and there is no advantage in waiting for more.

This commit deprecates the unnecessary settings in preparation for their
removal.

Relates #51806
@ShawnLi1014 ShawnLi1014 mentioned this issue Mar 20, 2020
Merged
@vishalthakur1972

This comment has been minimized.

Sign in to view
@codebrain codebrain mentioned this issue Apr 1, 2020
Closed
38 tasks
@rjernst rjernst added the Team:Distributed Meta label for distributed team label May 4, 2020
@zroll
Copy link

zroll commented Jun 10, 2020

Is this still open? Or was it fixed in #53845? Thanks!

@jainankitk
Copy link
Contributor

@DaveCTurner - Are we still planning to include this in 8.0 release?

@DaveCTurner
Copy link
Contributor Author

Yes, although #53845 appears to have stalled.

@DaveCTurner DaveCTurner removed good first issue low hanging fruit help wanted adoptme labels Feb 1, 2021
@DaveCTurner DaveCTurner self-assigned this Feb 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DaveCTurner DaveCTurner

Labels
:Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. Team:Distributed Meta label for distributed team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove support for delaying state recovery pending master ShawnLi1014/elasticsearch
7 participants
@rjernst @zroll @DaveCTurner @jainankitk @elasticmachine @aldenpeffer @vishalthakur1972