waited for all pods ready during generated hashring initialization

[christopherzli]

During the initial controller setup, we prefer to wait over all pods ready before generating hashring.

Deployed in our integrationtest cluster, found

1. it will wait for all pods ready before generating hashring for first time
2. else will be no-op if not all pods are ready and replicas do not exist in the local
   This is to handle no replicas information stored in local controller after controller restarts, we should not do anything until the cluster is healthy.

[jnyi]

lgtm, if my understanding is correct, it doesn't need to wait for all 3 sts ready but during scaling up, it will update hashring if 1 sts is entirely ready, and then sequentially, so in total it only update the hashring 3 times?

[christopherzli]

lgtm, if my understanding is correct, it doesn't need to wait for all 3 sts ready but during scaling up, it will update hashring if 1 sts is entirely ready, and then sequentially, so in total it only update the hashring 3 times?

@jnyi no, if you look at the code closely, it only saved hashring outside the for statefulsets loop, so will wait for the entire cluster to be ready. Let me know if you have concerns about this behaviour

[christopherzli]

jenkins merge