Remove artificial pod creation/update/delete rate limiting #2128
Conversation
k8s-ci-robot
added
cncf-cla: yes
k8s-ci-robot
added
the
approved
| @@ -75,7 +76,8 @@ steps: | |||
| action: start | |||
| checkIfPodsAreUpdated: {{$CHECK_IF_PODS_ARE_UPDATED}} | |||
| labelSelector: group = load | |||
| operationTimeout: 15m | |||
| operationTimeout: {{$operationTimeout}} | |||
| exactOperationTimeout: true | |||
There was a problem hiding this comment.
I think we could move exactOperationTimeout to be CL2 flag/env that will change how WaitForPods works for whole test instead of particular measurements.
|
/lgtm |
k8s-ci-robot
added
do-not-merge/hold
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: marseel, mborsz The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/test pull-perf-tests-clusterloader2-e2e-gce-scale-performance-manual |
|
/hold cancel |
k8s-ci-robot
removed
the
do-not-merge/hold
|
It looks like the change guarding doesn't work. The generated data leaks global100qps: https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/perf-tests/2128/pull-perf-tests-clusterloader2/1562701334413578240/artifacts/generatedConfig_load.yaml |
| @@ -168,7 +169,13 @@ steps: | |||
| params: | |||
| actionName: "create" | |||
| namespaces: {{$namespaces}} | |||
| {{if .RATE_LIMIT_POD_CREATION}} | |||
There was a problem hiding this comment.
This should be $RATE_LIMIT_POD_CREATION
Right now, the test tries to spread load to generate expected load on the control plane. It has been introduced when master was able to handle max 10 pods per second and was overloading with higher throughput.
Now we are in a state when it is able to handle 100 pods per second throughput (we have at least few manual runs with this PR, the metrics look comparable to regular runs).
Let's remove the artificial throttling to:
The feature is controlled by CL2_RATE_LIMIT_POD_CREATION (default true) to keep default behavior.
We will gradually swap all tests to false and then remove the env var.
/assign @marseel