Clusterloader2: check labels additionally to decide master node #1196

xial-thu · 2020-04-29T05:43:16Z

Some utils like kubeadm will label master node as
"node-role.kubernetes.io/master=''" so it would be better to check
labels as additional signal. If that label is not set, it does not
affect the existing logic.

fix #1191

k8s-ci-robot · 2020-04-29T05:43:19Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: login-issues@jira.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-ci-robot · 2020-04-29T05:43:24Z

Welcome @xial-thu!

It looks like this is your first PR to kubernetes/perf-tests 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/perf-tests has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2020-04-29T05:43:24Z

Hi @xial-thu. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

xial-thu · 2020-04-29T06:10:13Z

/assign @wojtek-t

xial-thu · 2020-04-29T06:23:04Z

"I signed it"

wojtek-t

/ok-to-test

wojtek-t · 2020-04-29T11:33:45Z

clusterloader2/pkg/measurement/common/scheduler_latency.go

@@ -86,7 +86,8 @@ func (s *schedulerLatencyMeasurement) Execute(config *measurement.MeasurementCon

 	var masterRegistered = false
 	for _, node := range nodes.Items {
-		if util.LegacyIsMasterNode(node.Name) {
+		node := node


Let's remove it - it's unnecessary copy that isn't needed if the LegacyIsMasterNode isn't mutating the object (which it obviously shouldn't).

wojtek-t · 2020-04-29T11:35:44Z

clusterloader2/pkg/measurement/util/gatherers/container_resource_gatherer.go

@@ -103,12 +103,26 @@ func NewResourceUsageGatherer(c clientset.Interface, host string, port int, prov
 				return nil, fmt.Errorf("listing pods error: %v", err)
 			}
 		}
+
+		var nodeList *corev1.NodeList


Let's remove it and simply change "=" to ":=" in the list call below

wojtek-t · 2020-04-29T11:37:42Z

clusterloader2/pkg/measurement/util/gatherers/container_resource_gatherer.go

@@ -103,12 +103,26 @@ func NewResourceUsageGatherer(c clientset.Interface, host string, port int, prov
 				return nil, fmt.Errorf("listing pods error: %v", err)
 			}
 		}
+
+		var nodeList *corev1.NodeList
+		var masterNodesMap map[string]bool = make(map[string]bool)


Let's move after the list (if the list call fails you have unnecessary memory allocation).

Also, change to simply:
masterNodesMap := mak(map[string]bool)
or even better reuse string set:
https://github.com/kubernetes/perf-tests/blob/master/clusterloader2/vendor/k8s.io/apimachinery/pkg/util/sets/string.go

Lessons are learned

wojtek-t · 2020-04-29T11:38:36Z

clusterloader2/pkg/measurement/util/gatherers/container_resource_gatherer.go

+			return nil, fmt.Errorf("listing nodes error: %v", err)
+		}
+		for _, node := range nodeList.Items {
+			node := node


avoid copy if not needed

wojtek-t · 2020-04-29T11:39:04Z

clusterloader2/pkg/measurement/util/gatherers/container_resource_gatherer.go


 		for _, node := range nodeList.Items {
-			if options.Nodes == AllNodes || pkgutil.LegacyIsMasterNode(node.Name) || dnsNodes[node.Name] {
+			node := node


avoid copy if not needed

still not done

wojtek-t · 2020-04-29T11:40:07Z

clusterloader2/pkg/util/cluster.go

@@ -165,11 +167,19 @@ func GetMasterIPs(c clientset.Interface, addressType corev1.NodeAddressType) ([]
 // This code will not be allowed to update to use the node-role label, since
 // node-roles may not be used for feature enablement.
 // DEPRECATED: this will be removed in Kubernetes 1.19
-func LegacyIsMasterNode(nodeName string) bool {
+func LegacyIsMasterNode(node *corev1.Node) bool {
+	// Check labels as an additional signal, fix #1191


Let's remove bug number.

wojtek-t · 2020-04-29T11:40:27Z

clusterloader2/pkg/util/cluster.go

+func LegacyIsMasterNode(node *corev1.Node) bool {
+	// Check labels as an additional signal, fix #1191
+	for key := range node.GetLabels() {
+		if key == keyMasterNodeLabel {


Check the value too - it can be set to false too.

But how to define its value? "false", "False", "No"... It can be set to anything
I would compare ToLower(value) with "false"

Go the other way. If it's set "node-role.kubernetes.io/master=skfjds", then I wouldn't treat it as master.

Let's change explicitly to:
strings.ToLower(val) == "true"

wojtek-t

Two more minor comments.

wojtek-t · 2020-04-30T12:14:56Z

clusterloader2/pkg/measurement/util/gatherers/container_resource_gatherer.go


 		for _, node := range nodeList.Items {
-			if options.Nodes == AllNodes || pkgutil.LegacyIsMasterNode(node.Name) || dnsNodes[node.Name] {
+			node := node


still not done

wojtek-t · 2020-04-30T12:16:26Z

clusterloader2/pkg/util/cluster.go

+func LegacyIsMasterNode(node *corev1.Node) bool {
+	// Check labels as an additional signal, fix #1191
+	for key := range node.GetLabels() {
+		if key == keyMasterNodeLabel {


Go the other way. If it's set "node-role.kubernetes.io/master=skfjds", then I wouldn't treat it as master.

Let's change explicitly to:
strings.ToLower(val) == "true"

Some utils like kubeadm will label master node as "node-role.kubernetes.io/master=''" so it would be better to check labels as additional signal. If that label is not set, it does not affect the existing logic.

wojtek-t · 2020-05-01T10:22:52Z

/lgtm
/approve

k8s-ci-robot · 2020-05-01T10:23:15Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wojtek-t, xial-thu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~clusterloader2/OWNERS~~ [wojtek-t]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added the do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. label Apr 29, 2020

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Apr 29, 2020

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 29, 2020

k8s-ci-robot requested review from oxddr and wojtek-t April 29, 2020 05:43

k8s-ci-robot assigned wojtek-t Apr 29, 2020

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Apr 29, 2020

k8s-ci-robot removed the do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. label Apr 29, 2020

wojtek-t reviewed Apr 29, 2020

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 29, 2020

xial-thu requested a review from wojtek-t April 30, 2020 01:36

wojtek-t reviewed Apr 30, 2020

View reviewed changes

Clusterloader2: check labels additionally to decide master node

170b679

Some utils like kubeadm will label master node as "node-role.kubernetes.io/master=''" so it would be better to check labels as additional signal. If that label is not set, it does not affect the existing logic.

xial-thu requested a review from wojtek-t May 1, 2020 03:11

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 1, 2020

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 1, 2020

k8s-ci-robot merged commit 726ca3e into kubernetes:master May 1, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clusterloader2: check labels additionally to decide master node #1196

Clusterloader2: check labels additionally to decide master node #1196

xial-thu commented Apr 29, 2020

k8s-ci-robot commented Apr 29, 2020

k8s-ci-robot commented Apr 29, 2020

k8s-ci-robot commented Apr 29, 2020

xial-thu commented Apr 29, 2020

xial-thu commented Apr 29, 2020

wojtek-t left a comment

wojtek-t Apr 29, 2020

xial-thu Apr 29, 2020

wojtek-t Apr 29, 2020

wojtek-t Apr 29, 2020

xial-thu Apr 29, 2020

wojtek-t Apr 29, 2020

xial-thu Apr 29, 2020

wojtek-t Apr 29, 2020

wojtek-t Apr 30, 2020

wojtek-t Apr 29, 2020

xial-thu Apr 29, 2020

wojtek-t Apr 29, 2020

xial-thu Apr 29, 2020 •

edited

Loading

wojtek-t Apr 30, 2020

wojtek-t left a comment

wojtek-t Apr 30, 2020

wojtek-t Apr 30, 2020

wojtek-t commented May 1, 2020

k8s-ci-robot commented May 1, 2020

Clusterloader2: check labels additionally to decide master node #1196

Clusterloader2: check labels additionally to decide master node #1196

Conversation

xial-thu commented Apr 29, 2020

k8s-ci-robot commented Apr 29, 2020

k8s-ci-robot commented Apr 29, 2020

k8s-ci-robot commented Apr 29, 2020

xial-thu commented Apr 29, 2020

xial-thu commented Apr 29, 2020

wojtek-t left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xial-thu Apr 29, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wojtek-t left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wojtek-t commented May 1, 2020

k8s-ci-robot commented May 1, 2020

xial-thu Apr 29, 2020 •

edited

Loading