Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: rbd encryption tests are failing due to vault #2657

Closed
Madhu-1 opened this issue Nov 19, 2021 · 2 comments
Closed

ci: rbd encryption tests are failing due to vault #2657

Madhu-1 opened this issue Nov 19, 2021 · 2 comments
Labels
component/rbd Issues related to RBD component/testing Additional test cases or CI work keepalive This label can be used to disable stale bot activiity in the repo Priority-0 highest priority issue

Comments

@Madhu-1
Copy link
Collaborator

Madhu-1 commented Nov 19, 2021

Looks like the e2e tests are broken due to vault issue

StorageClass "csi-rbd-sc": rpc error: code = InvalidArgument desc = invalid encryption kms configuration: failed connecting to Vault: failed to get the authentication token: Put "http://vault.cephcsi-e2e-ab3babb4a6ad.svc.cluster.local:8200/v1/auth/kubernetes/login": dial tcp: lookup vault.cephcsi-e2e-ab3babb4a6ad.svc.cluster.local: no such host
Nov 19 03:13:19.008: INFO: POD  NODE  PHASE  GRACE  CONDITIONS

Nov 19 03:13:19.308: INFO: At 2021-11-19 03:02:50 +0000 GMT - event for vault-init-job-4tgrs: {default-scheduler } Scheduled: Successfully assigned cephcsi-e2e-ab3babb4a6ad/vault-init-job-4tgrs to minikube
Nov 19 03:13:19.308: INFO: At 2021-11-19 03:02:51 +0000 GMT - event for vault-init-job-4tgrs: {kubelet minikube} Pulled: Container image "docker.io/library/vault:latest" already present on machine
Nov 19 03:13:19.308: INFO: At 2021-11-19 03:02:51 +0000 GMT - event for vault-init-job-4tgrs: {kubelet minikube} Started: Started container vault-init-job
Nov 19 03:13:19.308: INFO: At 2021-11-19 03:02:51 +0000 GMT - event for vault-init-job-4tgrs: {kubelet minikube} Created: Created container vault-init-job
Nov 19 03:13:19.308: INFO: At 2021-11-19 03:08:12 +0000 GMT - event for vault-init-job: {job-controller } SuccessfulCreate: Created pod: vault-init-job-rgm6r
Nov 19 03:13:19.308: INFO: At 2021-11-19 03:08:12 +0000 GMT - event for vault-init-job-rgm6r: {default-scheduler } Scheduled: Successfully assigned cephcsi-e2e-ab3babb4a6ad/vault-init-job-rgm6r to minikube
Nov 19 03:13:19.308: INFO: At 2021-11-19 03:08:13 +0000 GMT - event for vault-init-job-rgm6r: {kubelet minikube} Created: Created container vault-init-job
Nov 19 03:13:19.308: INFO: At 2021-11-19 03:08:13 +0000 GMT - event for vault-init-job-rgm6r: {kubelet minikube} Pulled: Container image "docker.io/library/vault:latest" already present on machine
Nov 19 03:13:19.308: INFO: At 2021-11-19 03:08:13 +0000 GMT - event for vault-init-job-rgm6r: {kubelet minikube} Started: Started container vault-init-job
Nov 19 03:13:19.314: INFO: POD                                            NODE      PHASE    GRACE  CONDITIONS
Nov 19 03:13:19.314: INFO: csi-cephfsplugin-provisioner-7f7bcf4d54-c722t  minikube  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:51:40 +0000 GMT  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:51:51 +0000 GMT  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:51:51 +0000 GMT  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:51:40 +0000 GMT  }]
Nov 19 03:13:19.314: INFO: csi-cephfsplugin-wvjvq                         minikube  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:51:40 +0000 GMT  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:51:50 +0000 GMT  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:51:50 +0000 GMT  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:51:40 +0000 GMT  }]
Nov 19 03:13:19.314: INFO: csi-rbdplugin-provisioner-6b7864574f-8t5ld     minikube  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:53:56 +0000 GMT  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:54:00 +0000 GMT  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:54:00 +0000 GMT  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:53:56 +0000 GMT  }]
Nov 19 03:13:19.314: INFO: csi-rbdplugin-vbbns                            minikube  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:58:30 +0000 GMT  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:58:32 +0000 GMT  } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:58:32 +0000 GMT  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:58:30 +0000 GMT  }]
Nov 19 03:13:19.314: INFO: vault-5bcdf8c6f7-dxfdn                         minikube  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:52:25 +0000 GMT  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:55:13 +0000 GMT ContainersNotReady containers with unready status: [monitor]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:55:13 +0000 GMT ContainersNotReady containers with unready status: [monitor]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:52:25 +0000 GMT  }]
Nov 19 03:13:19.314: INFO: vault-init-job-4tgrs                           minikube  Failed          [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 03:02:50 +0000 GMT  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-11-19 03:07:52 +0000 GMT ContainersNotReady containers with unready status: [vault-init-job]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-11-19 03:07:52 +0000 GMT ContainersNotReady containers with unready status: [vault-init-job]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 03:02:50 +0000 GMT  }]
Nov 19 03:13:19.314: INFO: vault-init-job-5htdl                           minikube  Failed          [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:57:38 +0000 GMT  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-11-19 03:02:40 +0000 GMT ContainersNotReady containers with unready status: [vault-init-job]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-11-19 03:02:40 +0000 GMT ContainersNotReady containers with unready status: [vault-init-job]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:57:38 +0000 GMT  }]
Nov 19 03:13:19.314: INFO: vault-init-job-j5rhv                           minikube  Failed          [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:52:35 +0000 GMT  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:57:38 +0000 GMT ContainersNotReady containers with unready status: [vault-init-job]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:57:38 +0000 GMT ContainersNotReady containers with unready status: [vault-init-job]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 02:52:35 +0000 GMT  }]
Nov 19 03:13:19.314: INFO: vault-init-job-rgm6r                           minikube  Failed          [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 03:08:12 +0000 GMT  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-11-19 03:13:14 +0000 GMT ContainersNotReady containers with unready status: [vault-init-job]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-11-19 03:13:14 +0000 GMT ContainersNotReady containers with unready status: [vault-init-job]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-11-19 03:08:12 +0000 GMT  }]

vault job is not in ready state.
@Madhu-1 Madhu-1 mentioned this issue Nov 19, 2021
Merged
@Madhu-1 Madhu-1 added component/rbd Issues related to RBD Priority-0 highest priority issue labels Nov 19, 2021
Madhu-1 added a commit to Madhu-1/ceph-csi that referenced this issue Nov 19, 2021
@Madhu-1
ci: use 1.8.5 vault for e2e
1a77616 
current latest vault release is 1.9.0 but
with the latest image our E2E is broken.
reverting back the vault version to 1.8.5
till we root cause the issue.

Note:- This is to unblock PR merging

updates: ceph#2657

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
@Madhu-1 Madhu-1 mentioned this issue Nov 19, 2021
Merged
@Madhu-1
Copy link
Collaborator Author

Madhu-1 commented Nov 19, 2021

@ceph/ceph-csi-contributors until this is fixed CI is broken for now. retest of any PR's won't help. please avoid retesting the PR's

Madhu-1 added a commit to Madhu-1/ceph-csi that referenced this issue Nov 19, 2021
@Madhu-1
ci: use 1.8.5 vault for e2e
534e37d 
current latest vault release is 1.9.0 but
with the latest image our E2E is broken.
reverting back the vault version to 1.8.5
till we root cause the issue.

Note:- This is to unblock PR merging

updates: ceph#2657

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
mergify bot pushed a commit that referenced this issue Nov 19, 2021
@Madhu-1 @mergify
ci: use 1.8.5 vault for e2e
5524b2d 
current latest vault release is 1.9.0 but
with the latest image our E2E is broken.
reverting back the vault version to 1.8.5
till we root cause the issue.

Note:- This is to unblock PR merging

updates: #2657

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Rakshith-R pushed a commit that referenced this issue Nov 24, 2021
@Madhu-1 @Rakshith-R
ci: use 1.8.5 vault for e2e
77061b2 
current latest vault release is 1.9.0 but
with the latest image our E2E is broken.
reverting back the vault version to 1.8.5
till we root cause the issue.

Note:- This is to unblock PR merging

updates: #2657

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 5524b2d)
@nixpanic nixpanic added component/testing Additional test cases or CI work keepalive This label can be used to disable stale bot activiity in the repo labels Nov 24, 2021
rootfs pushed a commit that referenced this issue Nov 24, 2021
@Madhu-1
ci: use 1.8.5 vault for e2e
7a88404 
current latest vault release is 1.9.0 but
with the latest image our E2E is broken.
reverting back the vault version to 1.8.5
till we root cause the issue.

Note:- This is to unblock PR merging

updates: #2657

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 5524b2d)
mergify bot pushed a commit that referenced this issue Nov 24, 2021
@Madhu-1 @mergify
ci: use 1.8.5 vault for e2e
84d7726 
current latest vault release is 1.9.0 but
with the latest image our E2E is broken.
reverting back the vault version to 1.8.5
till we root cause the issue.

Note:- This is to unblock PR merging

updates: #2657

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 5524b2d)
@Madhu-1
Copy link
Collaborator Author

Madhu-1 commented Jan 21, 2022

fixed in #2658 closing

@Madhu-1 Madhu-1 closed this as completed Jan 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component/rbd Issues related to RBD component/testing Additional test cases or CI work keepalive This label can be used to disable stale bot activiity in the repo Priority-0 highest priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nixpanic @Madhu-1