-
Notifications
You must be signed in to change notification settings - Fork 109
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Request to add Marvell FastLinQ QL45000 Series QLogic Corp. Vendor-id: 1077 as supported device for SR-IOV with K8s #234
Comments
Greetings ! If there are no special configurations required for the NIC to enable and configure SR-IOV then it should be a matter of adding QLogic NICs to the supported-nic-ids config map. To add it by default, i think it would be preferable that some CI testing is set up to run against QLogic hardware by editing You are welcome join Our Working Group meetings (K8s Network & Resource Management). |
We have successfully tested the Marvell interface using vanilla K8s by editing supported-nic-ids configmap we were able to get sriov-network-operator to work with QLogic NICs |
we are trying to validate vendor card using hack/run-e2e-test-kind.sh script with system-service but geeting error. docker build -t sriov-network-operator:latest -f /root/sriov-network-operator/Dockerfile /root/sriov-network-operator [root@k8sworker bin]# docker version [root@k8sworker bin]# kubectl version Please provide the process of validating cards. |
It looks like an issue with old docker version that doesn't support multi-stage builds? |
|
As discussed with community attaching the configMap
|
The contact information for this card HPE Synergy 6820C 25/50Gb Converged Network Adapter - BSN_dev_support@hpe.com |
Well known K8s platform is using this operator to allow supported network interfaces to provide SR-IOV capability to K8s Pods
When try this with Marvell Interface (Marvell FastLinQ QL45000 Series QLogic Corp. Vendor-id: 1077 )
Error from server (vendor 1077 is not supported): error when creating "SRIOVoperator/SriovNetworkNodePolicy.yaml": admission webhook "sriov-operator-webhook.sriovnetwork.k8s.cni.cncf.io" denied the request: vendor 1077 is not supported
We have Isolated the challenge to this being the configmap used for the setup of the operator leverage the following input yamls that do not have reference for vendor id 1077 we think following are the yaml inputs that need to modified to support the Marvell interface
• sriov-network-operator/supported-nic-ids_v1_configmap.yaml
• sriov-network-operator/sriov-network-operator_configmap.yaml
• sriov-network-operator/configmap.yaml
• sriov-network-operator/pr-ci-triggers.yml
We have successfully able to provide SR-IOV to POD within Well known K8s platform using Marvell Interface (Marvell FastLinQ QL45000 Series QLogic Corp. Vendor-id: 1077 ) using the https://github.com/k8snetworkplumbingwg/sriov-network-device-plugin approach
Here is some of the changes we made to get this working
$ kubectl create -f deployments/sriov-crd.yaml
Note: - Get the resource name (k8s.v1.cni.cncf.io/resourceName) by describe node as mentioned below
kubectl get node node1 -o json | jq '.status.allocatable'
$ kubectl get node node1 -o json | jq '.status.allocatable'
{
"allocable": {
"cpu": "39430m",
"ephemeral-storage": "86930620273",
"hugepages-1Gi": "0",
"hugepages-2Mi": "0",
"marvell.com/marvell_sriov_dpdk": "3",
"marvell.com/marvell_sriov_netdevice": "3",
"memory": "121944076Ki",
"pods": "250"
}
shown below:
{
"resourceList": [{
"resourceName": "marvell_sriov_netdevice",
"resourcePrefix": "marvell.com",
"selectors": {
"vendors": ["1077"],
"devices": ["1664"],
"drivers": ["qede","qed"]
}
},
{
"resourceName": "marvell_sriov_dpdk",
"resourcePrefix": "marvell.com",
"selectors": {
"vendors": ["1077"],
"devices": ["1664"],
"drivers": ["qede","qed"],
"pfNames": ["ens3f0"],
"needVhostNet": true
}
}
}
$ kubectl get node node1 -o json | jq '.status.allocatable'
{
"allocable": {
"cpu": "39430m",
"ephemeral-storage": "86930620273",
"hugepages-1Gi": "0",
"hugepages-2Mi": "0",
"marvell.com/marvell_sriov_dpdk": "3",
"marvell.com/marvell_sriov_netdevice": "3",
"memory": "121944076Ki",
"pods": "250"
}
Deploy test Pod connecting to pre-created SR-IOV network
$ kubectl create -f pod-tc1.yaml
pod "testpod1" created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
testpod1 1/1 Running 0 3s
Verify Pod network interfaces
$ kubectl exec -it testpod1 -- ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
4: net1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 9a:c0:0e:90:11:04 brd ff:ff:ff:ff:ff:ff
inet 10.56.217.5/8 brd 10.255.255.255 scope global net1
valid_lft forever preferred_lft forever
1246: eth0@if1247: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 26:f2:d1:a6:35:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 192.168.2.174/32 scope global eth0
valid_lft forever preferred_lft forever
Verify Pod routing table
$ kubectl exec -it testpod1 -- route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.3.23 0.0.0.0 UG 0 0 0 eth0
10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 net1
192.168.3.23 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
kubectl get po -A --kubeconfig $u1 |grep -i multus
kube-system kube-multus-ds-64d9q 1/1 Running 0 2d19h
kube-system kube-multus-ds-c4cnj 1/1 Running 0 2d19h
kubectl get po -A --kubeconfig $u1 |grep -i sriov
default sriovpod1 1/1 Running 0 45h
kube-system kube-sriov-device-plugin-amd64-kxfrg 1/1 Running 0 2d19h
kube-system kube-sriov-device-plugin-amd64-mwqwg 1/1 Running 0 2d19h
sriov-crd-network-attach.yaml.txt
describepod.yaml.txt
We have validated Marvell Interface (Marvell FastLinQ QL45000 Series QLogic Corp. Vendor-id: 1077 ) working with Well known K8s platform able to provide SR-IOV to pods using https://github.com/k8snetworkplumbingwg/sriov-network-device-plugin however is not the official way of doing this on Well known K8s platform
Hence we request your support and guidance on how have could have Marvell Interface (Marvell FastLinQ QL45000 Series QLogic Corp. Vendor-id: 1077 ) as supported network Interface for sriov-network-operator approach
The text was updated successfully, but these errors were encountered: