Containers

Last reply by 04-15-2022 Solved
Start a Discussion
2 Bronze
2 Bronze
663

Unity CSI Driver failed

Dear All:

I deploy a k8s cluster with 1 master and 1 worker. I've done all installation requirement from the installation guide as below. When I try to test the sample.yaml from csi driver package, everything does now work correctly

https://dell.github.io/csm-docs/docs/csidriver/installation/helm/unity/

Here is the environment info:

K8s version: 1.22.0

CSI Driver version: 2.22.0

The output related to sample.yaml

[root@k8s-master-1 ~]# kubectl get all -n test-unity
NAME READY STATUS RESTARTS AGE
pod/unitytest-0 0/1 Pending 0 45m

NAME READY AGE
statefulset.apps/unitytest 0/1 45m

[root@k8s-master-1 ~]# kubectl get pvc -n test-unity
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvol1 Pending unity-virt2212drzmzg-iscsi 46m
pvol2 Pending unity-virt2212drzmzg-nfs 46m

I only test iSCSI and NFS, so only 2 pvc were list

[root@k8s-master-1 ~]# kubectl describe -n test-unity pvc/pvol1
Name: pvol1
Namespace: test-unity
StorageClass: unity-virt2212drzmzg-iscsi
Status: Pending
Volume:
Labels: <none>
Annotations: volume.beta.kubernetes.io/storage-provisioner: csi-unity.dellemc.com
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Filesystem
Used By: unitytest-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ExternalProvisioning 2m14s (x181 over 47m) persistentvolume-controller waiting for a volume to be created, either by external provisioner "csi-unity.dellemc.com" or manually created by system administrator

[root@k8s-master-1 ~]# kubectl describe -n test-unity pvc/pvol2
Name: pvol2
Namespace: test-unity
StorageClass: unity-virt2212drzmzg-nfs
Status: Pending
Volume:
Labels: <none>
Annotations: volume.beta.kubernetes.io/storage-provisioner: csi-unity.dellemc.com
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Filesystem
Used By: unitytest-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ExternalProvisioning 2m32s (x181 over 47m) persistentvolume-controller waiting for a volume to be created, either by external provisioner "csi-unity.dellemc.com" or manually created by system administrator

PVC create were stock in pending state, waiting for volume create....

No PV were created

[root@k8s-master-1 ~]# kubectl get pv
No resources found

The controller driver log is as below

[root@k8s-master-1 ~]# kubectl logs -n unity unity-controller-65475c6457-dkhdq -c driver
Endpoint /var/run/csi/csi.sock
I0414 07:45:44.600881 1 leaderelection.go:243] attempting to acquire leader lease unity/driver-csi-unity-dellemc-com...
E0414 07:46:14.602401 1 leaderelection.go:325] error retrieving resource lock unity/driver-csi-unity-dellemc-com: Get "https://10.96.0.1:443/apis/coordination.k8s.io/v1/namespaces/unity/leases/driver-csi-unity-dellemc-c...": dial tcp 10.96.0.1:443: i/o timeout
E0414 07:46:53.231553 1 leaderelection.go:325] error retrieving resource lock unity/driver-csi-unity-dellemc-com: Get "https://10.96.0.1:443/apis/coordination.k8s.io/v1/namespaces/unity/leases/driver-csi-unity-dellemc-c...": dial tcp 10.96.0.1:443: i/o timeout
E0414 07:47:33.879394 1 leaderelection.go:325] error retrieving resource lock unity/driver-csi-unity-dellemc-com: Get "https://10.96.0.1:443/apis/coordination.k8s.io/v1/namespaces/unity/leases/driver-csi-unity-dellemc-c...": dial tcp 10.96.0.1:443: i/o timeout
E0414 07:48:12.870109 1 leaderelection.go:325] error retrieving resource lock unity/driver-csi-unity-dellemc-com: Get "https://10.96.0.1:443/apis/coordination.k8s.io/v1/namespaces/unity/leases/driver-csi-unity-dellemc-c...": dial tcp 10.96.0.1:443: i/o timeout
E0414 07:48:50.497694 1 leaderelection.go:325] error retrieving resource lock unity/driver-csi-unity-dellemc-com: Get "https://10.96.0.1:443/apis/coordination.k8s.io/v1/namespaces/unity/leases/driver-csi-unity-dellemc-c...": dial tcp 10.96.0.1:443: i/o timeout
E0414 07:49:28.049399 1 leaderelection.go:325] error retrieving resource lock unity/driver-csi-unity-dellemc-com: Get "https://10.96.0.1:443/apis/coordination.k8s.io/v1/namespaces/unity/leases/driver-csi-unity-dellemc-c...": dial tcp 10.96.0.1:443: i/o timeout
E0414 07:50:07.175095 1 leaderelection.go:325] error retrieving resource lock unity/driver-csi-unity-dellemc-com: Get "https://10.96.0.1:443/apis/coordination.k8s.io/v1/namespaces/unity/leases/driver-csi-unity-dellemc-c...": dial tcp 10.96.0.1:443: i/o timeout
E0414 07:50:42.570874 1 leaderelection.go:325] error retrieving resource lock unity/driver-csi-unity-dellemc-com: Get "https://10.96.0.1:443/apis/coordination.k8s.io/v1/namespaces/unity/leases/driver-csi-unity-dellemc-c...": dial tcp 10.96.0.1:443: i/o timeout

I found 10.96.0.1 was Kubernetes cluster IP, but why the controller show all this message.

Does anyone can help me to find out the problem?

Best Regards,

 

Solution (1)

Accepted Solutions
2 Bronze
2 Bronze
637

I have find out the actual problem was firewall related. After I done the follow command, everything worked as expect.

systemctl stop kubelet
systemctl stop docker
iptables --flush
iptables -tnat --flush
systemctl start kubelet
systemctl start docker

View solution in original post

Reply (1)
2 Bronze
2 Bronze
638

I have find out the actual problem was firewall related. After I done the follow command, everything worked as expect.

systemctl stop kubelet
systemctl stop docker
iptables --flush
iptables -tnat --flush
systemctl start kubelet
systemctl start docker
Latest Solutions
Top Contributor