Customer Env:

K8S 1.13.3

K8S host CentOS Linux 7.4

VxFlex 3.0.769

***********************************************

3SDS/MDM：
Data1:192.168.1.2-4
Data2:192.168.2.2-4
App：10.8.192.89-91

Gateway 10.8.192.89
Primary MDM 10.8.192.90

System ID MDPM IPs
7a5ada7d5a833e0f 10.8.192.89 10.8.192.90

**************************************************
K8S：

kube-master

10.8.192.27 kube-m1
10.8.192.28 kube-m2
10.8.192.40 kube-m3

kube-worker 10.8.192.39/41/44

**************************************************

As follow the below video and product guide:

https://github.com/coulof/vxflexos-csi-examples/tree/master/demos#enable-csi-in-k8s-cluster

We modified:/var/lib/kubelet/config.yaml Please see attachment
There is no below file in customer env:/etc/systemd/system/multi-user.target.wants/docker.service
We modified: /etc/kubernetes/manifests/kube-controller-manager.yaml Please see attachment
We modiefied: /etc/kubernetes/manifests/kube-scheduler.yaml Please see attachment
We modified: /etc/kubernetes/manifests/kube-apiserver.yaml Please see attachment
There is no in customer env /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
or /etc/ systemd/system/kubelet.service.d/10-kubeadm.conf
We modified /etc/systemd/system/multi-user.target.wants/kubelet.service Please see attachment

Attach myvalues,yaml as well

****************************************************************************************

[root@kube-m1 helm]# sh install.vxflexos
Kubernetes version v1.13.3
Kubernetes master nodes: 10.8.192.27 10.8.192.28 10.8.192.40
Kubernetes minion nodes: 10.8.192.41 10.8.192.39 10.8.192.44
Verifying the SDC installation.
Retrieved 1 mdm(s)
MDM-ID 7a5ada7d5a833e0f SDC ID e71b7b7500000004 INSTALLATION ID 0f7a6e0e2eb88757 IPs [0]-10.8.192.89 [1]-10.8.192.90
Retrieved 1 mdm(s)
MDM-ID 7a5ada7d5a833e0f SDC ID e71ba28500000007 INSTALLATION ID 0f7a6e0e2eb88757 IPs [0]-10.8.192.89 [1]-10.8.192.90
Retrieved 1 mdm(s)
MDM-ID 7a5ada7d5a833e0f SDC ID e71ba28600000008 INSTALLATION ID 0f7a6e0e2eb88757 IPs [0]-10.8.192.89 [1]-10.8.192.90
Verifying the feature gates.
node 10.8.192.27 proc /usr/bin/kubelet gate VolumeSnapshotDataSource=true failed
node 10.8.192.27 proc /usr/bin/kubelet gate KubeletPluginsWatcher=true failed
node 10.8.192.27 proc /usr/bin/kubelet gate CSINodeInfo=true failed
node 10.8.192.27 proc /usr/bin/kubelet gate CSIDriverRegistry=true failed
node 10.8.192.27 proc /usr/bin/kubelet gate BlockVolume=true failed
node 10.8.192.27 proc /usr/bin/kubelet gate CSIBlockVolume=true failed
node 10.8.192.28 proc /usr/bin/kubelet gate VolumeSnapshotDataSource=true failed
node 10.8.192.28 proc /usr/bin/kubelet gate KubeletPluginsWatcher=true failed
node 10.8.192.28 proc /usr/bin/kubelet gate CSINodeInfo=true failed
node 10.8.192.28 proc /usr/bin/kubelet gate CSIDriverRegistry=true failed
node 10.8.192.28 proc /usr/bin/kubelet gate BlockVolume=true failed
node 10.8.192.28 proc /usr/bin/kubelet gate CSIBlockVolume=true failed
node 10.8.192.40 proc /usr/bin/kubelet gate VolumeSnapshotDataSource=true failed
node 10.8.192.40 proc /usr/bin/kubelet gate KubeletPluginsWatcher=true failed
node 10.8.192.40 proc /usr/bin/kubelet gate CSINodeInfo=true failed
node 10.8.192.40 proc /usr/bin/kubelet gate CSIDriverRegistry=true failed
node 10.8.192.40 proc /usr/bin/kubelet gate BlockVolume=true failed
node 10.8.192.40 proc /usr/bin/kubelet gate CSIBlockVolume=true failed
node 10.8.192.41 proc /usr/bin/kubelet gate VolumeSnapshotDataSource=true failed
node 10.8.192.41 proc /usr/bin/kubelet gate KubeletPluginsWatcher=true failed
node 10.8.192.41 proc /usr/bin/kubelet gate CSINodeInfo=true failed
node 10.8.192.41 proc /usr/bin/kubelet gate CSIDriverRegistry=true failed
node 10.8.192.41 proc /usr/bin/kubelet gate BlockVolume=true failed
node 10.8.192.41 proc /usr/bin/kubelet gate CSIBlockVolume=true failed
node 10.8.192.39 proc /usr/bin/kubelet gate VolumeSnapshotDataSource=true failed
node 10.8.192.39 proc /usr/bin/kubelet gate KubeletPluginsWatcher=true failed
node 10.8.192.39 proc /usr/bin/kubelet gate CSINodeInfo=true failed
node 10.8.192.39 proc /usr/bin/kubelet gate CSIDriverRegistry=true failed
node 10.8.192.39 proc /usr/bin/kubelet gate BlockVolume=true failed
node 10.8.192.39 proc /usr/bin/kubelet gate CSIBlockVolume=true failed
node 10.8.192.44 proc /usr/bin/kubelet gate VolumeSnapshotDataSource=true failed
node 10.8.192.44 proc /usr/bin/kubelet gate KubeletPluginsWatcher=true failed
node 10.8.192.44 proc /usr/bin/kubelet gate CSINodeInfo=true failed
node 10.8.192.44 proc /usr/bin/kubelet gate CSIDriverRegistry=true failed
node 10.8.192.44 proc /usr/bin/kubelet gate BlockVolume=true failed
node 10.8.192.44 proc /usr/bin/kubelet gate CSIBlockVolume=true failed
fail: 1
*******************************************************************************
Warning: Kubernetes --feature-gates not correctly configured... it may not work
*******************************************************************************
NAME: vxflexos
LAST DEPLOYED: Wed Jun 5 13:48:46 2019
NAMESPACE: vxflexos
STATUS: DEPLOYED

RESOURCES:
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
vxflexos-node-55crv 0/2 ContainerCreating 0 0s
vxflexos-node-5dfg6 0/2 ContainerCreating 0 0s
vxflexos-node-qpfzj 0/2 ContainerCreating 0 0s
vxflexos-controller-0 0/4 ContainerCreating 0 0s

==> v1/StorageClass
NAME PROVISIONER AGE
vxflexos-xfs csi-vxflexos 1s
vxflexos (default) csi-vxflexos 1s

==> v1/ServiceAccount
NAME SECRETS AGE
vxflexos-controller 1 1s
vxflexos-node 1 1s

==> v1/ClusterRole
NAME AGE
vxflexos-controller 1s
vxflexos-node 1s

==> v1/ClusterRoleBinding
NAME AGE
vxflexos-controller 0s
vxflexos-node 0s

==> v1beta2/DaemonSet
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
vxflexos-node 3 3 0 3 0 0s

==> v1beta1/StatefulSet
NAME DESIRED CURRENT AGE
vxflexos-controller 1 1 0s

==> v1alpha1/CSIDriver
NAME AGE
vxflexos 0s

NAME READY STATUS RESTARTS AGE
vxflexos-controller-0 0/4 ErrImagePull 0 10s
vxflexos-node-55crv 0/2 ErrImagePull 0 10s
vxflexos-node-5dfg6 0/2 ImagePullBackOff 0 10s
vxflexos-node-qpfzj 0/2 ErrImagePull 0 10s
NAME READY STATUS RESTARTS AGE
vxflexos-controller-0 0/4 ErrImagePull 0 15s
vxflexos-node-55crv 0/2 ErrImagePull 0 15s
vxflexos-node-5dfg6 0/2 ImagePullBackOff 0 15s
vxflexos-node-qpfzj 0/2 ErrImagePull 0 15s
NAME READY STATUS RESTARTS AGE
vxflexos-controller-0 0/4 ImagePullBackOff 0 20s
vxflexos-node-55crv 0/2 ImagePullBackOff 0 20s
vxflexos-node-5dfg6 0/2 ImagePullBackOff 0 20s
vxflexos-node-qpfzj 0/2 ImagePullBackOff 0 20s
CSIDrivers:
NAME AGE
vxflexos 20s
CSINodeInfos:
NAME AGE
kube-n1 1d
kube-n2 1d
kube-n3 1d
StorageClasses:
NAME PROVISIONER AGE
vxflexos (default) csi-vxflexos 22s
vxflexos-xfs csi-vxflexos 22s
No resources found.
installing volumesnapshotclass
volumesnapshotclass.snapshot.storage.k8s.io/vxflexos-snapclass created
VolumeSnapshotClasses:
NAME AGE
vxflexos-snapclass 1s

Browse By Board

Containers

Automation

Devops Forum

How two setup volume access mode on VxFlex?

CSI Driver for Dell EMC PowerMax v1.0

Provisioning Test Dev Environments with Ansible and PowerMax

Getting Started Blog Link and Examples

Ansible Modules for Dell EMC PowerMax v1.0

Can we release an offline installation guide or package for CSI driver?

Cannot create volumesnapshotclass in install.vxflexos

CSI 1.0 deployment failed on k8s 1.13.3

Kubespray with VxFlexOS CSI driver support

CSI with CentOS 7.6, Kubernetes v1.14.1 and VxFlex OS 3.0

CSI Driver for VxFlex OS Demo

How does the VxFlexOS Plugin differs from the Kubernetes ScaleIO in Tree driver

Dell Support Resources

Latest Solutions

Data Refresh Latency in Open Vulnerability API: Is there a 30-day lag?

test color post

Unable to fetch version of array error when running Ansible module

Asset API Authentication

Unable to import PowerScale python library. Please install isilon-sdk. Please install the.