CSI Driver for PowerFlex shows OCI runtime create failed

Out of 20 controller and node pods, 17 were running.

A controller and node pod on the same node showed CrashLoopBackoff.

A node pod on a different node showed CrashLoopBackoff.

Describe of CSI Driver controller and node pods that did not come up showed the following event. The pods were unable to find the "csi-vxflexos.sh" script to run:

Warning Failed 47s kubelet Error: failed to start container "driver": Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/csi-vxflexos.sh": stat /csi-vxflexos.sh: no such file or directory: unknown

Using exec, on the driver container for the working controller pod showed that the script was in the working images:

# oc exec -it -n vxflexos vxflexos-controller-8874b487c-sv7th -c driver /bin/bash
...
bash-5.1$ pwd
/
bash-5.1$ ls
afs boot csi-vxflexos dev home lib64 media opt root sbin sys usr vxflexos-config
bin certs csi-vxflexos.sh etc lib licenses mnt proc run srv tmp var vxflexos-config-params

Deleting the pods did not allow the pods to come up successfully.

The user reported an issue with their internal repository and pulled images on the problem nodes.

The user found an issue with their internal repository. The user deleted the image from the problem nodes. On the next image pull, all the pods came up successfully.