PowerFlex: kubectl: command not found

When running any kubectl command from the MVM, you get a "permission denied" error.

There are incorrect permissions on various configuration files, "delladmin:root" should own them. If a PFMP cluster is deployed with the root account, the delladmin account does not have access to run kubectl commands.

Verify the files and who owns them.

cd /etc
ls -lat pki/trust/ rancher/ rancher/rke2/ docker/certs.d/ helmrepo/certs.d

Correct ownership below

#/etc/rancher/rke2/
-rw------- 1 delladmin root 7534 Dec 14 13:24 rke2.yaml
-rw------- 1 delladmin root  365 Dec  7 12:21 rke2-pss.yaml
-rw-r--r-- 1 delladmin root  325 Dec  7 12:20 registries.yaml
-r-------- 1 delladmin root 1144 Dec  7 12:20 config.yaml

#/etc/helmrepo/certs.d/
-rw-r--r-- 1 delladmin root  3243 Dec  7 13:30 client.key
-rw-r--r-- 1 delladmin root  2013 Dec  7 13:30 client.crt
-rw-r--r-- 1 delladmin root 13999 Dec  7 13:30 ca.crt

#/etc/docker/certs.d/
-rw-r--r-- 1 delladmin root  3243 Dec  7 12:58 client.key
-rw-r--r-- 1 delladmin root  1980 Dec  7 12:58 client.crt
-rw-r--r-- 1 delladmin root 13999 Dec  7 12:58 ca.crt

#/etc/rancher
drwxr-xr-x 1 delladmin root   96 Dec 14 13:24 rke2

#/etc/pki/trust/
drwxr-xr-x 1 delladmin root 468 Dec  7 12:21 anchors

If ownership is not delladmin:root, correct with this action plan.
*It is advised to take a snapshot of each MVM prior to running the following procedure, once complete deleting the snapshots.

On each MVM, run the commands below,

sudo chown delladmin /etc/pki/trust/anchors
sudo chown delladmin /etc/rancher/rke2
sudo chown delladmin /etc/rancher/rke2/*
sudo chown delladmin /etc/docker/certs.d/*
sudo chown delladmin /etc/helmrepo/certs.d/*
 
# The presence of this file owned by root can cause upgrades to fail with delladmin account
rm /tmp/ca.crt

To verify the fix,

kubectl get nodes

Repeat on the other MVMs