CSI Drivers: How to collect logs

CSI Driver log collection
This consists of enabling debug, reproducing the issue if possible, collecting logs for the CSI Driver controller, collecting logs from node pod.

Enable debug logs for the CSI Driver controller and node pods.
See "Dynamic Logging Configuration" feature in the documentation: https://dell.github.io/csm-docs/docs/csidriver

Determine the names of the controller and node pods.
Use the kubectl get pods command to list the pods in the driver namespace.

# kubectl get pods -n isilon
NAME                                READY   STATUS    RESTARTS        AGE
isilon-controller-66d7d9d7c-6j6f4   6/6     Running   0               11m
isilon-node-9wngk                   2/2     Running   0               27d

Collect CSI Driver Controller logs.
Logs should be collected for the controller pods. Collect logs from all controllers using the kubectl logs command. Ensure to pass in the relevant container such as driver.

kubectl logs -n <namespace> <controller> <container> > <file name>

Example:

kubectl logs -n isilon isilon-controller-66d7d9d7c-6j6f4 driver > isilon-controller-66d7d9d7c-6j6f4_driver.log

Repeat for all the required containers related to your issue (attacher, snapshotter, provisioner, driver, resizer, and so on). Be sure to change the file name of the output file to match the controller name and container.

Collect CSI Driver node pod logs.
The logs should also be collected for the worker nodes. This could include one node, or it could include multiple nodes if a problem pod was moving from one node to another. This collects the logs for each of the containers on the CSI Driver node pod(driver, registrar, and so on):

kubectl logs -n <namespace> <worker node> > <filename>.txt

Example:

kubectl logs -n isilon isilon-node-9wngk driver > isilon-node-9wngk_driver.txt
kubectl logs -n isilon isilon-node-9wngk registrar > isilon-node-9wngk_registrar.txt

Alternatives
Script
This generates all Controller logs individually(change "isilon" namespace 2 places if needed) and add containers as required.

for x in $(kubectl get pods -n isilon | grep controller | awk '{print $1}'); do for i in attacher provisioner driver snapshotter;do kubectl logs -n isilon $x -c $i > $x-$i.txt;done;done

This collects node logs(change "isilon" namespace in two places if needed) and add containers as required.

for x in $(kubectl get pods -n isilon | grep node | awk '{print $1}'); do for i in driver registrar;do kubectl logs -n isilon $x -c $i > $x-$i.txt;done;done

Since option
Collecting all the logs could yield more data than required. The "--since" option can be used to collect for a known amount of time. This option is followed by "5 s," "5 m," or "5 h" for seconds, minutes, hours depending on how far back the log collect should go. This is also helpful if the issue was reproduced, collect the last 5 minutes of logs (or the amount of time it took to reproduce):

Example:

kubectl logs -n isilon isilon-node-xxx driver --since 5m > isilon-node-xxx_driver.txt

All-containers
It is helpful in troubleshooting to collect the logs separately as outlined above. Sometimes, replacing the container name with "--all-containers" to collect all the pod's containers' logs can be useful.

Example:

kubectl logs -n isilon isilon-node-xxx --all-containers> isilon-node_driver.txt

Other required information:
Container Orchestrator (K8s, OpenShift, and so on) and version
Kubernetes version and flavor
Driver version
Array version or microcode
K8 nodes operating system and version