PowerFlex: PFMP_SUPPORT - Support Bundle Collection Utility

pfmp_support [OPTIONS]

DESCRIPTION

pfmp_support is a diagnostic utility that collects debug information from a PowerFlex Manager or a PowerFlex Manager Installer host and archives it into a compressed bundle for analysis by support personnel.

The utility gathers data from multiple sources, including:

• Kubernetes diagnostic data, configuration, and logs
• Helm release information
• Docker and container runtime data
• PowerFlex Manager installer logs
• Operating system configuration, logs, and runtime state
• Hardware inventory

When collecting data from multiple Kubernetes cluster nodes, a standard bundle (--skip-kubernetes-shared not specified) should be collected from at least one node, where kubectl and helm are available and with access to the Kubernetes control plane. On subsequent nodes --skip-kubernetes-shared may be used to reduce the size of the collected data.

The resulting bundle is a single compressed archive (tar/gz by default) that can be transferred to PowerFlex support for further analysis.

Only one instance of pfmp_support can run on a host at a given time. If there is not enough free space for its output, it will refuse to run (unless space checking is explicitly skipped).

OPTIONS

General Options

• -A DIR, --installer-root-path=DIR

  Use DIR as the base directory of the PowerFlex Manager Installer. Useful when collecting PowerFlex Installer data and the installer container (atlantic_installer) isn't running. Default: automatically detected, or /var.

• -d OUT_DIR, --output-dir=OUT_DIR

  Store the resulting bundle under directory OUT_DIR. Default: <WORK_DIR>/powerflex-pfmpsupport (see --work-dir).

• -h, --help

  Show the help message and exit. When combined with --tech, also display technician options.

• -J, --xz

  Use tar/xz format for the collected bundle instead of the default tar/gz. Ignored if the system's tar(1) does not support --use-compress-program or xz(1) is not found.

• -n NAMESPACE, --namespace=NAMESPACE

  Collect additional data from application-specific Kubernetes namespace NAMESPACE. Default: powerflex.

• -N, --skip-space-check

  Skip free disk space verification before data collection.

• -P PATH, --collect-path=PATH

  Collect the additional path PATH. Only absolute paths are accepted. Accepts wildcards; wildcards should be quoted. This option can be specified multiple times to collect multiple paths.

• -q, --quiet, --silent

  Suppress messages on standard output.

• -s, --skip-kubernetes-shared

  Skip collection of shared (cluster-wide) Kubernetes data. Per-node Kubernetes data and host data is still collected. Useful for multi-node collections to reduce size; cluster data should be collected from at least one node.

• -w WORK_DIR, --work-dir=WORK_DIR

  Use directory WORK_DIR for temporary files. Default: /tmp.

• -x FILE, --output-file=FILE

  Store the collected bundle as file named FILE. The appropriate file name suffix (.tgz, .zip, etc.) is added automatically. If FILE is - (dash), write the bundle to standard output (implies --quiet). When the bundle is written to standard output, no bundle file is created on disk. Default: pfmpSupport.

• -z, --zip

  Use zip format for the collected bundle instead of the default tar/gz. Ignored if zip(1) is not found on the system.

• --overwrite-output-file

  Overwrite the output file if it already exists. When an output file or directory is explicitly specified (via -x or -d), the default behavior is to refuse to overwrite; this option overrides that.

• --tech

  Include technician options in the help message output.

Technician Options

The following options are intended for use by support technicians and are shown in the help message only when --tech is specified.

• --keep-work-dir

  Retain the generated temporary work directory after bundle creation (normally cleaned up automatically).

• --old-pod-logs

  Collect older Kubernetes pod logs from local rsyslog storage. When omitted, the current (kubectl logs) and previous (kubectl logs --previous) container logs are still collected.

BUNDLE STRUCTURE

The output bundle is a single compressed archive.

• The bundle top-level directory is the hostname of the collected system.
• General host command outputs go into a server/ subdirectory. File name is <command> + <arguments> + suffix (.txt by default). Spaces replaced with _, non-alphanumeric characters stripped.
  Example: 

  server/ip_-s_addr.txt - output of ip -s addr

• kubectl, helm, and docker command outputs go into their respective subdirectories. They are further organized by scope, <namespace>/<pod>/<container>, where applicable.
  Command name, scope, and output format related arguments are stripped. File name is <subcommand> + <arguments> + suffix (.txt by default). Spaces replaced with _, non-alphanumeric characters stripped.
  kubectl exec command outputs are stored under exec/ of the pod's subdirectory.
  Examples:
  • kubectl/version.txt - output of kubectl version
  • kubectl/all-namespaces/get_pods.yaml -- output of kubectl -o=yaml get --all-namespaces pods
  • kubectl/powerflex/asmui-7d6944dcb8-zqqtk/asmui/logs_--previous.txt - output of kubectl logs -n powerflex --container asmui --previous asmui-7d6944dcb8-zqqtk
  • kubectl/powerflex/nats-server-box-6d685b4b7d-cf278/nats-box/exec/nats_stream_report.txt - output of kubectl exec -n powerflex --container nats-box nats-server-box-6d685b4b7d-cf278 -- nats stream report
  • helm/powerflex/get_all_asmmanager.txt - output of helm get -n powerflex all asmmanager
  • docker/ps_-a.txt - output of docker ps -a
• Files extracted from pod containers are placed under the container's directory at their in-container file-system path.
  Example: kubectl/powerflex/thin-deployer-5657659bc7-gzbq4/thin-deployer/opt/Dell/ASM/deployments/8aaa809298a9efe10198aa58c8874f35/deployment.log - copy of /opt/Dell/ASM/deployments/8aaa809298a9efe10198aa58c8874f35/deployment.log extracted from within the thin-deployer container of the thin-deployer-5657659bc7-gzbq4 pod
• Host file-system files are placed at their file-system path relative to bundle root.
  Examples:
  • etc/os-release - copy of /etc/os-release
  • var/log/messages - copy of /var/log/messages
  • proc/cpuinfo - copy of /proc/cpuinfo
  • var/lib/rancher/rke2/agent/logs/kubelet.log - copy of /var/lib/rancher/rke2/agent/logs/kubelet.log
• Hidden files (dot-prefixed) are "unhidden" by removing the leading dot.
• Utility execution log, pfmp_support.log, placed directly under the <hostname>/ root.

Bundle directory tree structure:

<hostname>/
|-- pfmp_support.log                   Utility execution log
|-- server/                            General command output directory
|   |-- ip_-s_addr.txt
|   |-- uptime.txt
|   |-- uname_-a.txt
|   |-- ps_-elF.txt
|   |-- dmesg_-T.txt
|   +-- ...                            (one file per collected command)
|
|-- kubectl/                           kubectl command output directory
|   |-- version.txt                    Non-namespace-specific commands
|   |-- ...
|   |-- all-namespaces/                Cross-namespace commands
|   |   |-- get_pods.txt
|   |   |-- get_pods.yaml
|   |   +-- ...
|   +-- <namespace>/                   Per-namespace Kubernetes data
|       +-- <pod>/                     Per-pod data
|           +-- <container>/           Per-container data
|               |-- logs.txt           Current container logs
|               |-- exec/              Commands executed within container (if any)
|               |   +-- ...
|               |-- <dir>/             Files extracted from container (if any)
|               +-- ...
|-- helm/                              helm command output directory
|-- docker/                            docker command output directory
|
|-- etc/                               Host files
|   |-- os-release
|   |-- sysconfig/
|   |-- network/
|   +-- ...
|-- var/
|   |-- log/
|   |   |-- messages
|   |   +-- ...
|   |-- lib/rancher/rke2/
|   +-- ...
|-- proc/
|   |-- cpuinfo
|   |-- meminfo
|   +-- ...
|-- sys/
|-- ...
|
|-- powerflex-pfmpsupport-extra/       Extra diagnostic data (if present)
+-- powerflex-pfmpsupport-backup/      Backed-up configuration files (if any)

DISK SPACE

Disk space requirements for temporary files and the resulting bundle can vary considerably.

The utility attempts to minimize temporary space usage; it is limited to command outputs and copies of collected virtual file system (/proc and /sys) files.

To minimize disk space usage on the PowerFlex Manager host, the bundle can be streamed from a remote host with --output-file=-. When streaming, the bundle file is written directly to standard output (stdout); it is not created on disk.

Before collecting data, the utility estimates the required disk space for both the temporary work directory and the output bundle.

If the estimated required space exceeds the available space on the relevant file system(s), the utility terminates with an error. This check can be bypassed with --skip-space-check.

The work directory and the output directory may reside on different file systems; each is checked independently.

The estimated space requirements are written to the utility's log file, pfmp_support.log.

EXIT STATUS

FILES

ENVIRONMENT

Prerequisites

• The utility must be run as root (or a user with sufficient privileges to read system files, execute diagnostic commands, interact with the Kubernetes cluster, and access /proc, /sys, etc.).
• Standard utilities: tar, gzip, stat, find, awk, sed, getopt(1) (enhanced), nice.
• Optional: zip (for --zip), xz (for --xz).

Concurrency

Only one instance of pfmp_support may run at a time. The utility checks for an existing running instance via pidof(1) and terminates if one is found.

Signal Handling

The utility traps INT, EXIT, and TERM signals during data collection. Upon receiving a signal, it:

1. Restores any backed-up configuration files.
2. Cleans up temporary directories.
3. Exits with status 1.

The execution log is preserved and its path is printed to standard error.

EXAMPLES

Collect a standard support bundle:

pfmp_support

Collect bundles from multiple nodes in a Kubernetes cluster.

• On one node with kubectl and helm access, collect all Kubernetes data:

  pfmp_support

• On subsequent nodes, skip shared Kubernetes data:

  pfmp_support --skip-kubernetes-shared

Stream a bundle over an SSH connection, without creating a bundle file on the remote PowerFlex Manager host:

ssh <host> 'pfmp_support --output-file=-' > pfmpSupport-<host>.tgz

Use a different work directory to avoid filling up /tmp:

pfmp_support --work-dir=/var/tmp

Include custom paths in the bundle:

pfmp_support --collect-path=/opt/custom/app/logs --collect-path='/var/log/app*'