PowerStore: How to generate and collect various logs from PowerStore

Support Materials

• If the issue being investigated is related to PowerStore replication or metro volumes, logs from all clusters and appliances should be collected and provided to Technical Support.
• If the PowerStore is not yet initialized (no management IPs yet), see PowerStore: How to connect to and collect support materials from an un-configured system.

Using the PowerStore Manager user interface

1. Log in to the PowerStore Manager user interface (UI) with your username and password.
2. Click the Settings menu or the system name.
3. Go to Gather Support Materials under the Support section.
4. Press the GATHER SUPPORT MATERIALS button. A new submenu appears on the right side.
5. The default options should be:
   • Description: Optional
   • Collection Timeframe (v3+ only): Leave is set to default (All) unless instructed to select something else by Technical Support.
   • Object Type: Appliances
   • Objects to collect data for: Select all appliances.
   • Send materials to Support when finished: Optional (requires SupportAssist).
   • Advanced support materials collection options: Do not select unless support asks for it. Note: Selecting the advanced options greatly increases the size of the Support Materials. Only chose this option when instructed to do so by your support representative.
6. Press START. This process may take up to 60 minutes depending on the size of the system.
7. Monitor the progress in the Jobs section of PowerStore Manager or within the Support Materials section.
8. Once the process above has finished, select the bundle to download from the Support Materials Library under the Gather Support Materials section.
9. Press DOWNLOAD or SEND TO SUPPORT
10. A submenu is displayed on the right side with a download link for each of the appliances.
11. Click Support material collection for all the appliances.

Using ssh/cli

Before you begin:

• IMPORTANT: Do not attempt to connect a USB device to the USB port on the PowerStore node as this may lead to unpredictable system behavior. For further details see PowerStore: Universal Serial Bus (USB) Port Use Guidelines.
• Running and downloading logs from CLI/ssh should not be required unless the PowerStore Manager user interface is not accessible.
• Review the article PowerStore: Understanding shell limitations due rbash and other security implementations
• Each appliance only contains its own logs. So if it is a multi-appliance cluster, connect with WinSCP to each appliance separately and download the respective logs. The details about each appliance IP address can be found in the PowerStore Manager user interface under Settings > Network IPs.

How to:

1. Log in to the system over ssh with the service user and password.
2. The service command to generate Support Materials is svc_dc.
   For a multi-appliance system, include all appliances in the collection, example for a two appliance cluster:

   svc_dc run -a A1,A2

   
   For detailed usage, run svc_dc -h or svc_dc run -h, or see the PowerStore Service Scripts Guide available from the Dell Support pages.
3. When the logs finish, download them.
   In order to facilitate this process, links to the logs are automatically created in the home directory (/home/service/user) when the logs finish running. List them with ls -l. If for some reason the link is not created, list the files manually, this is how that can be done:
   
   List the available Support Materials (DCs) with the service command svc_dc list. Example:

   [SVC:service@xxxxxxx-A user]$ svc_dc list start_timestamp      id             status   dump     description       profile     appliances       2020-04-08 12:48:37  xxxx-xxxx-xxxx SUCCESS  No                         essential   FNM00xxxxx..  2020-04-08 14:36:42  xxxx-xxxx-xxxx SUCCESS  No       Post NDU          detailed,.. FNM00xxxxx..  2020-04-09 13:33:57  xxxx-xxxx-xxxx SUCCESS  No                         detailed    FNM00xxxxx..  2020-04-15 15:05:34  xxxx-xxxx-xxxx SUCCESS  Yes      2020-04-15 Dail.. essential.. FNM00xxxxx..  2020-04-16 15:23:07  xxxx-xxxx-xxxx SUCCESS  No       2020-04-16 Dail.. essential   FNM00xxxxx..  2020-04-16 19:07:41  xxxx-xxxx-xxxx SUCCESS  No       Daily data coll.. essential   FNM00xxxxx..

   Here you can see the dates, id, status, if there are dumps, description, profiles used (svc_dc list_profiles), and the appliance the logs correspond to.
   Get the id and run the list command with it to see the full details, including file locations and so on.
   
   Example of the command (output not shown):

   [SVC:service@xxxxxxx-A user]$ svc_dc list xxxx-xxxx-xxxx

   If the bundle ID contains logs for multiple appliances, the full path to the files on each respective appliance is listed in the output.
    
4. Download the logs. There are various ways to download the log file bundles over cli/ssh from the appliances. Here are a few.

Performance Metrics Archives

Using PowerStore Manager

Using ssh/cli

1. Use PSTCLI to generate Performance Metrics. The PSTCLI command is:

   pstcli -service support_metrics_archive

   Either on or off node PSTCLI versions can be used. For full details on how to use PSTCLI, see PowerStore CLI Guide.
   
   Example of how to generate a performance archive bundle for the previous couple of days (offset in seconds). Example using on-node PSTCLI:

   pstcli -service -d ClusterIP -u admin -p MyPassword support_metrics_archive generate -start_offset 172800 -end_offset 0 Success  #  |              archive_id ----+--------------------------------------   1 | 6e869e4e-2612-45ed-b531-6884fa4c802a

   
   Other examples:
   • Last day until today: -start_offset 86400 -end_offset 0
   • Previous day only: -start_offset 172800 -end_offset 86400
   • Previous week until today: -start_offset 604800 (if -end_offset is not provided, data is captured until the present time).
     Note: These are only examples. A full week of performance data is not always available. If the performance event to investigate is older than two days, the performance data is likely not available anymore.
   The generated performance archive is found inside the following directory, where xxxx is a date folder or folders specific to your system.

   ls -lha /cyc_cfs/service/support_metrics/archived/xxxx

   The file format is PowerStore_ClusterID_StartDate_EndDate_support-metrics.tar.gz.
   Example: PowerStore_abcdefg_2020-04-21T12-56-32_2020-04-28T12-56-32_support-metrics.tar.gz
2. Download the file to a local machine directly with off-node PSTCLI, pscp as shown above, or copy the file to /home/service/user in order to download with WinSCP.
   
   Using off-node PSTCLI:
   Example using the off-node PSTCLI on a windows machine with the ID from above:

   pstcli -service -d ClusterIP -u admin -p MyPassword support_metrics_archive -d "C:\Temp" -archive_id 6e869e4e-2612-45ed-b531-6884fa4c802a download Downloaded 81920000 (78.12M) at 3.39 MB/s ... Download complete.

   The file is downloaded to the specified path (-d). If no path is specified, the file is downloaded to where your CMD path defaults to (this can be seen in the command prompt itself within CMD, example: C:\>Users\userX).
   
   Using on-node PSTCLI or direct copy to download with WinSCP:
   If running PSTCLI from the primary node directly, copy the file to the home directory, and then transfer with WinSCP.
   This can be done with the same command that is shown above (without the second -d switch or with -d /home/service/user).
   
   You can also manually copy the file to /home/service/user as shown here:
   List the files available by running

   ls -lha /cyc_cfs/service/support_metrics/archived/

   to determine the unique sub directory name. And then:

   [SVC:service@xxx-A user]$ ls -lha /cyc_cfs/service/support_metrics/archived/2021-03-09T12-09-50-512325/ total 183548 -rw-rw-r-- 1 root cycg 187945656 Mar  9 12:11 PowerStore_xxx_2021-03-02T12-09-50_2021-03-09T12-09-50_support-metrics.tar.gz

   
   Copy the file to your current directory (/home/service/user). The dot (.) at the end of the command denotes the target directory:

   [SVC:service@xxx-A user]$ cp /cyc_cfs/service/support_metrics/archived/2021-03-09T12-09-50-512325/PowerStore_xxx_2021-03-02T12-09-50_2021-03-09T12-09-50_support-metrics.tar.gz .

   
   Once the file is copied, download it with WinSCP.

Dump files

Using PowerStore Manager

Using ssh/cli

PowerStore X (Hypervisor logs)

Using PowerStore Manager

Using ssh/CLI

From vSphere

PowerStore Metro Witness

PowerStore UI/REST logs

Switch Logs

External storage import logs

Linux host plug-in

Windows host plug-in

EqualLogic

VPLEX and metro node

--