Isilon: How to use the fstat command to list the open files on a node

Procedure

1. Open an SSH connection on any node in the cluster and log in using the "root" account.
2. Do any of the following:
   • To list the open files on a node, run the following command:
      

     fstat

     
      
   • To list the open files in a directory, run the following command, where <directory> is the directory that you want to examine:
      

     fstat -f <directory>

     
      
   • To list the files opened by a particular process, run the following command, where <pid> is the process ID that you want to examine:
      

     fstat -p <pid>

     
      

Example

In the following example, the fstat command is used to troubleshoot a /var directory that is near capacity. If no large files are found in the /var directory, this may indicate that a file has become unlinked. The unlinked file continues to consume disk space because a process has the file open. You can use fstat to see if this is true.

1. Open an SSH connection on any node in the cluster and log in using the "root" account.
2. Run the following command:
    

   fstat -f /var | grep /var

   
   
   The grepcommand removes the sockets and pipes from the output, making the output easier to interpret. If a process is holding a file open, output similar to the following is displayed:

   root winbindd 98281 4 /var 69612 -rw------- 100120000 rw

    

   In this example, the winbindd process (process ID 98281) has a file open that is approximately 100 MB (100,120,000 bytes). The inode number for the file is 69612.

3. Run the following command to display the file name for the file:
    

   find -x /var -inum 69612 -print

   
    
4. If a process is holding a large file open and the inode cannot be found, the file has become unlinked. In this case, you can stop and restart the process or processes that are holding the file open. For more information about how to stop and restart a process, see Using the killall -9 command to stop a process, 89421