Patch install fails with output similar to the following:
# isi upgrade patches install 9.2.1.14_GA-RUP_2022-07_PSP-2429.pkg --rolling
The below patches are deprecated by this patch and will be removed automatically:
- 9.2.1.7_GA-RUP_2021-12_PSP-1698
Would you like to proceed? (yes/[no]): yes
This patch request will require nodes in the cluster to be rebooted either in succession or simultaneously, depending on the options you supplied. Both scenarios can result in lost connectivity. Would you like to proceed? (yes/[no]): yes
Failed to install patch, 724933132 bytes in /var partition are required.
Security fixes introduced in patches for OneFS 9.1.0, 9.2.1, and 9.3.0 branches have greatly increased the size of patches from around 50MB in size to greater than 150MB in size. Due to the increased size of the patches, a substantial amount of available space in the /var partition is required when installing them. The available space requirements may not be achievable in some instances and this article provides steps to create symbolic links to directories in the /var/crash partition as a workaround.
Note: The steps in this workaround cannot be performed on Compliance mode enabled clusters
Note: It is only necessary to perform the following proactive workaround for the install of any OneFS 9.1.0, OneFS 9.2.1, or OneFS 9.3.0 Roll-Up Patches due to their increased size and should not be deployed for other patch installs or on other OneFS releases.
Pre-requisites
Prior to performing the proactive workaround of creating symbolic links to directories in the /var/crash partition there are a few pre-requisites that must be met:
1. The upgrade and patch frameworks must be in an idle/clean state
a. Upgrade framework check; confirm there are no active upgrade processes by reviewing the output of the following command:
b. Patch framework checks:
i. Confirm all patches are in an Installed state in the output of the following command:
ii. Confirm the patch system is not active on any nodes by running the following command:
isi_patch isactive
iii. Confirm there are no files in the /var/patch/tmp/ and /var/patch/catalog/ directories.
isi_for_array 'find /var/patch/catalog/ /var/patch/tmp/ -depth 1 -ls'
Note: If the two previous patch framework checks [i and ii] did not identify any issues and files are found in the /var/patch/tmp/ or /var/patch/catalog/ directories, it is safe to move those files to /ifs/data/Isilon_Support.
2. There must be sufficient available space in the /var [500MB] and /var/crash [1GB] partitions. Clean up of the partitions should be performed on any nodes reporting less than the required amount of available space.
a. The following command will have output on nodes with less than 500MB of available space in the /var partition:
isi_for_array 'df -m /var' | awk '{if ($5 < 500) { print }}'
b. The following command will have output on nodes with less than 1GB of available space in the /var/crash partition:
isi_for_array 'df -m /var/crash' | awk '{if ($5 < 1000) { print }}'
Proactive Workaround
NOTE: Do not forget to perform remaining steps (after patch install is completed)!
1. Prior to the patch install and after satisfying all of the pre-requisites, run the following sequence of commands to make the directories that will be used under the /var/crash partition and create symbolic links to them:
isi_for_array 'mkdir -p /var/crash/patch/tmp /var/crash/patch/catalog'
isi_for_array 'ln -sF /var/crash/patch/tmp /var/patch/tmp'
isi_for_array 'ln -sF /var/crash/patch/catalog /var/patch/catalog'
2. Perform the patch install as documented in the patch README
3. Upon completion of the patch install, run the following commands to remove the symbolic links:
isi_for_array 'test -L /var/patch/catalog && unlink /var/patch/catalog'
isi_for_array 'test -L /var/patch/tmp && unlink /var/patch/tmp'
4. Recreate directory and confirm permissions:
isi_for_array 'umask 037 && mkdir /var/patch/tmp /var/patch/catalog'
isi_for_array 'ls -ld /var/patch/tmp /var/patch/catalog'
cluster-1: drwxr----- 2 root wheel 512 Feb 20 23:37 /var/patch/catalog
cluster-1: drwxr----- 2 root wheel 512 Feb 20 23:37 /var/patch/tmp
cluster-2: drwxr----- 2 root wheel 512 Feb 20 23:37 /var/patch/catalog
cluster-2: drwxr----- 2 root wheel 512 Feb 20 23:37 /var/patch/tmp