Dell Unity: Performing VDM Migration File Import with VNX dual Control Station environments can fail with Error Code:0x90001b6 or 0x900010b

When performing File Import operations from Unity, and if the source VNX1/VNX2 is running with dual Control Stations, there is the possibility that the Import Connection/Discovery, or any Import operation, could fail with the following:

GUI Example 1:
Create Remote Connection
Completed With Problems
(8/31/2016, 7:10:56 AM)
Details
Create remote system
Completed
Discover import objects 10.240.160.80
Failed
Details
Failed: The remote sys user home directory does not exist. Solution: for dual-control station VNX, failover control station via command '/nasmcd/sbin/cs_standby -failover'. Delete and re-create VNX remote system. (Error Code:0x90001b6)

GUI Example 2:
Create Remote Connection
Failed: Internal error: VNX command parser failed. (Error Code:0x900010b)

When performing certain Import operations, such as setting up the Import Connection from Unity, the Unity system connects to the VNX system with the provided SP IP address and credentials.  Once connected, the Unity system discovers the Control Station and logs in as the sysadmin@storageDomin user to perform File object discovery.  However, the discovery process will fail if the VNX system is failed over and running on CS1, since the /home/sysadmin directory is not normally present when running on CS1. 

This issue could also be seen if VNX Control Station 0 is missing the /home/sysadmin directory, or does not have all its logon and logoff profile files populated in the /home/sysadmin directory, even if CS0 is running normally as the primary.

Issue One Resolution: 
1)  Verify that your VNX system is running normally with CS0 as the primary Control Station. 
2)  If the VNX system is running with CS1 as the primary, you must fail back to CS0 before performing any File Import operations from Unity.  Use /nas/sbin/cs_standby -failover to failback to CS0, and wait at least 15 minutes for NAS Services to be fully restored on CS0 before continuing.
3)  Retry the Import Connection/Discovery step.

Issue Two Resolution:
If you are already running normally with CS0 as the primary, and are still seeing Error 0x90001b6, it is possible that the /home/sysadmin directory is missing or not properly provisioned.  Use the steps below to fix the /home/sysadmin directory issue.

1)  Create a script on CS0 named "fix_home.sh", make the script executable (chmod +x script_name), and execute the script.
Script Contents (copy following to notepad, then 'vi fix_home.sh' on CS0, and paste contents:

#!/bin/bash
MF=/etc/pam.d/system-auth
serial=`/nas/sbin/serial`
i=`grep -c "#added by EMC" $MF`

if [ "$i" -eq "0" ]; then
        echo "session     optional    pam_mkhomedir.so silent umask=0077 #added by EMC" >> $MF
        echo "Modification for $MF has been performed on primary CS of $serial box"
        else
        echo "No modification for $MF has been performed on primary CS of $serial box"
fi

ssh emcnasotherCS_i0 "grep -q 'added by EMC' /etc/pam.d/system-auth" 2>/dev/null
if [ $? -eq 1 ]; then
        ssh emcnasotherCS_i0 "echo 'session     optional    pam_mkhomedir.so silent umask=0077 #added by EMC' >> $MF" 2>/dev/null
        echo "Modification for $MF has been performed on secondary CS of $serial box"
else
        echo "No modification for $MF has been performed on secondary CS of $serial box"
fi

2)  Make the script executable (chmod +x) and then execute on CS0.  The script will place the following entry in CS0 & CS1's /etc/pam.d/system-auth file:
session     optional    pam_mkhomedir.so silent umask=0077 #added by EMC
3)  Login to CS0 via SSH as sysadmin@storageDomain using the sysadmin password
Note:  This will create the proper login/logoff scripts for the sysadmin user account, if they were previously missing
4)  Fail over to CS1, wait 15 minutes until the navicli -h <spa> domain -list shows the Control Station IP for CS1 on the list
5)  Login to CS1 via SSH as sysadmin@storageDomain
Note:  This will create the /home/sysadmin directory and all the profile files
6)  Fail back to CS0, wait 15 minutes for NAS Services to be restored and the domain -list shows CS0's IP address on the list

Caution:  Even though this script now maintains the /home/sysadmin directory on both Control Stations, it does not mean that File Import operations are supported when running on CS1--you should still failback to CS0 before running File Import tasks.

The following output is provided as examples of what the /home/sysadmin ownership, permissions, and profile files should look like:
[root@vnx8000 nasadmin]# /nas/sbin/getreason
10 - slot_0 primary control station  -->Slot_0 is CS0, reason code 10 indicates NAS Services are running
11 - slot_1 secondary control station

# ls -la /home
drwx------  3 sysadmin43 nasadmin  4096 Sep  8 13:40 sysadmin43  -->sysadmin user directory with ownership and permissions

[root@vnx8000 nasadmin]# tail /etc/passwd
sysadmin43:x:546:201::/home/sysadmin43:/bin/bash

[root@vnx8000 sysadmin43]# ls -la  -->Bash logon profile files, permissions and ownership
-rw-r--r-- 1 sysadmin43 nasadmin   33 Sep  8 12:13 .bash_logout
-rw-r--r-- 1 sysadmin43 nasadmin  763 Sep  8 12:12 .bash_profile
-rw-r--r-- 1 sysadmin43 nasadmin  131 Sep  8 12:11 .bashrc

Example content of the above files:
[root@vnx8000 sysadmin43]# cat .bashrc
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi

# User specific aliases and functions


[root@vnx8000 sysadmin43]# cat .bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH
NAS_DB=/nas
export NAS_DB
MANPATH=/usr/share/man:/usr/man:$NAS_DB/man
export MANPATH
PATH=$PATH:$NAS_DB/bin
export PATH
/nasmcd/.emc_login
alias useradd='echo Please use EMC Unisphere for User/Group management.'
alias userdel='echo Please use EMC Unisphere for User/Group management.'
alias usermod='echo Please use EMC Unisphere for User/Group management.'
alias groupadd='echo Please use EMC Unisphere for User/Group management.'
alias groupdel='echo Please use EMC Unisphere for User/Group management.'
alias groupmod='echo Please use EMC Unisphere for User/Group management.'


[root@vnx8000 sysadmin43]# cat .bash_logout
# ~/.bash_logout

/usr/bin/clear

Dell Unity 300, Dell EMC Unity 300F, Dell EMC Unity 400, Dell EMC Unity 400F, Dell EMC Unity 500, Dell EMC Unity 500F, Dell EMC Unity 600, Dell EMC Unity 600F, Dell EMC Unity Family |Dell EMC Unity All Flash, Dell EMC Unity Family , Dell EMC Unity Hybrid ... View More View Less