IBMi SRDF Migration Procedure

When customers run their IBMi client Logical Partitions (LPAR’s) on DELL EMC VMAX/PMAX external disk storage and want to refresh their current array to a new generation, they can leverage SRDF to run a background data migration and cutover to the new array with minimal downtime impact. This usage of SRDF is also referred to as SRDF/Data Mobility.
For customers where no downtime is allowed at all, see another article describing IBMi Non-Disruptive Migration (NDM) for VMAX/PMAX, found at this link: 
https://www.dell.com/support/kbdoc/en-us/000193832/vmax-powermax-non-disruptive-migrations-for-the-ibmi-host-platform

For any IBMi LPAR, ALL volumes must be included in the SRDF replication. This includes the Load Source (=bootdisk), due to the different nature of the IBMi platform with its unique LIC/OS design based on the AS/400 single-level-storage architectural design. For any IBMi storage replication, it is ALL or NOTHING. See the figure below for the description of the (temporary) setup during the migration.

The OLD and NEW source arrays in the PROD-DC are connected with temporary SRDF links for the migration. These will typically run through the SAN switches as well, so these must be zoned accordingly. Alternatives are a "direct connect" setup, or customers may be using Gige IP links (Gigabit Ethernet connections through LAN switches).


While the IBMi LPAR’s remain operational, the SRDF relationships between OLD and NEW arrays are created. The background asynchronous synchronization process of all the volumes' data is transparent to the IBMi host and its applications. New SAN connections from the existing SAN Fabrics to the new array must be created. For the existing IBMi FC or vFC host adapters, SAN zoning must be configured accordingly. When new connections and zoning are ready, verify on the new PMAX that the IBMi adapter WWPN’s are logging into the respective FA-ports on the new PMAX.

NOTE: For the IBMi platform, only "single-initiator>single-target" SAN-zoning must be used.

NOTE: Validate the IBMi adapter WWPN login from Unisphere or with SymCLI (sample) command: symaccess -sid 123 list logins -dirport 1c:0
======================================================================================================
Prepare Phase (sample commands):

Check SRDF connectivity from NEW array with SymCLI symsan command:

symsan list -sanrdf -sid 000420200123 -dir ALL -port ALL

Create the new IBMi target devices on the new array and add to a new StorageGroup (SG):

symdev create -sid 123 -tdev -emulation as400 -cap 82400 -captype cyl -N 64 -v -nop
symsg -sid 123 create sg_ibmi_lpar1_asp1_1 -srp SRP_1 -slo diamond
symaccess -sid 123 -name sg_ibmi_lpar1_asp1_1-type storage add devs 100-13F

Create respective InitiatorGroup(IG) and PortGroup(PG) on the NEW array:

symaccess -sid 123 create -name ig_ibmi_lpar1_asp1_1 -type init
symaccess -sid 123-name ig_ibmi_lpar1_asp1_1 -type init set ig_flags on OS2007 -disable
symaccess -sid 123-name ig_ibmi_lpar1_asp1_1 -type init add -wwn 0123456789abcde2
symaccess -sid 123-name ig_ibmi_lpar1_asp1_1 -type init add -wwn 0123456789abcde4
symaccess -sid 123-name ig_ibmi_lpar1_asp1_1 -type init add -wwn 0123456789abcde6
symaccess -sid 123-name ig_ibmi_lpar1_asp1_1 -type init add -wwn 0123456789abcde8
symaccess -sid 123-name ig_ibmi_lpar1_asp1_1 -type init add -wwn 0123456789abcde3
symaccess -sid 123-name ig_ibmi_lpar1_asp1_1 -type init add -wwn 0123456789abcde5
symaccess -sid 123-name ig_ibmi_lpar1_asp1_1 -type init add -wwn 0123456789abcde7
symaccess -sid 123-name ig_ibmi_lpar1_asp1_1 -type init add -wwn 0123456789abcde9

symaccess -sid 123 create -name pg_ibmi_lpar1_asp1_1 -type port -protocol SCSI_FC
symaccess -sid 123-name pg_ibmi_lpar1_asp1_1 -type port add -dirport 1c:0
symaccess -sid 123-name pg_ibmi_lpar1_asp1_1 -type port add -dirport 1c:1
symaccess -sid 123-name pg_ibmi_lpar1_asp1_1 -type port add -dirport 2c:0
symaccess -sid 123-name pg_ibmi_lpar1_asp1_1 -type port add -dirport 2c:1

Create a new temporary dynamic SRDF group between OLD and NEW array:

symrdf addgrp -sid 000420200123 -rdfg 100 -remote_sid 000 000297800456 -remote_rdfg 100 -dir 1D:03,2D:03,1D:07,2D:07 -remote_dir 1E:03,2E:03,1E:07,2E:07 -label LPAR1_MIG

Perform CreatePair for the respective (SG) and start the sync in adaptive copy disk mode:

symrdf createpair -sid 456 -type R1 -rdfg 100 -sg sg_ibmi_lpar1_asp1_1 -remote_sg sg_ibmi_lpar1_asp1_1 -establish -rdf_mode acp_disk

Monitor the sync process:

symrdf -sid 456 -rdfg 100 -sg sg_ibmi_lpar1_asp1_1 query

Due to the nature of the SRDF "adaptive copy" link state, with active I/O’s still coming into the source array from the IBMi host, the links may not reach a "synchronized" state, this is normal and expected.
When the sync process has copied the bulk of the data to the new array (less than 1000 outstanding invalid tracks), the offline cutover can be done according to customer planned maintenance window. We recommend planning for a 2-hour outage of the business apps on the host(s) that are migrated. That should provide adequate time for the stopping of application jobs and user sessions, powerdown of the system, performing the cutover, IPL the system, and restarting the application subsystems and jobs.
 
Cutover Phase (sample commands):

Check SRDF links and device pair states:

symrdf -sid 456 -rdfg 100 -sg sg_ibmi_lpar1_asp1_1 query

End applications, subsystems, and active users on LPAR.
PWRDWNSYS the LPAR.
Monitor the HMC to confirm LPAR is down in "Not Activated" state.

Set SRDF links in SYNC mode.

symrdf -sid 456 -rdfg 100 -sg sg_ibmi_lpar1_asp1_1 set mode sync

Monitor for status change and until all devices are "synchronized" with zero invalid tracks.

symrdf -sid 456 -rdfg 100 -sg sg_ibmi_lpar1_asp1_1 query

Split SRDF links.

symrdf -sid 456 -rdfg 100 -sg sg_ibmi_lpar1_asp1_1 split

NOTE: The SRDF Split process leaves the source volumes untouched and unchanged during the further migration process. This old SRDF source image holds a consistent "point-in-time" copy of the LPAR data from the time when the split was performed. Since the IBMi LPAR was down during the split, this image is 100% consistent. In a split-state, both the R1 and R2 copies are RW_enabled for the host. This R1-image on the OLD array can be used for an immediate fallback to the OLD array. In case any unexpected issues are encountered and the migration is cancelled. In that case the zoning/masking to OLD array must be re-instated.

After split, monitor for status change.

symrdf -sid 456 -rdfg 100 -sg sg_ibmi_lpar1_asp1_1 query

Create a backup of OLD array masking database.

Symaccess -sid 456 list view -v -detail>masking-456_<date>.txt

Delete OLD array masking for the respective IBMi LPAR.

Symaccess -sid 456 delete view mv_ibmi_lpar1_asp1_1

Create NEW array masking.

symaccess -sid 123 create view -name mv_ibmi_lpar1_asp1_1 -sg sg_ibmi_lpar1_asp1_1 -pg pg_ibmi_lpar1_asp1_1 -ig ig_ibmi_lpar1_asp1_1

Activate the LPAR from the HMC again with normal B-IPL mode.
Monitor the IPL process from the HMC.

At the sign-on screen, sign on with SST access authorization. Run STRSST and check status of disks and disk paths. Note the IBMi disk serial number change that reflects the new array volumes and serial ID.

Customer/apps admin team can now resume all normal operations on this LPAR.

This completes the migration.


Post-Migration Phase (sample commands):

When the customer has declared a successful cutover (no fallback required), the temporary configuration can be cleaned up.

Cleanup of SRDF migration device pairing.

symrdf deletepair -sid 456 -rdfg 100 -sg sg_ibmi_lpar1_asp1_1

symrdf removegrp -sid 000420200123 -rdfg 100

Clean up the SAN zoning on respective Switches:
From the IBMi host to the OLD array.
And remove SRDF zones from OLD to NEW array (after last migration is completed).