AppSync: many XIO snaps made at once, intermittently snaps some LUNs twice but misses others

When multiple snaps of XtremIO source LUNs are made simultaneously, it is possible that some LUNs may be snapped twice while other LUNs may be missed completely. There are no errors in the AppSync GUI. The issue can only be seen by monitoring the XtremIO GUI to see the snaps created. 

The issue only occurs in when using AppSync to make XtremIO snaps in an environment that also contains VPLEX. This condition is intermittent and may be seen on one or more arrays. 

In the example below, notice that LUN 6 is not snapped while LUN 8 is snapped twice.

This issue condition is caused by XtremIO REST queries returning multiple WWNs for a query for a single LUN. 

When this happens, AppSync does not validate the correct WWN and will pick the first one in the list to be snapped. In the example, AppSync would proceed to snap LUN 00074 even though LUN 00010 was the target. 

05-10-2017 07:26:30.000 INFO [Thread-149 (HornetQ-client-global-threads-833118000)] [com.emc.archway.service.vplex.VPLEXServiceBean] [] [] CQ 110140 : Custom data for getstorageviewsinfoscontainingwwn for wwn <WWN> is: Views exporting volume VPD83T3:<WWN>:
    View Array01_id_SV exports (7,device_XtremIO0604_LUN_00074_1_vol,VPD83T3:<WWN>).
    View Array01_id_SV exports (7,device_XtremIO6873_LUN_00010_1_vol,VPD83T3:<WWN>).

Interim Resolution:

AppSync Engineering has provided a hot fix for AppSync 3.1.0.2. This hot fix will introduce additional validation so that the correct WWN is always picked. 

Provide this information when contacting EMC Customer Support Center or your EMC Customer Service representative for technical assistance. Please quote the solution ID.

Resolution:

AppSync Engineering is currently addressing this problem, but has not provided a fix in a released patch. This solution will be updated with the patch when it has been released.