Hi everybody,

From all Solutions Enabler and Symm information I've seen, I understand that a symm (or at least Solutions Enabler) distinguishes between three types of SCSI reservations: Exclusive, Group and Persistent (see emc223960 for example). The first two are visible and breakable using SE commands, while the last one is not visibible and not breakable using SE. Persistent reservations can only be placed on a device after setting the "SCSI3_persist_reserv" flag (also known as the PER flag) on the device, otherwise the device will not accept them.

Reading the t10 SCSI documentation, I see three command pairs handling reservations: RESERVE(6)/RELEASE(6), RESERVE(10)/RELEASE(10) and PERSISTENT RESERVE IN/PERSISTENT RESERVE OUT. If I understand correctly, the two RESERVE/RELEASE commands only handle reservations that allow a single initiator access to a LUN (these are called "Exclusive" by SE). In other words, when a symm receives a RESERVE(6) or RESERVE(10) command, there's no way this can result in a "Group" or "Persistent" reservation. In an active/active multi-path environment, the use of RESERVE(6) and RESERVE(10) is therefore limited.

The PERSISTENT RESERVE OUT command does offer a way of reserving a LUN for multiple initiators. emc212199 mentions that PowerPath on AIX translates RESERVE commands into PERSISTENT RESERVE OUT requests. That article calls PERSISTENT RESERVE OUT a "group reservation" (0x5F is the operation code for the PERSISTENT RESERVE OUT command). So, it seems that what SE calls "Group" reservations can only be placed by the PERSISTENT RESERVE OUT command.

None of our devices have the SCSI3_persist_reserv flag set, but our AIX machines place Group reservations without problems. This must mean that the symm does accept the PERSISTENT RESERVE IN and OUT commands, even though the SCSI3_persist_flag is not set. The SCSI documentation does mention an optional feature of the PERSISTENT RESERVE OUT command. Normally, reservations placed using these commands do *not* survive a power cycle of the storage array. When reserving a LUN, a host can optionally request for the reservation to be persistent even across power cycles. The reservation and how it behaves is exactly the same, the only difference is that the reservation survives a power cycle of the storage. However, this is an optional feature: the storage is free to not support (or disable) this capability.

Is this what the SCSI3_persist_reserv flag controls? Does it enable the optional ability for a device to accept a request for a reservation to be persistent across power cycles? Is what I describe above correct; i.e. does it more or less correctly describe how reservations work?

The reason I'm interested in these details is because we had a Windows server just suddenly bluescreen (simple setup, no clustering or shared storage, just a multipath 2003 server using PowerPath). The error message indicated that it unexpectedly lost access to its system device (where Windows was installed). After that, it turned out that the Symmetrix device was not accessible for anybody anymore. The bluescreening host wouldn't reboot anymore: the QLogic BIOS would see the LUN (over both HBA's) but it couldn't access it. Allocation the device to another (running) Windows-server was the same: the other server would see it, but couldn't bring it online: "The requested resource is currently in use." Even an AIX server couldn't access it, with the same error. All the while, "symdev -sid xxx list -resv" did *not* show any reservations. However, in the resulting case it was mentioned that the device did have a persistent reservation on it, which got me puzzled.

Regards,

Jurjen Oskam