MFE SCF: SRDF/STAR MSC activation got OBT SEL failed RC 8 and then disabled
Summary: One MSC group in the GDDR C-system was performing an activation, received message SCF15C4W, OBT SEL failed RC 8, and then disabled.
Symptoms
One MSC group in the GDDR C-system was performing activation, received message SCF15C4W, OBT SEL failed RC 8, and then disabled. An example is given below.
00.19.52 STC02874 SCF1335I MSC - GROUP=mscgrp1 MOTHER TASK STARTED 00.19.52 STC02874 SCF1319I MSC - SET TO ADCOPY-DISK ON SRDFA DROP 00.19.52 STC02874 SCF1568I MSC - GROUP=mscgrp1 WEIGHT FACTOR = 0 00.19.52 STC02874 SCF1328I MSC - GROUP=mscgrp1 (080B4,007F) SRDFA ACTIVE 00.19.52 STC02874 SCF1569I MSC - GROUP=mscgrp1 STEAL LOCK AFTER = 10 MIN(S) 00.19.52 STC02874 SCF1326I MSC - GROUP=mscgrp1 (080B4,007F) SERIAL = 000197600361 00.19.52 STC02874 SCF1426I MSC - GROUP=mscgrp1 (080B4,007F) GETTING SEL LOCKS 00.19.52 STC02874 SCF15C4W MSC - GROUP=mscgrp1 (080B4,007F) R1 OBT SEL failed for LOCKID X'02A6B248', RC 8 00.19.52 STC02874 SCF1435I MSC - GROUP=mscgrp1 (080B4,007F) FREEING SEL LOCKS 00.19.53 STC02874 SCF1426I MSC - GROUP=mscgrp1 (080B4,007F) GETTING SEL LOCKS 00.19.53 STC02874 SCF15C4W MSC - GROUP=mscgrp1 (080B4,007F) R1 OBT SEL failed for LOCKID X'02A6B248', RC 8 00.19.53 STC02874 SCF1435I MSC - GROUP=mscgrp1 (080B4,007F) FREEING SEL LOCKS 00.19.54 STC02874 SCF1426I MSC - GROUP=mscgrp1 (080B4,007F) GETTING SEL LOCKS …… 00.20.22 STC02874 SCF1437I MSC - GROUP=mscgrp1 NOT ABLE TO OBTAIN ALL SEL LOCKS …… 00.21.22 STC02874 SCF1479I MSC - GROUP=mscgrp1 NOT ABLE TO OBTAIN ALL STAR SEL LOCKS 00.21.23 STC02874 SCF1435I MSC - GROUP=mscgrp1 (080B4,007F) FREEING SEL LOCKS 00.21.23 STC02874 SCF1480I MSC - GROUP=mscgrp1 (080B4,007E) FREEING STAR SEL LOCKS 00.21.28 STC02874 SCF1333I MSC - GROUP=mscgrp1 MOTHER TASK ENDED 00.21.28 STC02874 SCF15AAI MSC - GROUP=mscgrp1 DISABLE complete
Cause
Running MSC with high availability without sharing system enqueues, the storage system level locks are required to be obtained from the local and remote DASD subsystems. The issue occurred due to OBT SEL lock conflict, another MSC instance held the same local and remote locks. Both instances are using the same DASD subsystems.
GDDR allows C-system multitenancy, and in the GDDR manual, it mentions below:
'NOTE: C-instances can share Dell storage systems. However, MSC activation cannot occur simultaneously in two or more instances if they share a storage system. In this case, only one of the activations succeeds; the others have to be re-driven.'
Resolution
To avoid this issue, if using C-system multitenancy, do not simultaneously run MSC activation for multiple instances.