ViPR Controller: 在 VNX 上创建卷失败
Summary: ViPR Controller: 在 VNX 上创建卷失败
Symptoms
该问题发生在用户将 VNX 操作环境代码升级到以下版本之后:VNX:05.33.009.5.217
在未安装热修复程序的 05.33.009.5.231 上也观察到了此问题。
用户无法在 VNX 上创建卷。
请求会间歇性但定期地失败。
在运行 refsys、重新启动 SMI-S 服务器并重新启动 ECOM 后,问题仍然存在。
卷在失败后仍保留在阵列上。 使用相同卷名称重新提交订单将失败。
ViPR 控制器 UI 错误
[ERROR] Wed Aug 29 10:58:09 UTC 2018 Error 16000: Message: 作业失败:尝试更新卷属性时捕获异常:
找不到请求的对象。 (com.emc.cmp.osls.se.osl.Device.StorDevShow():280 C:ERROR_CLASS_SOFTWARE F:ERROR_FAMILY_NOT_FOUND R:1000052 L:2 C:ERROR_CLASS_SOFTWARE F:ERROR_FAMILY_NOT_FOUND R:1000052 No objects of the requested type were found : 2 : 7 : "The device name specified could not be found")
提醒:
此外,请参阅以下知识库文章:
523574 : ViPR Controller:ViPR Controller 要使用 Solutions Enabler 7.6.2.102/SMI-S 4.6.2.30,需要提交一份 RPQ https://support.emc.com/kb/523574(该文章需要 40 级访问权限)。
Cause
由于 SMI-S 未提供卷信息,因此 ViPR Controller 的回滚不会从阵列中删除卷。
Resolution
解决方案:
在确认了上文“问题”部分中显示的 ViPR Controller UI 错误后,针对每个要升级的 VNX,向“远程主动服务 (RCM):统一技术支持”团队
提交一个服务器请求。在请求中,需要获取 VNX HotFix ManagementServer_1.3.9.1.0306-1.upf(参考编号 AR 958882),并附上需要该热修复程序的原因,以及
当前的 VNX 阵列型号、VNX Block OE、Solutions Enabler、SMI-S 和 ViPR Controller 的版本信息。请参阅以下注释。
Additional Information
提醒: 卷创建失败时,后端卷将保留在原位,需要手动从阵列中删除。
ViPR C 日志
ViPR C 发出命令以创建卷:
Volume:<Volume Name removed> , IsThinlyProvisioned: true
vipr2 vipr2 controllersvc 2018-09-03 14:25:31,776 [1684|createVolumes|39a39976-b004-45e9-98c7-f1e463d25517c83c9b0f-fe18-485b-85c3-cdd8b25f2a8d] INFO SmisCommandHelper.java (line 201)
SMI-S Provider: <SMI-S Provider FQDN removed> -- Attempting invokeMethod CreateOrModifyElementFromStoragePool on objectPath=/root/emc:Clar_StorageConfigurationService.CreationClassName="Clar_StorageConfigurationService",Name="EMCStorageConfigurationService",SystemCreationClassName="Clar_StorageSystem",SystemName="CLARiiON+CKM<CLARiiON Serial Number removed>" with arguments:
inArg[0]=uint16 ElementType = 5;
inArg[1]=Clar_UnifiedStoragePool ref InPool = "root/emc:Clar_UnifiedStoragePool.InstanceID=\"CLARiiON+CKM<CLARiiON Serial Number removed>+<Pool Name removed>\"";
inArg[2]=uint64 Size = 10737418240;
inArg[3]=uint32 EMCNumberOfDevices = 1;
inArg[4]=string ElementName = "<Volume Name removed>";
inArg[5]=Clar_StoragePoolSetting ref Goal = "root/emc:Clar_StoragePoolSetting.InstanceID=\"CLARiiON+CKM<CLARiiON Serial Number removed>+D+T+3031128139+<Pool Name removed>\"";
Returned: 4096 with output arguments:
outArg=uint64 Size = 10737418240;
outArg=SE_ConcreteJob ref Job = "root/emc:SE_ConcreteJob.InstanceID=\"<1234567890>\"";
执行时间:0.591798 秒钟。
SMI-S Provider 日志:
SMI-S 接收订单并提供具体 ID。
Mon Sep 03 16:25:31 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 2013263616 StorageConfigurationService.BlockServices DEBUG EMC_StorageConfigurationService.cpp 1887 : 元素大小:10737418240
Mon Sep 03 16:25:31 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 2013263616 global DEBUG 163 : OslCache missing cache entry for array <Array type="CLARiiON" encoding="String" value="CKM<CLARiiON 序列号名称已删除>" />
Mon Sep 03 16:25:32 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 2013263616 global INFO Job.cpp 459 : 作业 <1234567890> 已提交
Mon Sep 03 16:25:32 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 2013263616 StorageConfigurationService.BlockServices DEBUG EMC_StorageConfigurationService.cpp 2705 : Submit jobId <1234567890> to run asynchronously
Mon Sep 03 16:25:32 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 2013263616 JobControl INFO JobTableMonitor.cpp 61 : 作业表监视器已启动。
Mon Sep 03 16:25:32 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 -1272031488 JobControl DEBUG JobTableMonitor.cpp 88 : 作业表监视器已启动。
Mon Sep 03 16:25:32 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 2013263616 JobControl DEBUG JobTable.cpp 107 : Added to job table job id: <1234567890>
Mon Sep 03 16:25:32 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 -1269930240 BlockServices.JobControlVolume DEBUG JOB_VolCreate.cpp 400 : 创建存储卷作业 ID:<1234567890>
Mon Sep 03 16:25:32 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 2013263616 JobControl INFO JobControl.cpp 86 :已启动的作业: <1234567890>
SMI-S 日志:
SMI-S 报告卷创建已完成:
Mon Sep 03 16:28:02 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 -1269930240 BlockServices.JobControlVolume DEBUG JOB_VolCreate.cpp 1891 : Completed performImmediateSync().
Mon Sep 03 16:28:02 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 -1269930240 BlockServices.JobControlVolume INFO JOB_VolCreate.cpp 1908 : 已创建卷:<Device name="<DeviceID replaced 12345>" />
Mon Sep 03 16:28:02 +0200 2018 <SMI-S Provider 名称已删除> ECOM 26037 -1269930240 global INFO Job.cpp 584 : 已标记作业 100% 已完成,作业 ID:<1234567890>
ViPR 日志:
ViPR 日志指示 SMI-S 作业已完成:
vipr2 vipr2 controllersvc 2018-09-03 14:28:02,413 [pool-58-thread-1] INFO SmisJob.java (line 197) Status value[0]: 17
vipr2 vipr2 controllersvc 2018-09-03 14:28:02,413 [pool-58-thread-1] INFO SmisJob.java (line 197) Status value[1]: 2
vipr2 vipr2 controllersvc 2018-09-03 14:28:02,413 [pool-58-thread-1] INFO SmisJob.java (line 200) SmisJob: <1234567890> 已成功
vipr2 vipr2 controllersvc 2018-09-03 14:28:02,413 [pool-58-thread-1] INFO SmisJob.java (line 243) SmisJob: 后期处理作业:ID <1234567890>,提供程序:<SMI-S Provider FQDN 已删除>
SMI-S 日志:
成功创建卷后,ViPR Controller 会查询 SMI-S 以更新数据库,但 SMI-S 无法提供卷的详细信息。
ViPR 日志:此错误由 SMI-S 传递给 ViPR C,导致订单失败。 由于在查询 SMI-S 时未找到该卷,因此在回滚时不会从阵列中删除该卷:
WBEMException: CIM_ERR_NOT_FOUND (The requested object cannot be found. (com.emc.cmp.osls.se.osl.Device.StorDevShow():280 C:ERROR_CLASS_SOFTWARE F:ERROR_FAMILY_NOT_FOUND R:1000052 L:2 C:ERROR_CLASS_SOFTWARE F:ERROR_FAMILY_NOT_FOUND R:1000052 No objects of the requested type were found : 2 : 7 : "The device name specified could not be found"))
at org.sblim.cimclient.internal.wbem.CloseableIteratorSAX.hasNext(CloseableIteratorSAX.java:121)
at org.sblim.cimclient.internal.wbem.WBEMClientCIMXML.getIterator(WBEMClientCIMXML.java:2134)
at org.sblim.cimclient.internal.wbem.WBEMClientCIMXML.getInstance(WBEMClientCIMXML.java:1257)
at com.emc.storageos.volumecontroller.impl.smis.job.SmisAbstractCreateVolumeJob.commonVolumeUpdate(SmisAbstractCreateVolumeJob.java:311)
at com.emc.storageos.volumecontroller.impl.smis.job.SmisAbstractCreateVolumeJob.processVolume(SmisAbstractCreateVolumeJob.java:219)
at com.emc.storageos.volumecontroller.impl.smis.job.SmisAbstractCreateVolumeJob.updateStatus(SmisAbstractCreateVolumeJob.java:120)
at com.emc.storageos.volumecontroller.impl.smis.job.SmisJob.poll(SmisJob.java:246)
at com.emc.storageos.volumecontroller.impl.job.QueueJobTracker.run(QueueJobTracker.java:90)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
vipr2 vipr2 controllersvc 2018-09-03 14:28:03,097 [pool-58-thread-1] INFO SmisAbstractCreateVolumeJob.java (line 372) Skipping step addVolumesToConsistencyGroup: volumes [<Volume URN removed>] do not reference a consistency group.
vipr2 vipr2 controllersvc 2018-09-03 14:28:03,097 [pool-58-thread-1] INFO SmisAbstractCreateVolumeJob.java (line 158) Updating status of job 39a39976-b004-45e9-98c7-f1e463d25517c83c9b0f-fe18-485b-85c3-cdd8b25f2a8d to SUCCESS
Created volume successfully .. NativeId: <DeviceID replaced 12345>, URI: <卷 URN 已删除>
vipr2 vipr2 controllersvc 2018-09-03 14:28:03,101 [pool-58-thread-1] INFO VolumeTaskCompleter.java (line 106) opType: CREATE_BLOCK_VOLUME 详细信息:VolumeCreateFailed:已创建卷。
vipr2 vipr2 controllersvc 2018-09-03 14:28:03,103 [pool-58-thread-1] INFO VolumeTaskCompleter.java (line 82) Bourne VolumeCreateFailed event recorded for Volume <卷 URN 已删除>
vipr2 vipr2 controllersvc 2018-09-03 14:28:03,126 [pool-58-thread-1] INFO WorkflowService.java (line 653) Updating workflow step: 39a39976-b004-45e9-98c7-f1e463d25517c83c9b0f-fe18-485b-85c3-cdd8b25f2a8d 状态错误:作业失败:尝试更新卷属性时捕获异常:无法找到请求的对象。 (com.emc.cmp.osls.se.osl.Device.StorDevShow():280 C:ERROR_CLASS_SOFTWARE F:ERROR_FAMILY_NOT_FOUND R:1000052 L:2 C:ERROR_CLASS_SOFTWARE F:ERROR_FAMILY_NOT_FOUND R:1000052 No objects of the requested type were found : 2 : 7 : "The device name specified could not be found")