PowerFlex SDNAS SDC 插槽关闭，出现错误 1006 连接断开

场景
由于具有本地异常套接字连接的 sdnas 客户端应用程序的调用功能中止网络相关问题，SDNAS 实例 SDC 间歇性地无法与 VxFlex 网关通信。

症状
在尝试执行卷“29eba43900000678”查询的sdnas_active日志中，SDNAS 使用的 HTTPS 命令收到错误代码“1006 Connection Disconnected”：

2020/07/22-07:51:00.464672 7f2e39821701 ADAPTER: 6:[GvSh_3] ScaleIOStorageAdapter::createHeaderJson:317 JSON headers <{"URI":"\/api\/instances\/Volume::29eba43900000678","HTTP_TREQ":1,"REQHEADER":{"Content-Type":"application\/json"}}>
2020/07/22-07:51:00.464682 7f2e39821701 ADAPTER: 6:[GvSh_3] HTTPAdapterCommunicator::requestReply:495 Enter

2020/07/22-07:51:00.464709 7f2e39821701 ADAPTER: 6:[SVDM_A] HTTPAdapterCommunicator::requestReply:593 :Sending via HTTP/S Client!!
2020/07/22-07:51:00.464732 7f2e39821701 ADAPTER: 6:[SVDM_A] HTTPAdapterCommunicatorReq::writeHeader:77
2020/07/22-07:51:00.464746 7f2e39821701 ADAPTER: 6:[SVDM_A] HTTPAdapterCommunicatorReq::writeHeader:112 KEY = Content-Type VALUE = application/json
2020/07/22-07:51:00.464774 7f2e39821701 ADAPTER: 6:[SVDM_A] HTTPAdapterCommunicator::requestReply:618 :Getting Resp via HTTP/S Client!!
2020/07/22-07:51:00.464914 7f2e3723b701 SOCK_STREAM: 3:[core] T_DATA_REQ: (fd=0xffffffff, NS=0) not connected
2020/07/22-07:51:20.462439 7f2e39821701 ADAPTER: 3:[SVDM_A] HTTPAdapterCommunicator::requestReply:623 :NO RESPONSE!! HTTPSTATUS = 1006 Connection Disconnected

2020/07/22-07:51:20.462460 7f2e39821701 ADAPTER: 3:[GvSh_3] ScaleIOStorageAdapter::sendRequestEx:559 Unable to send request, status 3
2020/07/22-07:51:20.462466 7f2e39821701 ADAPTER: 6:[GvSh_3] ScaleIOStorageAdapter::sendRequestEx:576 Platform communications error/disconnect
2020/07/22-07:51:21.462960 7f2e39821701 ADAPTER: 6:[GvSh_3] ScaleIOStorageAdapter::startSession:351 Successfully retrieved "sacred" credentials from NodeCredentialManager
2020/07/22-07:51:21.462985 7f2e39821701 ADAPTER: 6:[GvSh_3] ScaleIOStorageAdapter::startSession:369 Use local adminInfo.sdnasAdminAddr for SIO server address
2020/07/22-07:51:21.462991 7f2e39821701 ADAPTER: 6:[GvSh_3] HTTPAdapterCommunicator::createConnection:188 Enter
2020/07/22-07:51:21.463232 7f2e39821701 ADAPTER: 6:[GvSh_3] HTTPAdapterCommunicator::createConnection:319 Exit
2020/07/22-07:51:21.463262 7f2e39821701 ADAPTER: 6:[GvSh_3] ScaleIOStorageAdapter::createHeaderJson:317 JSON headers <{"URI":"\/api\/login","HTTP_TREQ":1,"REQHEADER":{"Content-Type":"application\/json"}}>
2020/07/22-07:51:21.463276 7f2e39821701 ADAPTER: 6:[GvSh_3] HTTPAdapterCommunicator::requestReply:495 Enter

2020/07/22-07:51:21.463297 7f2e39821701 ADAPTER: 6:[SVDM_A] HTTPAdapterCommunicator::requestReply:593 :Sending via HTTP/S Client!!
2020/07/22-07:51:21.463311 7f2e39821701 ADAPTER: 6:[SVDM_A] HTTPAdapterCommunicatorReq::writeHeader:77
2020/07/22-07:51:21.463322 7f2e39821701 ADAPTER: 6:[SVDM_A] HTTPAdapterCommunicatorReq::writeHeader:112 KEY = Content-Type VALUE = application/json
2020/07/22-07:51:27.267311 7f2e45ba5700 UFS: 6:[core] reStartSync: fsid:162 dirtyCount:1 (max:22) interval:30007734 maxPending:1 nthread:16 ioLatency:1199
2020/07/22-07:51:41.468135 7f2e39821701 ADAPTER: 3:[SVDM_A] HTTPAdapterCommunicator::requestReply:603 :Unable to Send status: 1006 Connection Disconnected!!

2020/07/22-07:51:41.468156 7f2e39821701 ADAPTER: 3:[GvSh_3] ScaleIOStorageAdapter::startSession:389 Unable to send login request, status 3
2020/07/22-07:51:41.468165 7f2e39821701 ADAPTER: 6:[GvSh_3] ScaleIOStorageAdapter::startSession:425 Forcing communicator pool to be like-new due to setup error "SDNAS_RC_PLATFORM_COMMUNICATION_ERROR"
2020/07/22-07:51:41.468190 7f2e39821701 ADAPTER: 3:[GvSh_3] ScaleIOStorageAdapter::fetchVolumeProperties:730 Unexpected ScaleIO error -1

当我们查看 VxFlex 网关 api 和 scaleio-trace 日志时，我们当时看不到任何登录或实例尝试（由于 sdnas 处于 UTC 状态，因此滞后了 4 小时）。在此期间，日志仅显示 API 中的结束 IP“132”，从现在开始，网关从未收到任何以 IP “136”结尾的 API 调用：

2020-07-22 03:48:07,919 [https-jsse-nio-443-exec-10] INFO  audit - 172.25.154.132:/api/types/Volume/instances/action/queryIdByKey {name=5f17efb2b82c4295d40005056bd5aa7}
2020-07-22 03:48:07,948 [https-jsse-nio-443-exec-5] INFO  audit - 172.25.154.132:/api/types/Volume/instances {volumeType=ThinProvisioned, name=5f17efb2b82c4295d40005056bd5aa7, storagePoolId=469c695200000000, volumeSizeInKb=6291456}
2020-07-22 03:48:07,965 [https-jsse-nio-443-exec-22] INFO  audit - 172.25.154.132:/api/instances/Volume::29eba43300000080/action/addMappedSdc {guid=82AFDBA9-E76B-41C0-9F34-BF4392236F57, allowMultipleMappings=TRUE}
2020-07-22 03:48:07,973 [https-jsse-nio-443-exec-17] INFO  audit - 172.25.154.132:/api/instances/Volume::29eba43300000080/action/addMappedSdc {guid=7EFE3A2F-59B3-43D3-B7A3-07CF35436F9F, allowMultipleMappings=TRUE}
2020-07-22 03:48:17,534 [https-jsse-nio-443-exec-13] INFO  audit - 172.25.15.64:/api/instances/Volume::29eb929f000005d8
2020-07-22 03:48:17,634 [https-jsse-nio-443-exec-10] INFO  audit - 172.25.15.64:/api/instances/Volume::29eb92a2000005f4
2020-07-22 03:48:17,784 [https-jsse-nio-443-exec-16] INFO  audit - 172.25.15.64:/api/instances/Volume::29eb92a1000005ed
2020-07-22 03:48:26,924 [https-jsse-nio-443-exec-23] INFO  audit - 172.25.154.132:/api/types/Volume/instances/action/queryIdByKey {name=5f17efc54a36ba4d4bf005056bd8714}
2020-07-22 03:48:26,949 [https-jsse-nio-443-exec-13] INFO  audit - 172.25.154.132:/api/types/Volume/instances {volumeType=ThinProvisioned, name=5f17efc54a36ba4d4bf005056bd8714, storagePoolId=469c695200000000, volumeSizeInKb=52428800}
2020-07-22 03:48:26,971 [https-jsse-nio-443-exec-10] INFO  audit - 172.25.154.132:/api/instances/Volume::29eba43400000669/action/addMappedSdc {guid=82AFDBA9-E76B-41C0-9F34-BF4392236F57, allowMultipleMappings=TRUE}
2020-07-22 03:48:26,980 [https-jsse-nio-443-exec-16] INFO  audit - 172.25.154.132:/api/instances/Volume::29eba43400000669/action/addMappedSdc {guid=BE2535F2-AF91-4B70-8BAA-E52F4CF9C9B9, allowMultipleMappings=TRUE}
2020-07-22 03:48:27,200 [https-jsse-nio-443-exec-23] INFO  audit - 172.25.19.22:/api/types/Volume/instances/action/queryIdByKey {name=5f17efc6d0f10999e45005056bd6acc}
2020-07-22 03:48:27,273 [https-jsse-nio-443-exec-13] INFO  audit - 172.25.19.22:/api/instances/System::7e558c9e2d40490f/action/snapshotVolumes {snapshotDefs=[{volumeId=29eae306000001d2, snapshotName=5f17efc6d0f10999e45005056bd6acc}]}
2020-07-22 03:48:27,568 [https-jsse-nio-443-exec-10] INFO  audit - 172.25.15.64:/api/instances/Volume::29eb929f000005d8
2020-07-22 03:48:27,667 [https-jsse-nio-443-exec-16] INFO  audit - 172.25.15.64:/api/instances/Volume::29eb92a2000005f4
2020-07-22 03:48:27,818 [https-jsse-nio-443-exec-20] INFO  audit - 172.25.15.64:/api/instances/Volume::29eb92a1000005ed
2020-07-22 03:48:35,154 [https-jsse-nio-443-exec-23] INFO  audit - 172.25.154.132:/api/instances/Volume::29eba43400000669
2020-07-22 03:48:36,219 [https-jsse-nio-443-exec-19] INFO  audit - 172.25.154.132:/api/types/Volume/instances/action/queryIdByKey {name=5f17efcfe6667f3a5d9005056bd8714}
2020-07-22 03:48:36,243 [https-jsse-nio-443-exec-10] INFO  audit - 172.25.154.132:/api/types/Volume/instances {volumeType=ThinProvisioned, name=5f17efcfe6667f3a5d9005056bd8714, storagePoolId=469c695200000000, volumeSizeInKb=52428800}
2020-07-22 03:48:36,261 [https-jsse-nio-443-exec-16] INFO  audit - 172.25.154.132:/api/instances/Volume::29eba43600000674/action/addMappedSdc {guid=BE2535F2-AF91-4B70-8BAA-E52F4CF9C9B9, allowMultipleMappings=TRUE}
2020-07-22 03:48:36,268 [https-jsse-nio-443-exec-20] INFO  audit - 172.25.154.132:/api/instances/Volume::29eba43600000674/action/addMappedSdc {guid=F80A1A85-376A-4318-9018-5742364EF531, allowMultipleMappings=TRUE}

影响 

SDNAS 实例的 REST API 登录和查询功能失败。

异常关闭的连接在本地发生 HTTP 错误“1006”。这意味着连接从未离开尝试连接的作系统。这就解释了为什么 VxFlex GW 和 MDM 从来都看不到登录或查询尝试。请参阅下面 RFC 中的 HTTPS 套接字通信“rfc 6455”

1006

1006 is a reserved value and MUST NOT be set as a status code in a Close control frame by an endpoint. It is designated for use in applications expecting a status code to indicate 
that the connection was closed abnormally, e.g., without sending or receiving a Close control frame.

HTTP 错误发生在 SDC 本身本地的调用函数中，并且从不联系 GW。

解决方法

通过绕过网络路径内的第 3 层路由器，问题已解决。这允许 SDNAS 实例 SDC 的直接第 2 层通信直接与 VxFlex 网关通信。

受影响的版本

任何 VxFlexOS/PowerFlex 版本

已修复问题的版本

不适用 — 不是 VxFlex 问题