Data Domain: Storage enclosure numbering is not sequential
摘要: This article describes scenarios where the shelf numbers, assigned to external storage enclosures (ES-30 & DS-60) on Data Domain appliances may not be sequential. Non-sequential numbering or gaps in the shelf numbering sequence may occur after a completed storage migration operation, chassis replacement for one of the storage enclosures, or physically moving a storage enclosure to a new path. Data Domain recommends that the steps detailed in this article be applied in order to re-number all of the enclosures sequentially. ...
本文适用于
本文不适用于
本文并非针对某种特定的产品。
本文并非包含所有产品版本。
症状
Future enhancements to Data Domain Operating System code will be made to mitigate the excessive rescan and discovery operations when shelf numbering is non-sequential. Until such changes are made , it is recommended that the necessary steps are applied to ensure that the shelf number is sequential after a physical change or serial number change which causes a gap in sequential shelf numbering.
Beginning in DDOS 5.7.1, storage shelf numbering is tied to the storage shelf serial number. We call this a shelf "persistent-id".
A storage enclosure can be listed as "Offline" under multiple different scenarios where a physical change has been made to the SAS topology.
In this example, a "storage migration" was recently completed, and the source storage enclosures were physically removed. Due to each storage enclosure having an assigned "persistent-id", there are now several "Offline" shelves (2,3,4,8) that are no longer attached, These "offline shelves are no longer required for the Filesystem to be intact and operational. The fact that there are phantom shelves or non-sequential numbering of attached storage enclosures, does not affect the overall operation of the Data Domain Filesystem, however it can cause excessive rescan and discovery operations during boot-up as well as SNMP cores when a query is made to a phantom enclosure. This results in longer than expected boot times and the Volume not being available immediately after the boot phase is complete.
Beginning in DDOS 5.7.1, storage shelf numbering is tied to the storage shelf serial number. We call this a shelf "persistent-id".
A storage enclosure can be listed as "Offline" under multiple different scenarios where a physical change has been made to the SAS topology.
In this example, a "storage migration" was recently completed, and the source storage enclosures were physically removed. Due to each storage enclosure having an assigned "persistent-id", there are now several "Offline" shelves (2,3,4,8) that are no longer attached, These "offline shelves are no longer required for the Filesystem to be intact and operational. The fact that there are phantom shelves or non-sequential numbering of attached storage enclosures, does not affect the overall operation of the Data Domain Filesystem, however it can cause excessive rescan and discovery operations during boot-up as well as SNMP cores when a query is made to a phantom enclosure. This results in longer than expected boot times and the Volume not being available immediately after the boot phase is complete.
Use the #enclosure show summary command to list all enclosures which have ever been issued a persisten-id along with the current state of the enclosure.
#Enclosure Show Summary ---------------------- Enclosure Model No. Serial No. State OEM Name OEM Value Capacity --------- --------- -------------- ------- -------- --------- -------- 1 DD9800 NVT10164000024 Online 4 Slots 2 DS60 APM00164936021 Offline 60 Slots 3 DS60 APM00164802045 Offline 60 Slots 4 DS60 APM00164802044 Offline 60 Slots 5 DS60 APM00164902721 Online 60 Slots 6 DS60 APM00164936019 Online 60 Slots 7 DS60 APM00164936020 Online 60 Slots 8 ES30 APM00164964966 Offline 15 Slots 9 DS60 APM00164902719 Online 60 Slots 10 DS60 APM00164902720 Online 60 Slots 11 DS60 APM00164902722 Online 60 Slots 12 ES30 APM00164964967 Online 15 Slots --------- --------- -------------- ------- -------- --------- --------
Use the #enclosure show topology CLI command to list which storage enclosures are currently attached. Looking at the SAS Topology confirms "Offline" shelves (2,3,4,8) are not really attached, and that the numerical number assigned to the attached shelves is non-sequential.
#Enclosure Show Topology ----------------------- Port enc.ctrl.port enc.ctrl.port ---- - ------------- - ------------- 2a > 9.B.H: 9.B.E > 10.B.H:10.B.E 2b > 5.B.H: 5.B.E > 11.B.H:11.B.E 2c > 7.B.H: 7.B.E 2d > 12.B.H:12.B.E 3a > 10.A.H:10.A.E > 9.A.H: 9.A.E 3b > 6.B.H: 6.B.E 3c > 7.A.H: 7.A.E 3d > 12.A.H:12.A.E 6a > 11.A.H:11.A.E > 5.A.H: 5.A.E 6b > 6.A.H: 6.A. 6d ---- - ------------- - ------------- Encl WWN Serial # ---- ---------------- -------------- 5 N/A APM00164902721 6 N/A APM00164936019 7 N/A APM00164936020 9 N/A APM00164902719 10 N/A APM00164902720 11 N/A APM00164902722 12 N/A APM00164964967 ---- ---------------- --------------
Note: Various outputs in the GUI or Autosupport may show the "Offline" shelf as absent 8.1 (absent) 8.2 (absent) 8.3 (absent) 8.4 (absent) 8.5 (absent) 8.6 (absent) 8.7 (absent) 8.8 (absent) 8.9 (absent) 8.10 (absent) 8.11 (absent) 8.12 (absent) 8.13 (absent) 8.14 (absent) 8.15 (absent) SNMP queries to phantom enclosures results in errors and subsequent cores. Corresponding messaging will be posted to the messages.engineering log
# snmpwalk -v2c -c Public localhost -t 5 -r 5 dataDomainMib Timeout: No Response from localhost
Apr 20 01:01:54 xx snmpd: WARNING: ddr/sm/snmp/enclosurePack_access.c:161 **** Enclosure id 3 is offline. Apr 20 01:01:54 xx ddr_procmon: NOTICE: MSG-PMON-00008: Failure on /usr/sbin/snmpd. Apr 20 01:01:54 xx ddr_procmon: WARNING: MSG-PMON-00011: Restarting /usr/sbin/snmpd. Apr 20 01:01:56 xx ddfs[11215]: NOTICE: ddr_key_del_temp: deleting key-v2-100 Apr 20 01:02:04 xx snmpd: WARNING: ddr/sm/snmp/enclosurePack_access.c:161 **** Enclosure id 3 is offline. Apr 20 01:02:04 xx ddr_procmon: NOTICE: MSG-PMON-00008: Failure on /usr/sbin/snmpd. Apr 20 01:02:04 xx ddr_procmon: WARNING: MSG-PMON-00011: Restarting /usr/sbin/snmpd. Apr 20 01:02:14 xx snmpd: WARNING: ddr/sm/snmp/enclosurePack_access.c:161 **** Enclosure id 3 is offline. Apr 20 01:02:14 xx ddr_procmon: NOTICE: MSG-PMON-00008: Failure on /usr/sbin/snmpd. Apr 20 14:02:42 xx snmpd: WARNING: dd_netsnmp_set_request_error: snmp error 129 occured
原因
Scenarios which can trigger this condition are as follows:
- Physically moving storage enclosures from one Data Domain appliance to another Data Domain appliance
- Chassis replacement on any in-use storage enclosure (new chassis will introduce a new serial number)
- Storage Migration - after a migration is completed, the source shelves are no longer needed by the FS, and will be physically removed.
- #storage remove CLI command was issues.
解决方案
To force the shelf "persistent-id" to re-number in sequential order, employ the following commands in SE mode.
SE# enclosure show persistent-id
The output will detail information from enclosures that are attached , and shelves that are no longer attached, along with their corresponding persistent-id. you will see that there are serial numbers which are no longer attached, or have been removed from the SAS topology. The ones I have highlighted are no longer attached, and #enclosure show summary will list them as "Offline".
Serial No. Model No. Persistent ID -------------- --------- ------------- APM00164936021 DS60 2 <<< Offline APM00164802045 DS60 3 <<< Offline APM00164802044 DS60 4 <<< Offline APM00164902721 DS60 5 APM00164936019 DS60 6 APM00164936020 DS60 7 APM00164964966 ES30 8 <<< Offline APM00164902719 DS60 9 APM00164902720 DS60 10 APM00164902722 DS60 11 APM00164964967 ES30 12 -------------- --------- ------------- 12 enclosure(s) persisted.
Use the #enclosure release persistent-id CLI to release the existing persistent-id's. You can release only the ones that are offline, or you can choose to release <all> of them. On the next boot-up, each attached shelf will be issued a new persistent-id and there will be no gaps in shelf numbering.
NOTE: On HA Systems, it is recommended to run "enclosure release persistent-id all" on both nodes, one at a time beginning with the active node before issuing system reboot. Rebooting the active node will trigger a failover to the standby node. Once, the HA reboot cycle is complete and system is highly available again, reboot the current active node to complete the process.
SE# enclosure release persistent-id
{<serialno>|<persistent-id>|all}
Example: #enclosure release persistent-id APMxxxxxxxxxxxxxx #enclosure release persistent-id 12 #enclosure release persistent-id all
Next, reboot the system to allow new persistent ID's to be issued to each storage enclosure.
#system reboot
After the system has rebooted ,verify that the shelf numbering is sequential and there are no storage enclosures listed as "Offline".
# enclosure show summary Enclosure Model No. Serial No. State OEM Name OEM Value Capacity --------- --------- -------------- ------ -------- --------- -------- 1 DD9800 NVT10162700023 Online 4 Slots 2 DS60 APM00164802045 Online 0 60 Slots 3 DS60 APM00164802043 Online 0 60 Slots 4 DS60 APM00164802044 Online 0 60 Slots 5 DS60 APM00164851510 Online 0 60 Slots 6 DS60 APM00164936021 Online 0 60 Slots 7 DS60 APM00164802046 Online 0 60 Slots 8 ES30 APM00164964966 Online 15 Slots --------- --------- -------------- ------ -------- --------- -------- 8 enclosures present.
SE# enclosure show persistent-id Serial No. Model No. Persistent ID -------------- --------- ------------- APM00164802045 DS60 2 APM00164802043 DS60 3 APM00164802044 DS60 4 APM00164851510 DS60 5 APM00164936021 DS60 6 APM00164802046 DS60 7 APM00164964966 ES30 8 -------------- --------- ------------- 7 enclosure(s) persisted.
受影响的产品
Data Domain产品
Data Domain, DD OS, DD OS Licensed Features文章属性
文章编号: 000054913
文章类型: Solution
上次修改时间: 19 7月 2023
版本: 3
从其他戴尔用户那里查找问题的答案
支持服务
检查您的设备是否在支持服务涵盖的范围内。