Unsolved
This post is more than 5 years old
2 Posts
0
2567
Not able to mount volume using multiple SDC networks
Hello all. I have 4 ESX hosts running SDC, 4 VMs running SDS (one per ESX host), and 3 VMs outside of the ESX cluster running all the MDM components. All are connected via two separate VLANs on separate interfaces. I have completed all the steps to get this setup, however, I am not able to mount the volume using BOTH networks (one works).
ESX1 10.209.46.192 10.209.47.192 /24
ESX2 10.209.46.193 10.209.47.193 /24
ESX3 10.209.46.194 10.209.47.194 /24
ESX4 10.209.46.195 10.209.47.195 /24
SDS1 10.209.46.1 10.209.47.1 /24
SDS2 10.209.46.2 10.209.47.2 /24
SDS3 10.209.46.3 10.209.47.3 /24
SDS4 10.209.46.4 10.209.47.4 /24
MDM1 10.209.46.150 10.209.47.150 /24 also has public IP for MDM cluster use
MDM1 10.209.46.151 10.209.47.151 /24 also has public IP for MDM cluster use
MDM1 10.209.46.152 10.209.47.152 /24 also has public IP for MDM cluster use
I setup the SDC using "scli --add_sdc --sdc_ip IP" and enabled --set_restricted_sdc_mode. I then mapped to the volume using the "allow_multi_map" option :
scli --map_volume_to_sdc --volume_name VOLUME01 --sdc_ip 10.209.46.192 --allow_multi_map
scli --map_volume_to_sdc --volume_name VOLUME01 --sdc_ip 10.209.47.192 --allow_multi_map
scli --map_volume_to_sdc --volume_name VOLUME01 --sdc_ip 10.209.46.193 --allow_multi_map
scli --map_volume_to_sdc --volume_name VOLUME01 --sdc_ip 10.209.47.193 --allow_multi_map
for all 8 IPs
# scli --query_all_sdc
MDM restricted SDC mode: Enabled
Query all SDC returned 8 SDC nodes.
SDC ID: 2620886900000000 Name: N/A IP: 10.209.47.194 State: Connected GUID: 5693D1EF-0F57-6D00-25F4-44D3CADA2622 Approved: yes
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 1 IOPS 1.0 KB (1024 Bytes) per-second
SDC ID: 2620886a00000001 Name: N/A IP: 10.209.46.195 State: Connected GUID: 5693D856-74A0-E69F-C16F-44D3CADA2682 Approved: yes
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 1 IOPS 1.0 KB (1024 Bytes) per-second
SDC ID: 2620886b00000002 Name: N/A IP: 10.209.46.193 State: Connected GUID: 5693D31A-A660-C180-5FA6-44D3CADA4E34 Approved: yes
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 1 IOPS 1.0 KB (1024 Bytes) per-second
SDC ID: 2620886c00000003 Name: N/A IP: 10.209.46.192 State: Connected GUID: 56938649-B16B-2DBC-2BD7-44D3CADA4D60 Approved: yes
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 1 IOPS 1.0 KB (1024 Bytes) per-second
SDC ID: 2620886d00000004 Name: N/A IP: 10.209.47.192 State: Disconnected GUID: 00000000-0000-0000-0000-000000000000 Approved: yes
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 0 IOPS 0 Bytes per-second
SDC ID: 2620886e00000005 Name: N/A IP: 10.209.46.194 State: Disconnected GUID: 00000000-0000-0000-0000-000000000000 Approved: yes
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 0 IOPS 0 Bytes per-second
SDC ID: 2620886f00000006 Name: N/A IP: 10.209.47.193 State: Disconnected GUID: 00000000-0000-0000-0000-000000000000 Approved: yes
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 0 IOPS 0 Bytes per-second
SDC ID: 2620887000000007 Name: N/A IP: 10.209.47.195 State: Disconnected GUID: 00000000-0000-0000-0000-000000000000 Approved: yes
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 0 IOPS 0 Bytes per-second
Notice the bottom four are disconnected and have no GUID. Also notice it's not always ".47" that is not working! I went ahead and unmapped everything and started over:
# scli --query_all_sdc
MDM restricted SDC mode: Disabled
Query all SDC returned 4 SDC nodes.
SDC ID: 2620886900000000 Name: N/A IP: 10.209.47.194 State: Connected GUID: 5693D1EF-0F57-6D00-25F4-44D3CADA2622
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 0 IOPS 0 Bytes per-second
SDC ID: 2620886a00000001 Name: N/A IP: 10.209.46.195 State: Connected GUID: 5693D856-74A0-E69F-C16F-44D3CADA2682
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 0 IOPS 0 Bytes per-second
SDC ID: 2620886b00000002 Name: N/A IP: 10.209.46.193 State: Connected GUID: 5693D31A-A660-C180-5FA6-44D3CADA4E34
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 0 IOPS 0 Bytes per-second
SDC ID: 2620887100000003 Name: N/A IP: 10.209.46.192 State: Connected GUID: 56938649-B16B-2DBC-2BD7-44D3CADA4D60
Read bandwidth: 0 IOPS 0 Bytes per-second
Write bandwidth: 0 IOPS 0 Bytes per-second
(you cannot remove an SDC once it is connected, only when it goes away)
# scli --query_sdc --sdc_ip 10.209.46.194
Error: MDM failed command. Status: Could not find the SDC
# scli --query_sdc --sdc_ip 10.209.47.195
Error: MDM failed command. Status: Could not find the SDC
# scli --query_sdc --sdc_ip 10.209.47.193
Error: MDM failed command. Status: Could not find the SDC
# scli --query_sdc --sdc_ip 10.209.47.192
Error: MDM failed command. Status: Could not find the SDC
(pinged all these IPs from MDM and SDS with success)
# scli --query_all_sds
Query-all-SDS returned 4 SDS nodes.
Protection Domain ca25d2cf00000000 Name: pd001
SDS ID: 0f16a72e00000005 Name: SDS4 State: Connected, Joined IP: 10.209.46.4,10.209.47.4 Port: 7072
SDS ID: 0f16a72d00000004 Name: SDS3 State: Connected, Joined IP: 10.209.46.3,10.209.47.3 Port: 7072
SDS ID: 0f16a72b00000002 Name: SDS2 State: Connected, Joined IP: 10.209.46.2,10.209.47.2 Port: 7072
SDS ID: 0f16a72900000000 Name: SDS1 State: Connected, Joined IP: 10.209.46.1,10.209.47.1 Port: 7072
(notice all IPs are joined)
From an ESX host :
# esxcli system module parameters list -m scini (cleaned up output)
Name Type Value
IoctlIniGuidStr string 56938649-b16b-2dbc-2bd7-44d3cada4d60
IoctlMdmIPStr string 10.209.46.150,10.209.46.151,10.209.47.150,10.209.47.151
I checked the ESX host for local firewall rules - the SDC VIB created a rule to allow 7072, 9011, 6611 outbound. I would assume a firewall issue would affect both IPs though. Any ideas?
-Alex
daverush
51 Posts
0
February 13th, 2016 14:00
Alex,
This is a common misunderstanding regarding SDC configuration, made more unclear due to restricted SDC mode requiring the manual addition of an SDC via SCLI.
Without that mode enabled, SDCs just need a GUID and the MDM IPs to register themselves.
Restricted-mode SDCs do not need all IPs added to the MDM's configuration, only one. In fact, you could use the management network IP of the SDC. The SDC then pulls the MDM and SDS clusters' layout and connects to the SDS IPs that it can reach.
In your case, if you can vmkping -I vmkX 10.209.46.1 (and 2,3,4, etc) as well as vmkping -I vmkY 10.209.47.1 (and 2,3,4, etc), then you have connectivity to the SDSes on both subnets.
The SDC will use all available subnets to reach the SDSes. If you have 3 subnets for data on the SDSes and only 2 configured on the SDCs, then you will connect via the two available subnets. If you add the 3rd subnet to the SDCs, they will then also connect to the SDSes on the third subnet. (The exception to this being SDS IP role configuration, which dedicates an IP/subnet to either SDC-only traffic, or SDS-only traffic.)
Best practice is to maintain a 1:1 ratio of NICs/vSwitches to subnets.
daverush
51 Posts
0
February 13th, 2016 14:00
Also,
https://support.emc.com/kb/466097 details ESX SDC config and troubleshooting, minus the restricted SDC mode.
Alex10
2 Posts
1
February 19th, 2016 12:00
This makes a lot more sense. I have no real way to know if it is using those networks though. I'll get this all setup and see how it performs. I really appreciate the detailed answer, Rush!