Adding an extra network to existing MDM cluster

Question

Hello, playing around with different network configurations I found out that it is hard to add an extra network to already established MDM cluster. Maybe you know a way how to achieve it. In our lab we originally had one management network and one data network for Scaleio cluster on VMware (installed via vCenter plugin). Later we decided to investigate the steps needed to add a second data network (physically independent) to already running system to further increase its availability. Adding extra IP adresses to SDSs is easy and so it is for the SDCs to learn about the new available data paths. However, to have a fully redundant network setup (similar to SAN A & B in Fibre Channel world) it is also necessary to add the second network to the MDM cluster. Because there is no command to directly achive it, I tried switching the cluster to single mode, deleting secondary MDM, deleting TB, and later adding a new secondary MDM already with two IP adresses. Unfortunatelly this doesn't work and the command fails with information that the primary and secondary MDMs need equal number of IP addresses. # scli --add_secondary_mdm --secondary_mdm_ip 192.168.89.120,192.168.99.120 Error: MDM failed command.  Status: Illegal number of IP addresses. The Primary MDM, the Secondary MDM, and the Tie-Breaker must each have the same number of IP addresses.

alexkh · Accepted Answer

Currently, ScaleIO doesn't have an option to add data IP to already configured system. We are working on adding this functionality to the next service pack, which should be released at the end of July

pavel.heidrich · Accepted Answer

Thank you both, Sanjeev and Alex. This answers my question and confirms our findings. It is a lab environment now, so no problem to rebuild it from scratch, we were just investigating on the possibilities to reconfigure production systems later.

SanjeevMalhotra · Answer

scli --add_primary_mdm --primary_mdm_ip IP1,IP2 --mdm_management_ip IP1,IP2  (Correction) (here you can give multiple IP addresses for MDM)

SanjeevMalhotra · Answer

Did you configure the Primary MDM with 2 IP addresses?

I believe there are 2 ways to achieve this. Using the CSV File (Page 37 of the User Guide)

IP Roles.png

There is an example of CSV file thereafter wherein you can provide multiple IP addresses for MDM IPs seperated by commas.

In your problem description i understand that you have not yet assigned the multiple IP address to the Primary MDM and are assigning multiple IP address to the secondary IP MDM. That is the reason it is giving an error. So if this is a lab environment, you can break the ScaleIO and reconfigure.

scli --add_primary_mdm --primary_mdm_ip IP1 --mdm_management_ip IP1,IP2 (here you can give multiple IP addresses for MDM)

Login:-

scli --login --mdm_ip IP1 --username admin --password admin

Reset the password:-

scli --set_password --mdm_ip IP1 --old_password admin --new_password

scli --query_cluster (This command will list only 1 IP address for the Primary MDM. Use that IP address in the next command for --mdm_ip parameter)

scli --add_secondary_mdm --mdm_ip IP1 --secondary_mdm_ip IP3,IP4 (IP3,IP4 can be the IP address of the secondary MDM)

Also note when adding new SDS there are 3 IP roles

SDC_ONLY – IP address will only be used by the SDCs for application IO

SDS_ONLY – IP address will be used by MDM and SDS

ALL – IP address will be used for all traffic.

The system constantly validates that every SDS has IPs with all roles. Either at least one IP with the ALL role, Or at least one IP with SDC_ONLY and one IP with SDS_ONLY

If you use All IP role, it will also be used for MDM communication.

littleboy1 · Answer

Hi,Sanjeev I wana to know how to break it.I can switch to single mode,then remove secondary MDM and TB.Then how remove the primary MDM?

HGTNG · Answer

I have faced a same kind of situation when adding another IP of MDM to an existing VxFlex cluster. I'm trying this on VxFlex OS v.2.6.1

When I try "scli --add_primary_mdm --primary_mdm_ip IP1 --mdm_management_ip IP1,IP2 " commands, it says

scli: unrecognized option '--add_primary_mdm'

russocar_TIM · Answer

Hi all, I found this thread.

Now I am involved in the same problem with PowerFlex 3.5.1.

I add ip to sds, simply as already described in the original post.

Now I try to add IP also for MDM in this way:

scli --modify_cluster_virtual_ips --clear_all
scli --modify_virtual_ip_interfaces --new_mdm_virtual_ip_interface bond1.1056,bond2.1070 --target_mdm_ip 163.162.230.234
scli --modify_virtual_ip_interfaces --new_mdm_virtual_ip_interface bond1.1056,bond2.1070 --target_mdm_ip 163.162.230.235
scli --modify_cluster_virtual_ips --cluster_virtual_ip 192.168.56.250,192.168.70.250

where the address/vlan with .70 is the new networking domain.

When i query the cluster I can see the new VIP but each IP for each MDM is only the previous one, I expected to find both IP:

[root@scaleio02-tovb ~]# scli --query_cluster
Cluster:
Name: ScaleIO-tovb, Mode: 3_node, State: Normal, Active: 3/3, Replicas: 2/2
Virtual IP Addresses: 192.168.56.250, 192.168.70.250
Master MDM:
Name: MDM2, ID: 0x1b292daa1fa99011
IP Addresses: 192.168.56.4, Management IP Addresses: 163.162.230.235, Port: 9011, Virtual IP interfaces: bond1.1056, bond2.1070
Status: Normal, Version: 3.5.1200
Slave MDMs:
Name: MDM1, ID: 0x3cd645b45b7db300
IP Addresses: 192.168.56.2, Management IP Addresses: 163.162.230.234, Port: 9011, Virtual IP interfaces: bond1.1056, bond2.1070
Status: Normal, Version: 3.5.1200
Tie-Breakers:
Name: TB1, ID: 0x452a4fa35085c802
IP Addresses: 192.168.56.6, Port: 9011
Status: Normal, Version: 3.5.1200
[root@scaleio02-tovb ~]#

Someone can help me?

PowerFlex

Was this post helpful?