pavel.heidrich

updated

11 years ago

0

7243

June 23rd, 2015 10:00

Adding an extra network to existing MDM cluster

Hello, playing around with different network configurations I found out that it is hard to add an extra network to already established MDM cluster. Maybe you know a way how to achieve it.

In our lab we originally had one management network and one data network for Scaleio cluster on VMware (installed via vCenter plugin). Later we decided to investigate the steps needed to add a second data network (physically independent) to already running system to further increase its availability. Adding extra IP adresses to SDSs is easy and so it is for the SDCs to learn about the new available data paths. However, to have a fully redundant network setup (similar to SAN A & B in Fibre Channel world) it is also necessary to add the second network to the MDM cluster.

Because there is no command to directly achive it, I tried switching the cluster to single mode, deleting secondary MDM, deleting TB, and later adding a new secondary MDM already with two IP adresses. Unfortunatelly this doesn't work and the command fails with information that the primary and secondary MDMs need equal number of IP addresses.

# scli --add_secondary_mdm --secondary_mdm_ip 192.168.89.120,192.168.99.120

Error: MDM failed command.  Status: Illegal number of IP addresses. The Primary MDM, the Secondary MDM, and the Tie-Breaker must each have the same number of IP addresses.

  • alexkh

    60 Posts

    6421

    0

    Posted June 24th, 2015 09:00

    Currently, ScaleIO doesn't have an option to add data IP to already configured system.

    We are working on adding this functionality to the next service pack, which should be released at the end of July

  • 6421

    0

    Posted June 24th, 2015 22:00

    Thank you both, Sanjeev and Alex. This answers my question and confirms our findings. It is a lab environment now, so no problem to rebuild it from scratch, we were just investigating on the possibilities to reconfigure production systems later.

  • 6421

    0

    Posted June 24th, 2015 09:00

    scli --add_primary_mdm --primary_mdm_ip IP1,IP2 --mdm_management_ip IP1,IP2  (Correction) (here you can give multiple IP addresses for MDM)

  • 6421

    0

    Posted June 24th, 2015 09:00

    Did you configure the Primary MDM with 2 IP addresses?

    I believe there are 2 ways to achieve this. Using the CSV File (Page 37 of the User Guide)

    IP Roles.png

    There is an example of CSV file thereafter wherein you can provide multiple IP addresses for MDM IPs seperated by commas.

    In your problem description i understand that you have not yet assigned the multiple IP address to the Primary MDM and are assigning multiple IP address to the secondary IP MDM. That is the reason it is giving an error. So if this is a lab environment, you can break the ScaleIO and reconfigure.

    scli --add_primary_mdm --primary_mdm_ip IP1 --mdm_management_ip IP1,IP2    (here you can give multiple IP addresses for MDM)

    Login:-

    scli --login --mdm_ip IP1 --username admin --password admin

    Reset the password:-

    scli --set_password --mdm_ip IP1 --old_password admin --new_password

    scli --query_cluster   (This command will list only 1 IP address for the Primary MDM. Use that IP address in the next command for --mdm_ip parameter)

    scli --add_secondary_mdm --mdm_ip IP1 --secondary_mdm_ip IP3,IP4    (IP3,IP4 can be the IP address of the secondary MDM)

    Also note when adding new SDS there are 3 IP roles

    SDC_ONLY – IP address will only be used by the SDCs for application IO

    SDS_ONLY – IP address will be used by MDM and SDS

    ALL – IP address will be used for all traffic. 

    The system constantly validates that every SDS has IPs with all roles.  Either at least one IP with the ALL role, Or at least one IP with SDC_ONLY and one IP with SDS_ONLY

    If you use All IP role, it will also be used for MDM communication.

  • littleboy1

    1 Rookie

    53 Posts

    6421

    0

    Posted August 5th, 2015 02:00

    Hi,Sanjeev

    I wana to know how to break it.I can switch to single mode,then remove secondary MDM and TB.Then how remove the primary MDM?

  • HGTNG

    6 Posts

    6420

    0

    Posted December 21st, 2018 00:00

    I have faced a same kind of situation when adding another IP of MDM to an existing VxFlex cluster. I'm trying this on VxFlex OS v.2.6.1

    When I try "scli --add_primary_mdm --primary_mdm_ip IP1 --mdm_management_ip IP1,IP2 " commands, it says

    scli: unrecognized option '--add_primary_mdm'

  • russocar_TIM

    1 Message

    2399

    0

    Posted May 7th, 2021 08:00

    Hi all, I found this thread.

    Now I am involved in the same problem with PowerFlex 3.5.1.

    I add ip to sds, simply as already described in the original post.

    Now I try to add IP also for MDM in this way:

    scli --modify_cluster_virtual_ips --clear_all
    scli --modify_virtual_ip_interfaces --new_mdm_virtual_ip_interface bond1.1056,bond2.1070 --target_mdm_ip 163.162.230.234
    scli --modify_virtual_ip_interfaces --new_mdm_virtual_ip_interface bond1.1056,bond2.1070 --target_mdm_ip 163.162.230.235
    scli --modify_cluster_virtual_ips --cluster_virtual_ip 192.168.56.250,192.168.70.250

    where the address/vlan with .70 is the new networking domain.

    When i query the cluster I can see the new VIP but each IP for each MDM is only the previous one, I expected to find both IP:

    [root@scaleio02-tovb ~]# scli --query_cluster
    Cluster:
    Name: ScaleIO-tovb, Mode: 3_node, State: Normal, Active: 3/3, Replicas: 2/2
    Virtual IP Addresses: 192.168.56.250, 192.168.70.250
    Master MDM:
    Name: MDM2, ID: 0x1b292daa1fa99011
    IP Addresses: 192.168.56.4, Management IP Addresses: 163.162.230.235, Port: 9011, Virtual IP interfaces: bond1.1056, bond2.1070
    Status: Normal, Version: 3.5.1200
    Slave MDMs:
    Name: MDM1, ID: 0x3cd645b45b7db300
    IP Addresses: 192.168.56.2, Management IP Addresses: 163.162.230.234, Port: 9011, Virtual IP interfaces: bond1.1056, bond2.1070
    Status: Normal, Version: 3.5.1200
    Tie-Breakers:
    Name: TB1, ID: 0x452a4fa35085c802
    IP Addresses: 192.168.56.6, Port: 9011
    Status: Normal, Version: 3.5.1200
    [root@scaleio02-tovb ~]#

    Someone can help me?