UNSOLVED

MaximUnited

updated

17 years ago

M

MaximUnited

111 Posts

0

1989

June 8th, 2009 07:00

Symmtrix disaster?

Hey guys,

Ok, I have a hypothetical scenario:
A Symmetrix has a disaster of some sort, and the entire box is unusable.
If I have warranty, EMC will bring me a new Symmterix to replace the old one.
My question is how will EMC know how to configure the new Symmetrix? Which devices to create, SRDF pairing, etc..? Especially if I make daily changes of configuring meta, clone relations and more..
I have daily backups of the VCMDB, but I guess this is not enough.

Maxim
  • RRR

    6 Operator

    •

    5739 Posts

    1052

    0

    Posted June 8th, 2009 09:00

    Backups of the vcmdb are needed and I guess SRDF info can be retrieved from the other DMX's. But AFAIK info about metas aren't in the vcmdb. I guess you have a real issue there.

    This is an interesting question ! I'm curious as well !
  • MaximUnited

    111 Posts

    1052

    0

    Posted June 8th, 2009 09:00

    Local EMC personel told me that during call home procedure, the Service Processor sends back the BIN file.
    That only happens only just once a week (at best), which is not good.
    A lot of the time there are network/modem issues and the boxes don't dial home.
    We need a better solution!
    And I think so do everyone!
  • RRR

    6 Operator

    •

    5739 Posts

    1052

    0

    Posted June 8th, 2009 09:00

    My guess is that the "laptop/pizza box" inside a Symm/DMX contains a lot of info as well. Let's just hope fire didn't ruin the hardware.
  • MaximUnited

    111 Posts

    1051

    0

    Posted June 8th, 2009 10:00

    Yes, this is the BIN file.
    But how can I save it myself?
    Going with a disk-on-key to the Service processor daily is obvioulsy not an option...
  • Allen Ward

    6 Operator

    •

    2124 Posts

    1051

    0

    Posted June 8th, 2009 10:00

    We actually keep track of all allocated devices and their associated VDEVs, BCVs, R1s, R2, Clones, etc as well as the size of each device allocated to each host and which FAs they are mapped through. You might want to consider starting to track this information somewhere other than in the Symm itself, even though it is a bit of a pain.
  • RRR

    6 Operator

    •

    5739 Posts

    1051

    0

    Posted June 8th, 2009 11:00

    I use spreadsheets..... And yes, reconfiguring from the start after you get a new DMX will be a pain, but it does represent what you had ;)
  • InsaneGeek

    46 Posts

    509

    0

    Posted June 8th, 2009 13:00

    Here is an example of a striped meta device from the symapi_db.bin file I copied to my workstation: it shows the device and where all it's sub devices are on the disks and it it's got a timefinder session, where it's mapped, etc as well. The output got a bit munged when copying into the webpage (don't think it took the tabs & spaces well).

    symdev -sid -offline show 14CF

    Device Physical Name : Not Visible

    Device Symmetrix Name : 14CF
    Device Serial ID : N/A
    Symmetrix ID : 0001901

    Attached BCV Device : N/A

    Attached VDEV TGT Device : N/A

    Vendor ID :
    Product ID :
    Product Revision :
    Device WWN : 60060
    Device Emulation Type : FBA
    Device Defined Label Type: N/A
    Device Defined Label : N/A
    Device Sub System Id : 0x0015
    Cache Partition Name : DEFAULT_PARTITION

    Device Block Size : 512

    Device Capacity
    {
    Cylinders : 227200
    Tracks : 3408000
    512-byte Blocks : 436224000
    MegaBytes : 213000
    KiloBytes : 218112000
    }

    Device Configuration : RAID-6 (Meta Head,
    Non-Exclusive Access)

    Device is WORM Enabled : No
    Device is WORM Protected : No

    SCSI-3 Persistent Reserve: Disabled

    Dynamic Spare Invoked : No

    Dynamic RDF Capability : RDF1_OR_RDF2_Capable

    STAR Mode : No
    STAR Recovery Capability : None
    STAR Recovery State : NA

    Device Service State : Normal

    Device Status : Ready (RW)
    Device SA Status : Ready (RW)

    Front Director Paths (2):
    {
    ----------------------------------------------------------------------
    POWERPATH DIRECTOR PORT LUN
    --------- ---------- ---- -------- ---------
    PdevName Type Type Num Sts VBUS TID SYMM Host
    ----------------------------------------------------------------------
    Not Visible N/A FA 08B:0 RW 000 00 05E N/A
    Not Visible N/A FA 09B:0 RW 000 00 05E N/A
    }

    Meta Configuration : Striped
    Meta Stripe Size : 960k (2 Cylinders)
    Meta Device Members (8) :
    {
    ----------------------------------------------------------------------
    BCV DATA RDF DATA
    ---------------------------- --------------------------
    Sym Cap Std Inv BCV Inv Pair R1 Inv R2 Inv Pair
    Dev (MB) Tracks Tracks State Tracks Tracks State
    ----------------------------------------------------------------------
    --> 14CF 26625 - - N/A - - N/A
    14D0 26625 - - N/A - - N/A
    14D1 26625 - - N/A - - N/A
    14D2 26625 - - N/A - - N/A
    14D3 26625 - - N/A - - N/A
    14D4 26625 - - N/A - - N/A
    14D5 26625 - - N/A - - N/A
    14D6 26625 - - N/A - - N/A
    ----------------------------------------------------------------------
    213000 - - - -
    }

    Mirror Set Type : [RAID-6,N/A,N/A,N/A]

    Mirror Set DA Status : [RW,N/A,N/A,N/A]

    Mirror Set Inv. Tracks : [0,0,0,0]

    Back End Disk Director Information
    {
    Hyper Type : RAID-6
    Hyper Status : Ready (RW)
    Disk [Director, Interface, TID] : [N/A,N/A,N/A]
    Disk Director Volume Number : N/A
    Hyper Number : N/A
    }

    RAID-6 Device Information
    {
    Number of Tracks in a Stripe : 4
    Overall Ready State of RAID-6 Device : Ready
    Overall WriteProtect State of RAID-6 Device : Enabled
    Failing Member Mask : N/A
    RAID-6 Hyper Devices (6+2):
    {
    Device : 14CF (M)
    {
    ------------------------------------------------------
    Disk DA Hyper Member Disk
    DA :IT Vol# Num Cap(MB) Num Status Grp# Cap(MB)
    ------------------------------------------------------
    05A:C10 720 8 4438 8 RW 0 286102
    11A:C4 131 8 4438 1 RW 0 286102
    05B:D10 1519 8 4438 4 RW 0 286102
    11B:D4 565 8 4438 5 RW 0 286102
    06C:D4 564 8 4438 3 RW 0 286102
    12C:D10 1523 8 4438 2 RW 0 286102
    06D:C4 131 8 4438 7 RW 0 286102
    12D:C10 722 8 4438 6 RW 0 286102
    }
    Device : 14D0 (m)
    {
    ------------------------------------------------------
    Disk DA Hyper Member Disk
    DA :IT Vol# Num Cap(MB) Num Status Grp# Cap(MB)
    ------------------------------------------------------
    06A:D4 564 8 4438 1 RW 0 286102
    12A:D10 1475 8 4438 8 RW 0 286102
    06B:C4 131 8 4438 5 RW 0 286102
    12B:C10 721 8 4438 4 RW 0 286102
    05C:C10 721 8 4438 2 RW 0 286102
    11C:C4 131 8 4438 3 RW 0 286102
    05D:D10 1472 8 4438 6 RW 0 286102
    11D:D4 565 8 4438 7 RW 0 286102
    }
    Device : 14D1 (m)
    {
    ------------------------------------------------------
    Disk DA Hyper Member Disk
    DA :IT Vol# Num Cap(MB) Num Status Grp# Cap(MB)
    ------------------------------------------------------
    06A:C7 194 8 4438 1 RW 0 286102
    12A:C11 568 8 4438 8 RW 0 286102
    06B:D7 625 8 4438 5 RW 0 286102
    12B:D11 1616 8 4438 4 RW 0 286102
    05C:D11 1617 8 4438 2 RW 0 286102
    11C:D7 630 8 4438 3 RW 0 286102
    05D:C11 567 8 4438 6 RW 0 286102
    11D:C7 195 8 4438 7 RW 0 286102
    }
    Device : 14D2 (m)
    {
    ------------------------------------------------------
    Disk DA Hyper Member Disk
    DA :IT Vol# Num Cap(MB) Num Status Grp# Cap(MB)
    ------------------------------------------------------
    05A:D11 1667 8 4438 8 RW 0 286102
    11A:D7 688 8 4438 1 RW 0 286102
    05B:C11 565 8 4438 4 RW 0 286102
    11B:C7 194 8 4438 5 RW 0 286102
    06C:C7 194 8 4438 3 RW 0 286102
    12C:C11 562 8 4438 2 RW 0 286102
    06D:D7 625 8 4438 7 RW 0 286102
    12D:D11 1672 8 4438 6 RW 0 286102
    }
    Device : 14D3 (m)
    {
    ------------------------------------------------------
    Disk DA Hyper Member Disk
    DA :IT Vol# Num Cap(MB) Num Status Grp# Cap(MB)
    ------------------------------------------------------
    05A:C12 782 8 4438 8 RW 0 286102
    11A:C6 193 8 4438 1 RW 0 286102
    05B:D12 1581 8 4438 4 RW 0 286102
    11B:D6 627 8 4438 5 RW 0 286102
    06C:D6 626 8 4438 3 RW 0 286102
    12C:D12 1585 8 4438 2 RW 0 286102
    06D:C6 193 8 4438 7 RW 0 286102
    12D:C12 784 8 4438 6 RW 0 286102
    }
    Device : 14D4 (m)
    {
    ------------------------------------------------------
    Disk DA Hyper Member Disk
    DA :IT Vol# Num Cap(MB) Num Status Grp# Cap(MB)
    ------------------------------------------------------
    06A:D6 625 8 4438 1 RW 0 286102
    12A:D12 1536 8 4438 8 RW 0 286102
    06B:C6 192 8 4438 5 RW 0 286102
    12B:C12 782 8 4438 4 RW 0 286102
    05C:C12 782 8 4438 2 RW 0 286102
    11C:C6 192 8 4438 3 RW 0 286102
    05D:D12 1533 8 4438 6 RW 0 286102
    11D:D6 626 8 4438 7 RW 0 286102
    }
    Device : 14D5 (m)
    {
    ------------------------------------------------------
    Disk DA Hyper Member Disk
    DA :IT Vol# Num Cap(MB) Num Status Grp# Cap(MB)
    ------------------------------------------------------
    06A:Dc 810 8 4438 8 RW 0 286102
    11A:Ce 441 8 4438 7 RW 0 286102
    06B:Cc 377 8 4438 2 RW 0 286102
    11B:Dc 813 8 4438 5 RW 0 286102
    06C:De 874 8 4438 4 RW 0 286102
    11C:Cc 376 8 4438 1 RW 0 286102
    06D:Cc 378 8 4438 6 RW 0 286102
    11D:Dc 811 8 4438 3 RW 0 286102
    }
    Device : 14D6 (m)
    {
    ------------------------------------------------------
    Disk DA Hyper Member Disk
    DA :IT Vol# Num Cap(MB) Num Status Grp# Cap(MB)
    ------------------------------------------------------
    06A:Cd 380 8 4438 7 RW 0 286102
    06A:De 871 8 4438 5 RW 0 286102
    06B:Dd 811 8 4438 3 RW 0 286102
    11B:Cd 381 8 4438 6 RW 0 286102
    06C:Cd 380 8 4438 8 RW 0 286102
    11C:Dd 816 8 4438 2 RW 0 286102
    06D:Dd 811 8 4438 1 RW 0 286102
    11D:Cd 381 8 4438 4 RW 0 286102
    }
    }
    }
  • HankDorsett

    66 Posts

    1051

    0

    Posted June 8th, 2009 13:00

    I use symconfigure and save all of my config files to local disk on a Solaris box. If the DMX was the only system hit by (insert system crushing object here) I would have EMC load the BIN from from a week ago and run any config files between backup and crash. Doing it this way would only add a couple of hours to a week, or more, long recovery.

    I think ECC and SMC has the ability to save the config output, I have both but don't use them for configuration.
  • InsaneGeek

    46 Posts

    1051

    1

    Posted June 8th, 2009 13:00

    Personally, if they are brining you a new symm... to me that's an opportunity to make the environment better and fix some past "ills" and start fresh (but that's just me not in a crunch situation, where time is infinite). If it's an SRDF device, you can look at the other running symm and get the appropriate device information of what it should be on the other side. i.e. you have the pairing so you know the sym device ID's of both sides, looking at the source device will tell you about the device on the other side: is it a meta, what is it's side (assuming you aren't doing anything like different sized SRDF target devices).

    But to answer your question, I believe the symapi database should cover the "how is the symm configed" question. Under unix it's normally in the /var/symapi/db directory (windows similar look for the symapi\db directory). The problem would be in getting a change that was not created on that server directly. You can do a "symcfg discover" to force a database update, but you would still have a gap. If that last change isn't critical or you have a process to regularly force a refresh that fits within your RPO objective I think would work for you.

    One of the cool things you can do is take the symapi_db.bin file to a host completely unconnected to the storage and run solutions enabler commands. i.e. I just copied over that file from a sun host to my linux host and can run "symdev -offline -sid list" and it spits out the devices, same thing with "symsnap -offline -sid list" dumps all of the timefinder snap sessions.
  • MaximUnited

    111 Posts

    875

    0

    Posted June 9th, 2009 00:00

    Hmm... I save some symconfigure files, but some are randonly spread on various unix machines in /tmp ... The rest of the changes made by my team are using the ECC.

    But mainly, I'm talking about a disaster. If I have a lot of time I will reconfigure the Symm better. But I need data to be available ASAP.

    Furthermore, I think the restore process should be EMC's responsibility. I'm in charge of backing up the data itself, but the configuration should be on them. At least in my opinion.