Unsolved
This post is more than 5 years old
3 Posts
1
4227
Block replication of iSCSI LUN between VSA's stuck at 66%
I configured the replication interfaces and verified, then created a connection. When I try to replicate a block LUN to VSA-02 the progress is hung at 66% (Creating replication session after provisioning destination). I've tried rebooting the 2nd VSA and it is stuck in service mode. Any ideas how to reset it so I can try again?
chenw87
43 Posts
0
May 23rd, 2016 08:00
Try running these commands in the console as the 'service' user:
svc_rescue_state -c
svc_shutdown -r
CockneyUSA
60 Posts
0
May 25th, 2016 08:00
I have the same issue, sort of. The identical situation occurred with the replication, but I was replicating a NAS server.. The source went down and only comes up in service mode. I can login into the VSA with service credentials but none of the recommended fixes work. I had to re-initialize the VSA.
SO, is there a problem with the replication that corrupts the VM?
CockneyUSA
60 Posts
0
May 25th, 2016 10:00
Problem is repeatable. Tried replication from new VSA and it crashed during initial replication of a LUN. Consistently crashing now. Will come up in Service mode but will require reimaging (full wipe).
Won't be trying this again until I hear a fix.
CockneyUSA
60 Posts
0
May 26th, 2016 06:00
I'll set it up again today. I have to re-initialize the source VSA and sometimes after the replication I can no longer get into it even in service mode. I can get the logs from the destination VSA though.
chenw87
43 Posts
0
May 26th, 2016 06:00
I just tried this and it works fine for me. I can't seem to reproduce this issue. Are you able to collect and upload the logs from both VSAs? You can get them from Service -> Service Tasks -> Collect Service Information.
CockneyUSA
60 Posts
0
May 26th, 2016 12:00
Where do I upload the logs to?
chenw87
43 Posts
0
May 26th, 2016 12:00
Can you attach it here? If not, you can upload them to this FTP site:
https://ftp.emc.com/action/login?domain=ftp.emc.com&username=Y3psfamKX&password=2f272BIBII
CockneyUSA
60 Posts
0
May 27th, 2016 09:00
I'll upload when I get it done. I'm having to re-initialize both VSAs as I cannot clean up the old replication settings!!
chenw87
43 Posts
1
May 27th, 2016 11:00
Thank you for taking the time to reproduce and collect the data. I have the logs and we'll investigate.
CockneyUSA
60 Posts
0
May 27th, 2016 11:00
I've uploaded 3 log files. 1 from the source and 2 from the destination. The second destination log is from after the session started and failed. The source VSA has crashed three times so far, one during the replication session (66%) and twice when trying to access the GUI.
I will try to generate a log from the console if I can.
4th log file uploaded. Had to time it just right between crashes.
chenw87
43 Posts
0
May 31st, 2016 11:00
Engineering isn't seeing any replication related activity in the logs
Can you provide some the exact steps you performed to reproduce the issue?
Can you also confirm these were collected after the SP went into service mode?
If the system is still available, can you enable SSH and check the following directories for any other logs/dumps?
/EMC/backend/service/data_collection
/EMC/backend/service/data_collection/cores
Thanks!
CockneyUSA
60 Posts
0
May 31st, 2016 12:00
The logs I uploaded are from both VSAs, before and after the replication started. Replication always hung at 66% before the crash. I'll see if I can get to the logs/dumps you requested.
Alan
chenw87
43 Posts
0
June 2nd, 2016 07:00
Thanks Alan. If you could upload the dumps from the /EMC/backend/service/data_collection/cores directory, that would be very helpful.
CockneyUSA
60 Posts
0
June 2nd, 2016 10:00
Working on it this afternoon.
Regards
Alan Warwick | Senior Storage Solutions Engineer
chenw87
43 Posts
0
June 2nd, 2016 10:00
Can you also run the following command on your ESXi server and provide us with the output:
esxcli hardware cpu cpuid get -c 1
Thanks!