1. You're using the right tool, but the wrong version. You need to use version 4.17 or later, and add the /ignoredhsm switch to the command (it'll strip off the offline extended file attrib during copy).
2. You have to change the DHSM settings per filesystem for backup operations;
a) first get a list of all filesystems with DHSM enabled with 'fs_dhsm -list'
b) now change the backup method from offline to passthrough for all of those filesystems (see in bold below). Note that this will affect backups via NDMP as well, so if you do normal NDMP backups of the Celerra/VNX in question expect those backup sets to grow significantly, and take longer.
Error 2100: usage: fs_dhsm
-list
| -info [ |id= ]
| -modify { |id= } [-state {enabled|disabled}]
[-popup_timeout ] [-backup {offline|passthrough}]
3. That's it, it should take care of this issue, be sure that the account doing the copy is a member of local administrators and backup operators on the source CIFS server, and a member of the backupadmin right on the target isilon cluster, or has run-as-root permissions to the target migration share. (disclaimer: be very careful with this, and only give these rights to that one account).
Last as an FYI, if you have historical SIDs on your source dataset left over as an artifact from a previous AD migration, then you'll want to use the /sidmapfile switch as well and provide it a translation table of the HSID:newSID, but you can read more about that in the emcopy documentation.
I hope this helps, please mark your question as answered if it does.
First and foremost, always make sure you're using the latest version of EMCopy. They are constantly fixing little issues and adding functionality. This sounds like an issue I recall from 4.14, but I think the issue has long-since been fixed.
If you're on the latest version of EMCopy and still see this issue, I'd take a look at your recall rules in your filesystem. If you have sufficient space, I would experiment with changing your "read policy override" to passthrough or partial or full. Note that "full" tells the NS80 to copy the entire file back the filesystem when EMCopy reads the file, so this can quickly fill the filesystem. You might try partial first, and see if that has the desired effect. Be sure to closely monitor filesystem consumption, as even partial recalls may use blocks in the active filesystem.
Finally, if you're up-to-date with your CTA code version, why not just use the CTA to migrate the files? The file migration feature is pretty well documented. You can simply add the Isilon as a target and have it do the copy for you.
And another short follow-up item. Do not change the read-policy override as suggested earlier. That's going to cause you a lot of problems, because it causes just on read to fully re-hydrate the archived data into the primary filesystem. In most cases like this with CTA and some archive platform like Centera, there just isn't enough space to do so on the Celerra, so you run the filesystems full. The default behavior of DHSM on read is to pass-through the reads, meaning if the data is read and not modified, it's loaded into RAM of the datmover then sent over the wire, and with no changes, the read is discarded, and nothing changes on disk. Read-Policy-Override recalls everything when it's read.
We tried to migrate stubbed NS* filesystems with isi_vol_copy_vnx, but for filesystems bigger than around 5TB, we were not succesfull. It was very slow, the table, isi_vol_copy_vnx has to maintain gets huge and incrementals fail for variuos resons. As it was NFS only exported filesystems we ended up with a set of scripts using rsync and running several rsync threads in parallel.
No matter which route you take make shure call backs are on pass-trough, otherwise chances are big you fill up the primary filesystem
We have one SMB share remaining and planned to use robocopy, i wonder if emcopy would suite better.
And a little rant: in our opinion EMC is absolutely not avare what a nightmare it is to migrate from NS* or VNX to Isilon, the tools provided are unsuitable when you have to migrate hundreds of TB of data. Why is there no function where you can mount the old filesystems directly to the isilon and start a multithreaded background migration job?
The general recommendation would be to leverage EMCopy over Robocopy. As Chris earlier mentioned, we have added many enhancements to EMCopy to deal with several issues that we have experienced with migrations.
If you can provide me your contact information, I'd like to understand more of your thoughts on how migrations could be improved. Feel free to send me an email at scott.owens@emc.com
Are you replacing NS/CTA/Centera with Isilon? If so then yes you'll do a pass-through read / recall so that the full size on Isilon is the stub + the archived content. Use emcopy with the /ignoredhsm switch to remove the offline extended file attribute. If you're just replacing Centera with Isilon as the new archive target, then no, it's just a CTA repository migration, the CTA will update the stubs on the NS as the data is moved, and shuffle the data across the back-end.
You use a stub-aware migration tool, or you use a block-based migration method.
So in other words, use Celerra Replicator, or use emcopy. But make sure you understand how to configure DHSM on the new VNX to talk to the CTA/CAS. You’ve posted this in an Isilon product forum, hence the confusion.
I need a solution where I have VNX and CTA and ECS .. we are archiving the data via CTA to ECS.
Now setting up a DR solution for this where same setup will be in DR site.
Now i want to understand what shall we do for the CTA to understand the stub in DR site so that it cal point to the ECS data which is replcated from source ECS
crklosterman
450 Posts
0
February 18th, 2015 08:00
A few items here:
1. You're using the right tool, but the wrong version. You need to use version 4.17 or later, and add the /ignoredhsm switch to the command (it'll strip off the offline extended file attrib during copy).
a) you can get that version of emcopy here: https://download.emc.com/downloads/DL14101_EMCOPY_File_migration_tool_4.17.exe
2. You have to change the DHSM settings per filesystem for backup operations;
a) first get a list of all filesystems with DHSM enabled with 'fs_dhsm -list'
b) now change the backup method from offline to passthrough for all of those filesystems (see in bold below). Note that this will affect backups via NDMP as well, so if you do normal NDMP backups of the Celerra/VNX in question expect those backup sets to grow significantly, and take longer.
Error 2100: usage: fs_dhsm
-list
| -info [ |id= ]
| -modify { |id= } [-state {enabled|disabled}]
[-popup_timeout ] [-backup {offline|passthrough}]
3. That's it, it should take care of this issue, be sure that the account doing the copy is a member of local administrators and backup operators on the source CIFS server, and a member of the backupadmin right on the target isilon cluster, or has run-as-root permissions to the target migration share. (disclaimer: be very careful with this, and only give these rights to that one account).
Last as an FYI, if you have historical SIDs on your source dataset left over as an artifact from a previous AD migration, then you'll want to use the /sidmapfile switch as well and provide it a translation table of the HSID:newSID, but you can read more about that in the emcopy documentation.
I hope this helps, please mark your question as answered if it does.
Thanks,
Chris Klosterman
Senior Solution Architect
EMC Isilon Offer & Enablement Team
email: chris.klosterman@emc.com
twitter: @croaking
umichklewis
3 Apprentice
•
1.2K Posts
1
February 18th, 2015 06:00
First and foremost, always make sure you're using the latest version of EMCopy. They are constantly fixing little issues and adding functionality. This sounds like an issue I recall from 4.14, but I think the issue has long-since been fixed.
If you're on the latest version of EMCopy and still see this issue, I'd take a look at your recall rules in your filesystem. If you have sufficient space, I would experiment with changing your "read policy override" to passthrough or partial or full. Note that "full" tells the NS80 to copy the entire file back the filesystem when EMCopy reads the file, so this can quickly fill the filesystem. You might try partial first, and see if that has the desired effect. Be sure to closely monitor filesystem consumption, as even partial recalls may use blocks in the active filesystem.
Finally, if you're up-to-date with your CTA code version, why not just use the CTA to migrate the files? The file migration feature is pretty well documented. You can simply add the Isilon as a target and have it do the copy for you.
Let us know if that helps!
Karl
AShah7
25 Posts
0
February 18th, 2015 07:00
thanks for the suggestion
we are at DART 6.065
FMA 7.5 b226
All our old kit is out of support hence the reason for emcopy
I dont think we can use isi_vol_copy?
Any ideas how I can check the read overide policy?
thanks
crklosterman
450 Posts
1
February 18th, 2015 08:00
And another short follow-up item. Do not change the read-policy override as suggested earlier. That's going to cause you a lot of problems, because it causes just on read to fully re-hydrate the archived data into the primary filesystem. In most cases like this with CTA and some archive platform like Centera, there just isn't enough space to do so on the Celerra, so you run the filesystems full. The default behavior of DHSM on read is to pass-through the reads, meaning if the data is read and not modified, it's loaded into RAM of the datmover then sent over the wire, and with no changes, the read is discarded, and nothing changes on disk. Read-Policy-Override recalls everything when it's read.
~Chris
fox_inti1
12 Posts
1
February 18th, 2015 23:00
We tried to migrate stubbed NS* filesystems with isi_vol_copy_vnx, but for filesystems bigger than around 5TB, we were not succesfull. It was very slow, the table, isi_vol_copy_vnx has to maintain gets huge and incrementals fail for variuos resons. As it was NFS only exported filesystems we ended up with a set of scripts using rsync and running several rsync threads in parallel.
No matter which route you take make shure call backs are on pass-trough, otherwise chances are big you fill up the primary filesystem
We have one SMB share remaining and planned to use robocopy, i wonder if emcopy would suite better.
And a little rant: in our opinion EMC is absolutely not avare what a nightmare it is to migrate from NS* or VNX to Isilon, the tools provided are unsuitable when you have to migrate hundreds of TB of data. Why is there no function where you can mount the old filesystems directly to the isilon and start a multithreaded background migration job?
Regards
Andi
scott_owens
60 Posts
0
February 20th, 2015 10:00
Andi,
The general recommendation would be to leverage EMCopy over Robocopy. As Chris earlier mentioned, we have added many enhancements to EMCopy to deal with several issues that we have experienced with migrations.
If you can provide me your contact information, I'd like to understand more of your thoughts on how migrations could be improved. Feel free to send me an email at scott.owens@emc.com
-Scott
crklosterman
450 Posts
0
July 23rd, 2015 13:00
Are you replacing NS/CTA/Centera with Isilon? If so then yes you'll do a pass-through read / recall so that the full size on Isilon is the stub + the archived content. Use emcopy with the /ignoredhsm switch to remove the offline extended file attribute. If you're just replacing Centera with Isilon as the new archive target, then no, it's just a CTA repository migration, the CTA will update the stubs on the NS as the data is moved, and shuffle the data across the back-end.
~Chris
crklosterman
450 Posts
0
July 23rd, 2015 15:00
You use a stub-aware migration tool, or you use a block-based migration method.
So in other words, use Celerra Replicator, or use emcopy. But make sure you understand how to configure DHSM on the new VNX to talk to the CTA/CAS. You’ve posted this in an Isilon product forum, hence the confusion.
Chris Klosterman
Email: chris.klosterman@emc.com
Advisory Solution Architect
Offer and Enablement Team
EMC²| Emerging Technologies Division
Pankie
1 Message
0
October 6th, 2019 00:00
I need a solution where I have VNX and CTA and ECS .. we are archiving the data via CTA to ECS.
Now setting up a DR solution for this where same setup will be in DR site.
Now i want to understand what shall we do for the CTA to understand the stub in DR site so that it cal point to the ECS data which is replcated from source ECS