You can us "du" to compare sizes of each location.. You can also compare file counts using fine. Something like "find targetdir -type f | wc -l" on source and destination..
We are also planning to migrate data from Celerra to Islion. Can u share any docs related to this and also let me know what are the key points I shld keep in mind during the activity.
As mentioned above there are several ways to get the size of a directory that is being copied over:
On the Celerra or VNX:
server_df ALL will get the entire size of a filesystem.
On The isilon cluster, you can do a du -sh /ifs/path . However keep in mind that du -sh will return the size including parity, so it won't be an apples to apples comparison. Use a capital A in your du Syntax to get the equivelant size excluding Parity. That's not to say that the Isilon cluster depending on the number of nodes isn't more efficient, it's just that a Celerra or VNX hides the RAID parity hit from you because it is per raid group, rather than per file. Here is an example from my lab:
isi01-1# du -sh /ifs/isi01/home
734M /ifs/isi01/home
isi01-1# du -shA /ifs/isi01/home
487M /ifs/isi01/home
A du can take a good deal of time to calculate. An alternative is to (if you own the SmartQuotas license), place an advisory quota on the target directory on Isilon. Then a simple isi quota quotas list --v will show you the details, without the waiting and treewalk associated with du. Just be advised that once you create a directory quota on a path, that path cannot be deleted, until the quota is deleted first. Last if you have InsightIQ, every day an FsAnalyze job will run, which gives you the filesystem analytics data that is visible in InsightIQ. It can lab by as much as 24 hours compared to live data, but it is much easier than a du, or using quotas.
Celerra or VNX to Isilon Migrations
As to Celerra or VNX to Isilon migration. If you are the OneFS 7.0.2.1 or above code, there is a feature built into OneFS, called isi_vol_copy_vnx. It's goal is to perform an NDMP-based pull migration from a Celerra or VNX to Isilon. It is extremely fast and powerful. A NetApp version has existed for years, simply called isi_vol_copy. Note that this is not currently documented in the OneFS CLI or Administration guide, because until recently it was only leveraged by EMC Global Services to perform migrations. Public-facing collateral on support.emc.com should be available soon.
Exceptions: As it sits today, Do not use isi_vol_copy_vnx with :
1. multiprotocol filesystems (because of the inherint complexity of how the permissions are gelled based upon the filesystem's accesspolicy upon mount)
2. VNXe (not supported)
3. FLR (retaining retention dates to migrate to Isilon SmartLock is not yet supported)
4. Sources using DHSM/Filemover API to tier data off with CTA/FMA, Archivio, Symanted eVault, etc.
The other migration options you might consider would be:
1. Rsync: Great for NFS-only filesystems/data sets (you can run rsync directly from NFS mounts
2. Emcopy: wonderful for SMB-only migrations.
Understanding Multiprotocol Differences between Celerra/VNX and Isilon
Just make sure that you are aware in any migration from VNX or Celerra to Isilon that the 2 handle multiprotocol permissions differently as there is no defined industry standard for multiprotocol permissions. The big bullet point there is that Isilon only retains one set of permissions on each and every file, either POSIX, or ACLs, but not both. This is by-design and ensures that regardless of access method, SMB, NFS, FTP, sFTP, FTPs, etc. any given user has exactly the same rights to the data. The permissions cannot therefore be circumvented by merely chosing a different protocol to access the data with. A Celerra or VNX, conversely with a default accesspolicy of NATIVE keeps a totally separate set of NTFS ACL's and POSIX permissions on every single file and directory.
File Integrity
Also the last point, if you need to ensure for compliance data integrity across a migration, rsync has a --checksum option. Emcopy has a --cm /md5 switch. I would run either one with a verbose log output, just in case there are any later questions.
Madhug1
2 Intern
•
127 Posts
0
August 28th, 2013 10:00
Hi,
you can find the size of the User directory using this command : du -csh /ifs/directoryname
Thanks
Madhu
ysrk
2 Posts
0
August 28th, 2013 15:00
Thank you..
If you get a chance..can you check the below scenario
I need some suggestions regarding Celerra to Isilon migration.we did Celerra to Isilon migration(all NFS shares) using Rsync...
here i want to know do we have any tools or script to compare the source directory and target directory
Purpose : To know or confirm the 1:1 rsync copy ...is it copy all the files in the source directory to target directory
cincystorage
2 Intern
•
467 Posts
0
August 28th, 2013 19:00
You can us "du" to compare sizes of each location.. You can also compare file counts using fine. Something like "find targetdir -type f | wc -l" on source and destination..
Don't robocopy has the /MIR option.
jedai_2192b0
1 Message
0
November 13th, 2013 23:00
Hi
We are also planning to migrate data from Celerra to Islion. Can u share any docs related to this and also let me know what are the key points I shld keep in mind during the activity.
crklosterman
450 Posts
4
November 15th, 2013 12:00
All,
File/Directory Sizes
As mentioned above there are several ways to get the size of a directory that is being copied over:
On the Celerra or VNX:
server_df ALL will get the entire size of a filesystem.
On The isilon cluster, you can do a du -sh /ifs/path . However keep in mind that du -sh will return the size including parity, so it won't be an apples to apples comparison. Use a capital A in your du Syntax to get the equivelant size excluding Parity. That's not to say that the Isilon cluster depending on the number of nodes isn't more efficient, it's just that a Celerra or VNX hides the RAID parity hit from you because it is per raid group, rather than per file. Here is an example from my lab:
isi01-1# du -sh /ifs/isi01/home
734M /ifs/isi01/home
isi01-1# du -shA /ifs/isi01/home
487M /ifs/isi01/home
A du can take a good deal of time to calculate. An alternative is to (if you own the SmartQuotas license), place an advisory quota on the target directory on Isilon. Then a simple isi quota quotas list --v will show you the details, without the waiting and treewalk associated with du. Just be advised that once you create a directory quota on a path, that path cannot be deleted, until the quota is deleted first. Last if you have InsightIQ, every day an FsAnalyze job will run, which gives you the filesystem analytics data that is visible in InsightIQ. It can lab by as much as 24 hours compared to live data, but it is much easier than a du, or using quotas.
Celerra or VNX to Isilon Migrations
As to Celerra or VNX to Isilon migration. If you are the OneFS 7.0.2.1 or above code, there is a feature built into OneFS, called isi_vol_copy_vnx. It's goal is to perform an NDMP-based pull migration from a Celerra or VNX to Isilon. It is extremely fast and powerful. A NetApp version has existed for years, simply called isi_vol_copy. Note that this is not currently documented in the OneFS CLI or Administration guide, because until recently it was only leveraged by EMC Global Services to perform migrations. Public-facing collateral on support.emc.com should be available soon.
Exceptions: As it sits today, Do not use isi_vol_copy_vnx with :
1. multiprotocol filesystems (because of the inherint complexity of how the permissions are gelled based upon the filesystem's accesspolicy upon mount)
2. VNXe (not supported)
3. FLR (retaining retention dates to migrate to Isilon SmartLock is not yet supported)
4. Sources using DHSM/Filemover API to tier data off with CTA/FMA, Archivio, Symanted eVault, etc.
The other migration options you might consider would be:
1. Rsync: Great for NFS-only filesystems/data sets (you can run rsync directly from NFS mounts
2. Emcopy: wonderful for SMB-only migrations.
Understanding Multiprotocol Differences between Celerra/VNX and Isilon
Just make sure that you are aware in any migration from VNX or Celerra to Isilon that the 2 handle multiprotocol permissions differently as there is no defined industry standard for multiprotocol permissions. The big bullet point there is that Isilon only retains one set of permissions on each and every file, either POSIX, or ACLs, but not both. This is by-design and ensures that regardless of access method, SMB, NFS, FTP, sFTP, FTPs, etc. any given user has exactly the same rights to the data. The permissions cannot therefore be circumvented by merely chosing a different protocol to access the data with. A Celerra or VNX, conversely with a default accesspolicy of NATIVE keeps a totally separate set of NTFS ACL's and POSIX permissions on every single file and directory.
File Integrity
Also the last point, if you need to ensure for compliance data integrity across a migration, rsync has a --checksum option. Emcopy has a --cm /md5 switch. I would run either one with a verbose log output, just in case there are any later questions.
I hope this helps answer some of your questions.
Sincerely,
Chris Klosterman, ICSP, ICIE, CCNA, VCP
Email: chris.klosterman@emc.com
Senior Solution Architect
Offer and Enablement Team
EMC²| Isilon Storage Division
Twitter: @croaking