Unsolved
This post is more than 5 years old
25 Posts
0
2288
md5 checksum differs on same file Centera (CAS)
We are attempting to run md5sum on the source and destination locations after migrating
from the CAS -> FMA - > NS80 (NS80 front-end) to the Isilon to check for possible corruptions
I have noticed that we are getting different checksums (md5) being generated for the same file
This issue seems to only happen on some files and is not consistent across all files. I cannot
see any pattern in terms of size or anything else.
The issue also manifests when there is I/O in progress on the Centera and a checksum is taken
during the same time.
Again issue is not happening all the time, but maybe seen once in around 1000 files...
Because of the above we are getting a lot of md5 mismatches when running a compare but with no corruption seem on the destination......
I'm not sure whether the way the Centera is hydrating the files has anything to do with this at all?
md5 checking is enabled on the EMCopy flag but this hasn't proved reliable as there are files which have been copied with different checksums but Emcopy has not reported them
I'm not sure other customers have experienced this before but due to the sensitive data onsite a requirement
has been put into place to ensure integrity when migrating. We have files on archive files going back 1995 and older
thanks
Gotsche_Helmut
48 Posts
0
May 20th, 2015 07:00
Centera is not "hydrating" files, as files are returned in exactly the same binary form as received, protected by the content Address, which is a combination of MD5 and SHA1 and a marker in position 14
Please describe in more detail in which state you see a MD5 mismatch.
e.g md5sum is executed after CTA has recalled the file from Centera and then after the file has been copied to Isilon?
mfh2
208 Posts
0
May 20th, 2015 08:00
Are the affected files the same number of bytes on the source and the target? Does re-testing conclusively determine there is a binary difference without fail? I've found it to be fairly common for the Celerra/CTA/Centera combination to intermittently fail to return data for a file read when under high load (such as when a NAS-level data migration is in progress). You might throttle back your testing (if you are currently doing parallelized reads, or copying and testing concurrently) to try to get more reliable results.
Best Regards,
Mike Horgan
mfh2
208 Posts
0
May 21st, 2015 07:00
Hi Shah7 -
Your final test result (with no CTA/CAS in the chain) is very odd indeed. I would probably open a ticket to get EMC support involved.
Good Luck,
Mike Horgan
AShah7
25 Posts
0
May 21st, 2015 07:00
The md5sum is executed on the stubfile and then again once it is hydrated on the Isilon
(fyi - hydrated and non-hydrated stubs give the same md5sum)
I have seen the md5sum of the same source file change, typically when there is i/o happening on the CAS
When running my script which generates md5sum's, some files would fail with i/o errors which then need to be processed again. I also see null hash's generated for files with data inside (d41d8cd98f00b204e9800998ecf8427e) which probably means nothing is being pulled back. The number of bytes for the files seem to match when running a diff or cmp the two files are flagged as different so no surprise that the checksums don't match.
I have tried to throttle the emcopy to use 30-50 threads, perhaps this should be lower
I'm also seeing the same issue with moving data from the NS80 (no CAS) to the Isilon. On some files the md5sums do not match because the files are different. Copying to my local drive so far has not yet manifested this issue anyway.