Highlighted
8 Krypton

isi_vol_copy issues

Jump to solution

What is the best tool to do data migration from NetApp to Isilon ? Running code 7.0.2.1 and 7.0.2.5 ( Prod and DR)

Facing lot of issues with isi_vol_copy tool while doing file migration from NetApp to Isilon. The volumes on the source array is <3TB in size, <80% utilized and there is enough space for checkpoints.

The isi_vol_copy command works, but files on Isilon have different "create date" time stamps. File properties, create date appear as "Wednesday, ‎December ‎31, ‎1969, ‏‎7:00:00 PM" instead of the acutal create date of files.

Resolution: will be fixed in new onefs version.

Incremental copy fails with following error message on code 7.0.2.1.

Resolution: The issue Will be fixed in new onefs version. But a fresh new copy job to a different folder on the cluster worked.

inode number: 221847-1-0

flags: REMOVED|KEEP

LogMessage {type NDMP_LOG_NORMAL} {msg_id 0} {msg DUMP: Network communication error

} {associated_msg 0} {associated_msg_seq 0}

LogMessage {type NDMP_LOG_NORMAL} {msg_id 0} {msg DUMP: DUMP IS ABORTED

} {associated_msg 0} {associated_msg_seq 0}

LogMessage {type NDMP_LOG_NORMAL} {msg_id 0} {msg DUMP: Deleting "/vol/XXXXX/../snapshot_for_backup.227" snapshot.

} {associated_msg 0} {associated_msg_seq 0}

LogMessage {type NDMP_LOG_NORMAL} {msg_id 0} {msg Connection or IO Error.

} {associated_msg 0} {associated_msg_seq 0}

nasa: Connection halted: HALT: Connect error!

The transfer aborted with error.

isilon1-1#

Incremental copy failure. Resolution: will be fixed in new onefs version.

LogMessage {type NDMP_LOG_NORMAL} {msg_id 0} {msg DUMP: dumping (Pass IV) [regular files]

} {associated_msg 0} {associated_msg_seq 0}

fopen: No such file or directory

cannot open mode file #

directory mode, owner, and times not set

Bad magic: 0x3003e (swabl: 0x3e000300) (nfs: 0xea6c)

/build/mnt/src/isilon/bin/isi_restore/tape.c:2396 unknown tape header type -535703600

LogMessage {type NDMP_LOG_NORMAL} {msg_id 0} {msg DUMP: Network communication error

} {associated_msg 0} {associated_msg_seq 0}

LogMessage {type NDMP_LOG_NORMAL} {msg_id 0} {msg DUMP: DUMP IS ABORTED

} {associated_msg 0} {associated_msg_seq 0}

LogMessage {type NDMP_LOG_NORMAL} {msg_id 0} {msg Connection or IO Error.

} {associated_msg 0} {associated_msg_seq 0}

nasc: Connection halted: HALT: Connect error!

The transfer aborted with error.

isi_vol_copy did not run properly

isilon1-1#

Incremental copy failure, it also causes the /var parition to go 100% full, copy fails, kill the isi_vol_copy process (stuck) to regain space in /var.

Resolution: will be fixed in future onefs version.

Error while looking up ACL inode 1384703, err:No such file or directory

db->get failed, err:No such file or directory

acl_db_get failed for aclino:1384703

Error while looking up ACL inode 1384703, err:No such file or directory

db->get failed, err:No such file or directory

acl_db_get failed for aclino:1384703

Error while looking up ACL inode 1384703, err:No such file or directory

db->get failed, err:No such file or directory

acl_db_get failed for aclino:1384703

Error while looking up ACL inode 1384703, err:No such file or directory

db->get failed, err:No such file or directory

acl_db_get failed for aclino:1384703

Error while looking up ACL inode 1384703, err:No such file or directory

db->get failed, err:No such file or directory

thanks

-H

Tags (2)
0 Kudos
1 Solution

Accepted Solutions
Vanitha13
6 Indium

Re: isi_vol_copy issues

Jump to solution

Hi,

What type of files are you migrating, is it CIFS or NFS ?

I was not able to find anything with the error messages that are pasted, but came across 2 white paper which may be helpfull  :

NFS FILE MIGRATION TO AN EMC ISILON CLUSTER ( Guidance for optimal data migration of NFS workflows )

SMB FILE MIGRATION TO EMC ISILON (Guidance for optimal data migration of SMB workflows)

let me know if this helps

Thanks

Vanitha

0 Kudos
7 Replies
Vanitha13
6 Indium

Re: isi_vol_copy issues

Jump to solution

Hi,

What type of files are you migrating, is it CIFS or NFS ?

I was not able to find anything with the error messages that are pasted, but came across 2 white paper which may be helpfull  :

NFS FILE MIGRATION TO AN EMC ISILON CLUSTER ( Guidance for optimal data migration of NFS workflows )

SMB FILE MIGRATION TO EMC ISILON (Guidance for optimal data migration of SMB workflows)

let me know if this helps

Thanks

Vanitha

0 Kudos
8 Krypton

Re: isi_vol_copy issues

Jump to solution

Isi_vol_copy can be used to migrate any type of data irrespective of SMB or NFS. The volumes that I need to migrate are CIFS only type. There is no primus available for these errors and if you open any isi_vol_copy SR with support they have only one solution and it will not resolve the issue. I will have to use third party migration tool for this.

This is the only solution from support.

# isi_vol_copy –sa ndmp:password –cleanup <ntap-filer1-ip>

Vanitha13
6 Indium

Re: isi_vol_copy issues

Jump to solution

Hi,

Thanks for sharing

Regards,

Vanitha

0 Kudos
8 Krypton

Re: isi_vol_copy issues

Jump to solution

Were you able to overcome the time stamp issues

0 Kudos
skonde
1 Nickel

Re: isi_vol_copy issues

Jump to solution

Adding to this, I'm not the original poster but I'm encountering this issue running isi_vol_copy_vnx.

isi_vol_copy_vnx sets the created time stamp for December 31st 1969.  OneFS version is 7.1.0.3.

The fix we were told by EMC support was to run robocopy with /timfix /copy:d /dcopy:t.

securityjm
6 Indium

Re: isi_vol_copy issues

Jump to solution

I was getting error "FAILED ASSERTION m @ /b/mnt/src/sys/kern/isi_format.c:134" which then caused many errors like below.


db->get failed, err:No such file or directory

acl_db_get failed for aclino:1384703


The issue also caused isi_vol_copy to stop, requiring a restart many times.


After working with engineering for quite some time, the issue has been fixed in 7.2.1.3, however I didn’t not see any mention of it in the release notes, but i can confirm the issue is resolved completely.


As a unrelated note, I would like to mention that whenever the source Netapp did a autogrow on the source volume, I needed to delete the contents on the target directory and redo a isi_vol_copy full.  I suspect this is because the inode mapping was changed.  I have not seen this listed as a limitation, but it proved to be 100% consistent as a problem/solution for me.

0 Kudos
alan_oleski
6 Indium

Re: isi_vol_copy issues

Jump to solution

To not fill up the /var partition when running multiple, large isi_vol_copy jobs, set isi_vol_copy to write its log under the /ifs partition.

Run this command on each node; do not use isi_for_array.

# isi_ilog -a isi_vol_copy -f /ifs/ivc_logs/`uname -a | awk '{print $3}'`

# isi_ilog -a restore -f /ifs/ivc_logs/`uname -a | awk '{print $3}'`

0 Kudos