Unsolved
This post is more than 5 years old
7 Posts
0
2068
VNX7600 NAS file system splitting methodology on same vdm
Hi Everyone,
We are using vnx7600 unified box in our environment. We have NAS vdm's configured and backups are configured as NDMP on EMC data domain. Now we have two different problem, one of nas file system has grown up to 10tb and contains billions of logs files into it.
1) Backup are running very slow since since we expand the celerra pool with 2tb of lun and expand the file system with 200gb. Storage pool has 5tb space left and around 95% utilized. Backup has never got completed after that and we are not sure what caused this. The only change we made is as above I mentioned.
2) We are planning to split this fs on directory level. Plan is to create a new filesystem say live1 on same vdm and due to application dependency we need to migrate the sub-directory from older fs live to live1 and accordingly network team change will change the rules and windows team change the permission. Is there any migration approach we can follow to perform on storage end to get this done successful.
Eg. live/test/folder1/folder2 to live1/test/folder1/folder2
Any suggestion would be highly appreciated.
Rainer_EMC
8.6K Posts
0
December 15th, 2016 08:00
copying at sub-fs granularity will have to be file-level
either from the client using emcopy/rsync/... or maybe ndmpcopy
Did you investigate whether it is possible to split up your backups instead to just backup a subset of directories ?
most backup Software can also backup at a directory level below the complete file system.
If only parts of your file system contain lots of very small files you could investigate putting just these directory trees into a separate file system
dynamox
1 Rookie
1 Rookie
•
20.4K Posts
0
December 15th, 2016 13:00
Rainer, don't forget about good ol' friend server_archive
panks2020
6 Posts
0
December 16th, 2016 01:00
Hi Rainer,
Thanks for your response!
Could you please provide the link to download ndmp copy tool.
we are using NBU 7.7.2 and so far what we have analysed that out of 10tb fs only 5tb data which is being used by application generated logs and the count is in billions. So we have split the backup into 3-4 policies along with some exclusions and backups are happening as a workaround.
If only parts of your file system contain lots of very small files you could investigate putting just these directory trees into a separate file system
yeah, the rest of 5tb fs has billions of file. but the thing here is we have everything on root /live directory tree and then presented cifs share on top level. Is it possible to split the fs with same directory structure on same vdm.
Rainer_EMC
8.6K Posts
0
December 16th, 2016 04:00
if it is different subdirectories - yes - see NMFS
or work with symbolic links
Rainer_EMC
8.6K Posts
0
December 16th, 2016 04:00
ndmpcopy should be on support.emc.com
or via google - it is not EMC specific
Rainer_EMC
8.6K Posts
0
December 16th, 2016 04:00
Hi dynamox,
I didnt forget it - just thought there are simpler commands if you dont know the system very well
panks2020
6 Posts
0
December 16th, 2016 05:00
Rainer,
Could you please further guide me if we use symbolic link for this.
Rainer_EMC
8.6K Posts
1
December 16th, 2016 07:00
mount the VDM root to a Unix Client an use a ln -s
or use server_export NFS alias functionality which also creates a symlink
or looks at creating a DFS standalone root and use DFS path to access
panks2020
6 Posts
0
January 5th, 2017 06:00
Hi Rainer,
We are trying test ndmpcopy on one of our fs and we are getting below error highlighted in bold. Could you help what actually I am missing here.
where 10.215.72.16 is cifs_ndmp interconnect.
[root@stcschsbkp0225 NDMPcopy-1.2_Linux]# ./ndmpcopy 10.215.72.16:/root_vdm_3/CHSNAS01_fs/USB_TEAM/ndmpcopy 10.215.72.16:/root_vdm_3/CHSNAS01_fs1/USB_TEAM1 -sa testndmp:testndmp -da testndmp:testndmp -sport 10000 -dport 10000 -level 0 -v -v -v -v
Creating NDMP connection.
Connecting to 10.215.72.16.
Authorizing connection with TEXT authorization.
user = testndmp, password = testndmp.
Authenticated root with NDMP_AUTH_TEXT.
Creating NDMP connection.
Connecting to 10.215.72.16.
Authorizing connection with TEXT authorization.
user = testndmp, password = testndmp.
Authenticated root with NDMP_AUTH_TEXT.
Sending MOVER_LISTEN message.
DUMP: Sending backup request, mover 0xad74810, port 58581.
Environment:
UPDATE = y
TYPE = dump
USER = root
HIST = n
FILESYSTEM = /root_vdm_3/CHSNAS01_fs/USB_TEAM/ndmpcopy/
LEVEL = 0
EXTRACT = n
DUMP: Done! returning.
RESTORE: Sending recover request.
RESTORE: Done! returning.
Main in infinite loop...
10.215.72.16: Received notify_connect request: version: 4,
10.215.72.16: CONNECT: Connection established.
10.215.72.16: Received notify_connect request: version: 4,
10.215.72.16: CONNECT: Connection established.
10.215.72.16: LOG: SnapSure file system creation fails10.215.72.16: Connection halted: HALT: The operation was aborted!
Elapsed time: 0 hours, 0 minutes, 4 seconds.
Also support updated me below:
I don’t think this is properly configured. There’s no Snapsure value set for the ndmpcopy job. The Snapsure creation failing immediately following the backup directory not found suggests that the destination directory is not accessible or properly configured.
Let me know if you would like me to reach out to the local account team and get you a resource to help with this.
2017-01-04 10:34:58: 26041581568: NDMP: 6: NDMP gets listening port 59706, hport 15081
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 370 (thread ndmp370) < Backup type: dump >
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 370 (thread ndmp370) Name: UPDATE Value: y
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 370 (thread ndmp370) Name: TYPE Value: dump
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 370 (thread ndmp370) Name: USER Value: root
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 370 (thread ndmp370) Name: HIST Value: n
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 370 (thread ndmp370) Name: FILESYSTEM Value: /CHSNAS01_fs/USB_TEAM/ndmpcopy
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 370 (thread ndmp370) Name: LEVEL Value: 0
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 370 (thread ndmp370) Name: EXTRACT Value: n
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 371 (thread ndmp371) Name: UPDATE Value: y
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 371 (thread ndmp371) Name: TYPE Value: dump
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 371 (thread ndmp371) Name: USER Value: root
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 371 (thread ndmp371) Name: HIST Value: n
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 371 (thread ndmp371) Name: FILESYSTEM Value: /CHSNAS01_fs/USB_TEAM/ndmpcopy
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 371 (thread ndmp371) Name: LEVEL Value: 0
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 371 (thread ndmp371) Name: EXTRACT Value: n
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 371 (thread ndmp371) < Backup type: dump >
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 371 (thread ndmp371) tape service is on a remote side, CALLER_ADDRESS 0x0001bea963.
2017-01-04 10:34:58: 17451646976: NDMP: 4: Thread ndmp371 uses a set of Pax threads with handle 3.
2017-01-04 10:34:58: 26041581568: NDMP: 6: Thread ndmp371 filterDialect could not be set - prereqs not met [pax_cmd_thread::setFilterDialect]
2017-01-04 10:34:58: 13161529344: CMD: 3: backupDir (/CHSNAS01_fs/USB_TEAM/ndmpcopy) error: NotFound
2017-01-04 10:34:58: 17451646976: NDMP: 4: Session 370 (thread ndmp370) NdmpdData::startBackup, SnapSure creation for backup failed
2017-01-04 10:34:58: 17451646976: NDMP: 4: Thread nasa03 Read failed on archive volume 1
2017-01-04 10:34:58: 150596943877: PAX: 3: Thread nasa03 fails in buf_fill, fileName=../buf_subs.cxx, lineNo=629.
2017-01-04 10:34:58: 13156679680: NDMP: 3: Thread nasa03 Sorry, unable to determine archive format.
2017-01-04 10:34:58: 150596943873: PAX: 3: Thread nasa03 fails in get_arc (-1), options(0) or st_rd(0), isNDMPArch=0.
Could you suggest what I am missing here.
Thanks