Unsolved
This post is more than 5 years old
12 Posts
0
3022
Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
Hi there,
Have you experienced a problem during a full NDMP backp with Celerra? I have the following environment:
NS-120FC 6.0.41.-4
TSM
Tape Driver HP
Differential backup works fine
We have 1TB free space on Pool and 5.6TB allocated with 4.6TB used.
After increasing free pool space to 1TB, the first full backup worked, but no more.
There are no checkpoints, everyone has already cleaned up.
Do you have any idea? How is the recomendable free space in pool (now is 20% from total File System size)?
regards,
Daniel
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
June 25th, 2014 09:00
what is the error message in "q actlog" ?
Daniel_Borba
12 Posts
0
June 25th, 2014 10:00
Hi dynamox,
How can I see this log?
I have the output from TSM log:
(c96t0l0). (SESSION: 83286, PROCESS: 213)
06/11/2014 14:48:13 ANR1340I Scratch volume VD2215L4 is now defined in storage
pool STG_NAS. (SESSION: 83286, PROCESS: 213)
06/11/2014 14:48:13 ANR0513I Process 213 opened output volume VD2215L4.
(SESSION: 83286, PROCESS: 213)
06/11/2014 15:17:10 ANR9999D_1212503635 (ndlog.c:120) Thread<48>: NDMPLOG: E
Backup is aborted.(SESSION: 83286, PROCESS: 213)
06/11/2014 15:17:10 ANR9999D Thread<48> issued message 9999 from: (SESSION:
83286, PROCESS: 213)
06/11/2014 15:17:11 ANR0515I Process 213 closed volume VD2215L4. (SESSION:
83286, PROCESS: 213)
06/11/2014 15:17:11 ANR8336I Verifying label of NAS volume VD2215L4 in drive
DRIVE01 (c96t0l0). (SESSION: 83286, PROCESS: 213)
06/11/2014 15:19:16 ANR8468I NAS volume VD2215L4 dismounted from drive DRIVE01
(c96t0l0) in library MSL4048. (SESSION: 83286, PROCESS:
213)
06/11/2014 15:19:16 ANR1068W NAS Backup process 213 terminated - process
cancelled. (SESSION: 83286, PROCESS: 213)
regards,
T-SYSTEMS DO BRASIL LTDA.
GCU Volkswagen - Storage
Daniel Segabinazzi Borba
Rodovia Anchieta Km 23,5 – ALA22
09823-901 - Demarchi – São Bernardo do Campo, SP
+ 55 11 2596-7324 (fone)
E-mail: daniel.borba@t-systems.com.br
Internet: www.t-systems.com.br
BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.
Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any dissemination or copying is strictly prohibited.
If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you.
De: dynamox
Enviada em: quarta-feira, 25 de junho de 2014 13:16
Para: Daniel Segabinazzi Borba
Assunto: Re: - Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
ECN
Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
reply from dynamox in Celerra Support Forum - View the full discussion
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
June 25th, 2014 11:00
"q actlog" is how you look at TSM logs. What do you see on the datamover side when this error occurs ? (server_log server_2)
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
June 25th, 2014 11:00
i don't remember if there a way to look at older logs on Celerra.
Daniel_Borba
12 Posts
0
June 25th, 2014 11:00
I’ll have to simulate the error again because this log is only from 3 days ago…
Is there another log that archives at least 2x weeks?
T-SYSTEMS DO BRASIL LTDA.
GCU Volkswagen - Storage
Daniel Segabinazzi Borba
Rodovia Anchieta Km 23,5 – ALA22
09823-901 - Demarchi – São Bernardo do Campo, SP
+ 55 11 2596-7324 (fone)
E-mail: daniel.borba@t-systems.com.br
Internet: www.t-systems.com.br
BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.
Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any dissemination or copying is strictly prohibited.
If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you.
De: dynamox
Enviada em: quarta-feira, 25 de junho de 2014 15:22
Para: Daniel Segabinazzi Borba
Assunto: Re: - Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
ECN
Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
reply from dynamox in Celerra Support Forum - View the full discussion
Daniel_Borba
12 Posts
0
June 26th, 2014 04:00
We started a full backup yestarday night:
2014-06-26 00:01:12: NDMP: 6: Active NDMP backup/restore streams: 1, system configured concurrent streams: 4, maximum concurrent sessions supported: 4.
2014-06-26 00:01:13: NDMP: 4: Session 954 (thread ndmp954) < Backup type: dump >
Since there I’ve noticed:
2014-06-26 00:01:27: NDMP: 6: Thread ndmp954 filterDialect could not be set - prereqs not met
A lot of:
2014-06-26 00:01:31: SMB: 6: logoff of user EMCCAVA_CUR$ failed on server vwbrcusfdfs02 with client emccava_cur status:c000020c
2014-06-26 00:01:41: SMB: 4: Warning: select stream timeout No error reported on server vwbrcusfdfs02 (10.134.99.49) port 445 with client emccava_cur (10.134.99.36)
2014-06-26 00:01:41: CEPP: 3: 9: Error on CEPP server 10.134.99.49 of pool cqm: MSRPC ERROR, ntStatus: CONNECTION_DISCONNECTED.
2014-06-26 00:01:50: CEPP: 3: 9: Error on CEPP server 10.134.99.49 of pool cqm: MSRPC ERROR, ntStatus: OBJECT_NAME_NOT_FOUND.
2014-06-26 00:02:11: SMB: 4: last message repeated 1 times
2014-06-26 00:02:11: CEPP: 3: 9: Error on CEPP server 10.134.99.49 of pool cqm: MSRPC ERROR, ntStatus: CONNECTION_DISCONNECTED.
2014-06-26 00:02:50: SMB: 4: last message repeated 2 times
2014-06-26 00:02:50: CEPP: 3: 9: Error on CEPP server 10.134.99.49 of pool cqm: MSRPC ERROR, ntStatus: OBJECT_NAME_NOT_FOUND.
2014-06-26 00:03:01: SMB: 4: last message repeated 1 times
2014-06-26 00:03:11: SMB: 6: disconnect of share IPC$ failed on server vwbrcusfdfs02 with client emccava_cur status:c000020c
2014-06-26 00:03:21: SMB: 4: Warning: select stream timeout No error reported on server vwbrcusfdfs02 (10.134.99.49) port 445 with client emccava_cur (10.134.99.36)
Follow attached log file.
Regards,
T-SYSTEMS DO BRASIL LTDA.
GCU Volkswagen - Storage
Daniel Segabinazzi Borba
Rodovia Anchieta Km 23,5 – ALA22
09823-901 - Demarchi – São Bernardo do Campo, SP
+ 55 11 2596-7324 (fone)
E-mail: daniel.borba@t-systems.com.br
Internet: www.t-systems.com.br
BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.
Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any dissemination or copying is strictly prohibited.
If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you.
De: dynamox
Enviada em: quarta-feira, 25 de junho de 2014 15:51
Para: Daniel Segabinazzi Borba
Assunto: Re: - Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
ECN
Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
reply from dynamox in Celerra Support Forum - View the full discussion
1 Attachment
Celerra_error_log_since_June_26th_midnight.txt
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
June 26th, 2014 06:00
can you post output from this command:
server_param server_2 -facility PAX -list
Daniel_Borba
12 Posts
0
June 26th, 2014 07:00
Sure. Follow below:
T-SYSTEMS DO BRASIL LTDA.
GCU Volkswagen - Storage
Daniel Segabinazzi Borba
Rodovia Anchieta Km 23,5 – ALA22
09823-901 - Demarchi – São Bernardo do Campo, SP
+ 55 11 2596-7324 (fone)
E-mail: daniel.borba@t-systems.com.br
Internet: www.t-systems.com.br
BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.
Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any dissemination or copying is strictly prohibited.
If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you.
De: dynamox
Enviada em: quinta-feira, 26 de junho de 2014 10:11
Para: Daniel Segabinazzi Borba
Assunto: Re: - Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
ECN
Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
reply from dynamox in Celerra Support Forum - View the full discussion
1 Attachment
image001.png
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
June 26th, 2014 07:00
that's what i have too, hmm. Can you manually create SnapSure snapshot of the file system you are trying to backup ?
Daniel_Borba
12 Posts
0
June 26th, 2014 08:00
I think there is no problem to create a snapshot manually...
Do you know how to do it by cli?
T-SYSTEMS DO BRASIL LTDA.
GCU Volkswagen - Storage
Daniel Segabinazzi Borba
Rodovia Anchieta Km 23,5 – ALA22
09823-901 - Demarchi – São Bernardo do Campo, SP
+ 55 11 2596-7324 (fone)
E-mail: daniel.borba@t-systems.com.br
Internet: www.t-systems.com.br
BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.
Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any dissemination or copying is strictly prohibited.
If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you.
De: dynamox
Enviada em: quinta-feira, 26 de junho de 2014 11:42
Para: Daniel Segabinazzi Borba
Assunto: Re: - Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
ECN
Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
reply from dynamox in Celerra Support Forum - View the full discussion
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
June 26th, 2014 18:00
fs_ckpt -Create
Daniel_Borba
12 Posts
0
June 27th, 2014 05:00
Dynamox,
I created a manual snapshot (below) and started full bkp on 26th 16hs.
The bkp executed up to 26th 23:59 and was aborted:
2014-06-26 23:59:13: NDMP: 4: Thread bkup116 Backup was aborted
2014-06-26 23:59:13: NDMP: 4: < LOG type: 3, msg_id: 0, entry: Backup was aborted, hasAssociatedMsg: 0, associatedMsgSeq: 0 >
2014-06-26 23:59:13: NDMP: 4: Thread bkup077 Abort is called during readdir (dp=0x0071e3c208) for NDMP DUMP format file history
2014-06-26 23:59:13: NDMP: 4: Thread bkup112 Abort is called during readdir (dp=0x00ba9ab208) for NDMP DUMP format file history
Apparently error message changed at TSM Server and indicates that server log achieved 95% utilization.
06/26/2014 23:59:17 ANR2997W The server log is 95 percent full. The server
will delay transactions by 300 milliseconds. (SESSION:
16140, PROCESS: 93)
By the way,
Do you think could have old snapshots to remove? Or is there a problem with CEPP Server?
Regards,
T-SYSTEMS DO BRASIL LTDA.
GCU Volkswagen - Storage
Daniel Segabinazzi Borba
Rodovia Anchieta Km 23,5 – ALA22
09823-901 - Demarchi – São Bernardo do Campo, SP
+ 55 11 2596-7324 (fone)
E-mail: daniel.borba@t-systems.com.br
Internet: www.t-systems.com.br
BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.
Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any dissemination or copying is strictly prohibited.
If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you.
De: dynamox
Enviada em: quinta-feira, 26 de junho de 2014 22:31
Para: Daniel Segabinazzi Borba
Assunto: Re: - Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
ECN
Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
reply from dynamox in Celerra Support Forum - View the full discussion
3 Attachments
image002.png
image001.png
image003.png
Daniel_Borba
12 Posts
0
June 27th, 2014 06:00
TSM 5.5
We have only this production FS. The other ones are for restore area, home.
We are trying to segment in FS with 2TB but it will take time to acquire disks.
T-SYSTEMS DO BRASIL LTDA.
GCU Volkswagen - Storage
Daniel Segabinazzi Borba
Rodovia Anchieta Km 23,5 – ALA22
09823-901 - Demarchi – São Bernardo do Campo, SP
+ 55 11 2596-7324 (fone)
E-mail: daniel.borba@t-systems.com.br
Internet: www.t-systems.com.br
BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.
Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any dissemination or copying is strictly prohibited.
If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you.
De: dynamox
Enviada em: sexta-feira, 27 de junho de 2014 10:01
Para: Daniel Segabinazzi Borba
Assunto: Re: - Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
ECN
Celerra (NS-120FC 6.0.41.-4) - NDMP full backup problem
reply from dynamox in Celerra Support Forum - View the full discussion
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
June 27th, 2014 06:00
What version of TSM are you on, still on 5.x ? Can you try to backup a much smaller file system ?
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
June 27th, 2014 07:00
create a new , small file system and try to back it up. This will tell you if it's TSM related (high log utilization). I do not know what the file system structure looks like but within TSM you can break it up into multiple "virtual mount points" and back them up individually.