looking for info on using avtar and debugging logfile errors for avamar HPUX
client.
I am restoring 180 gigs of filesystem data from avamar to hpux filesystem. I am using avtar xvf. It dies with a "FATAL ERROR" after a lot of Internal Warning: Very large amount of pending data in filepipe (134236471 bytes).
Except for avtar --help, I have yet to find any documention for avtar options within any of the EMC sites (or anywhere on the web via google).
I have not found any documention on the meaning behind the error in the log file.
fyi: avtar runs from /opt/AVMRCLNT/bin/avtar.
so my two questions: 1. any documentation? 2. why did it die? (Ah... why I want the documentation).
With regard to messages in the avtar logs, there is an event code catalog on the Avamar server itself and it can be found in several formats in the /usr/local/avamar/doc directory on the utility node. After having a quick look through it, however, there is not really enough information about most client events to be useful. You will likely find that the description for any kind of internal error or warning instructs you to contact support.
Besides the --help text, the only public documentation for avtar that I'm aware of is found in the Backup Clients User Guide:
Best practice for running backups and restores is to use the Avamar Administrator GUI or the mccli to launch them. Running avtar manually is generally not recommended since these raw avtar jobs (also called "naked ad-hoc" or NAH backups) do not report status or progress to the Avamar Administrator so they can't be monitored or managed through the GUI or mccli. If you prefer command line over GUI interfaces, there is a full guide for the mccli here:
If I run the restore via avtar is starts restoring 10 files at a time (each file is 17.5 gigs). If I run the restore via avamar gui console on windows, it starts restoring 1 file at a time (each file is 17.5 gigs).
The thing I am restoring. about 200gigs, 38 files.
Can you post the complete log? You should strip out sensitive information (hostnames, IP addresses, etc.) before posting it.
Without seeing the exact messages, there is no way to determine what went wrong. If you're running a command line restore and want to record the log information to a file, specify the log filename by adding the following flag to the avtar command line call:
2013-08-05 09:24:08 avtar Info <5241>: Logging to /var/avamar/fromprod_tmp.log 2013-08-05 09:24:08 avtar Info <5551>: Command Line:
///opt/AVMRclnt//bin/avtar.bin
--vardir=/opt/AVMRclnt/var
--bindir=/opt/AVMRclnt/bin
--sysdir=/opt/AVMRclnt/etc
--extract --server=WestCoast /diska/backups
--id=myid@/REPLICATE/EastCoast/grp-HPUX
--password=***********
--account=/REPLICATE/EastCoast/grp-HPUX/prodbox
--after="2013-08-04 16:00 "
--logfile=/var/avamar/fromprod_tmp.log 2013-08-05 09:24:08 avtar Info <5552>: Connecting to Server (EastCoast) 2013-08-05 09:24:08 avtar Info <5554>: - Connecting to one node in each datacenter 2013-08-05 09:24:08 avtar Info <5993>: - Connect: Connected to WestCoast:#####, Priv=0, TCP 2013-08-05 09:24:08 avtar Info <5693>: - Local client time and Server time are significantly different (delta=628.1792 secs) 2013-08-05 09:24:08 avtar Info <5559>: - Trying connection xxxxxx:##### 2013-08-05 09:24:08 avtar Info <5560>: - Connected to node xxxxxx:#####, Priv=0, TCP 2013-08-05 09:24:08 avtar Info <5693>: - Local client time and Server time are significantly different (delta=628.1838 secs) 2013-08-05 09:24:08 avtar Info <5583>: Login User: "myid", Domain: "default", Account: "/REPLICATE/EastCoast/grp-HPUX/prodbox" 2013-08-05 09:24:08 avtar Info <5580>: - Logging in on connection 0 2013-08-05 09:24:08 avtar Info <5582>: - Login successful 2013-08-05 09:24:08 avtar Info <5018>: - Session ID: 1375712648679855 2013-08-05 09:24:08 avtar Info <6509>: Successfully logged into Server 2013-08-05 09:24:08 avtar Info <5295>: Starting restore at 2013-08-05 09:24:08 CDT as "root" on "drbox" (4 CPUs) [5.0.105-169] 2013-08-05 09:24:08 avtar Info <7651>: Selecting backups after "2013-08-04 16:00:00 CDT" 2013-08-05 09:24:08 avtar Info <5949>: Backup file system character encoding is UTF-8. 2013-08-05 09:24:08 avtar Info <8695>: Backup from HP-UX host "/REPLICATE/EastCoast/grp-HPUX/prodbox" (prodbox) with plugin 4001 - HP-UX Filesystem 2013-08-05 09:24:08 avtar Info <5538>: Backup #608 label "grp-HPUX-grp-HPUX-1375668000065" timestamp 2013-08-04 22:35:02 CDT, 204,387 files, 228.1 GB 2013-08-05 09:24:08 avtar Info <5291>: Estimated size for "/diska/backups" is 193.9 GB 2013-08-05 09:24:08 avtar Info <5260>: Restoring files from "/diska/backups" to directory "/diska/backups". 2013-08-05 09:39:08 avtar Info <5100>: Status 2013-08-05 09:39:08, 38 files, 31.83 GB (38 files, 9.794 GB, 30.77% new) 162MB 67% CPU datafile.08041320232.bck7 2013-08-05 09:54:08 avtar Info <5100>: Status 2013-08-05 09:54:08, 38 files, 64.25 GB (38 files, 19.72 GB, 30.70% new) 311MB 68% CPU datafile.08041320232.bck7 2013-08-05 09:58:22 avtar Warning <7492>: Internal Warning: Very large amount of pending data in filepipe (134224872 bytes) 2013-08-05 09:58:48 avtar Warning <7492>: Internal Warning: Very large amount of pending data in filepipe (134236542 bytes) 2013-08-05 09:58:54 avtar Warning <7492>: Internal Warning: Very large amount of pending data in filepipe (134236471 bytes) 2013-08-05 09:59:07 avtar FATAL <5889>: Fatal signal 11 in pid 4 2013/08/05-14:59:07.40902 [ms[0]recv] FATAL ERROR: <0001> Fatal signal 11 2013/08/05-14:59:07.41002 [ms[0]recv] ERROR: <0324> handlefatal aborting program pid=4, sig=11 2013-08-05 09:59:07 avtar FATAL <5890>: handlefatal: Aborting program with code 176, pid=4, sig=11
There's not much information here about what might be going wrong other than the client not being able to flush restored data to disk which can happen for a number of different reasons. I would recommend re-running this test with the --debug flag added to the command line, opening a service request and attaching the log file to the SR.
Is there any documentation to explain the error? I have support, and there is nothing wrong with it. But it would be a pain for me to be allowed use it.
The messages in the log are messages about symptoms, not causes. It's not clear what is causing the problem, only that data stops being written to disk (the internal warning), then the client crashes (fatal signal 11).
Let me repeat myself. I am looking for documentation on the errors within the log file and the syntax of running avtar so I can solve my own problems. Making a beeline to solving this specific problem is nice but what about the next 50 problems? The "--help" option on the avtar command is a bit sparse.
ionthegeek
2 Intern
•
2049 Posts
2509
1
Posted August 7th, 2013 06:00
With regard to messages in the avtar logs, there is an event code catalog on the Avamar server itself and it can be found in several formats in the /usr/local/avamar/doc directory on the utility node. After having a quick look through it, however, there is not really enough information about most client events to be useful. You will likely find that the description for any kind of internal error or warning instructs you to contact support.
Besides the --help text, the only public documentation for avtar that I'm aware of is found in the Backup Clients User Guide:
https://support.emc.com/docu39181_Avamar-6.1-Backup-Clients-User-Guide.pdf?language=en_US
Best practice for running backups and restores is to use the Avamar Administrator GUI or the mccli to launch them. Running avtar manually is generally not recommended since these raw avtar jobs (also called "naked ad-hoc" or NAH backups) do not report status or progress to the Avamar Administrator so they can't be monitored or managed through the GUI or mccli. If you prefer command line over GUI interfaces, there is a full guide for the mccli here:
https://support.emc.com/docu39179_Avamar-6.1-Management-Console-Command-Line-Interface-(MCCLI)-Programmer-Guide-.pdf?language=en_US
I wish I had something better to give you but I hope this is helpful anyway.