NetWorker: How to troubleshoot failing email notifications
Summary: This KB provides an overview of how to troubleshoot email notification-related issues. Emails are configured in NetWorker resources, but emails are not being received after triggered events, such as backup completion. This issue can appear during an initial email configuration or on an existing configuration. This KB focuses on various different symptoms that may be observed during a communication issue between the NetWorker server and SMTP mail server. ...
Symptoms
Policy, Device, or client owner's notifications are not sent.
The command in the action field of the notification field is correct.
See the nsr_notification section of the NetWorker Command Reference Guide: https://www.dell.com/support/product-details/product/networker/overview
Configuring NetWorker Policy notifications is covered in the following KB: NetWorker: How To Configure Policy Email Notifications
The workflow may include an error message like:
connect to SMTP host failed: No such file or directory
Attempts to send an email from the command line may report an issue:
Windows:
C:\Users\Administrator.NETWORKER>echo "hello world" | smtpmail -h mailhost -s "test smtpmail" bkupadmin@dell.com 1 3 1088516 2 0 0 11112 8980 0 win-srvr01.networker.lan smtpmail NSR warning 11 %s: %s (%d) 3 0 48 communication failure receiving from SMTP server 0 63 An existing connection was forcibly closed by the remote host. C:\Users\Administrator.NETWORKER>echo "hello world" | smtpmail -h mailhost -s "test smtpmail" -p 587 bkupadmin@dell.com connect to SMTP host failed: Unknown error
Linux:
Jan 23 15:17:26 nsr postfix/smtp[206391]: 631E440C02AC: to=<RECIPIENT_EMAIL>, relay=none, delay=6351, delays=6291/0.1/60/0, dsn=4.4.1, status=deferred (connect to MAIL_SERVER_NAME[MAIL_SERVER_IP]:SMTP_PORT: Connection timed out) Jan 23 15:21:26 nsr postfix/error[206631]: 70A8240C02A0: to=<RECIPIENT_EMAIL>, relay=none, delay=15211, delays=15211/0.02/0/0.04, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to MAIL_SERVER_NAME[MAIL_SERVER_IP]:SMTP_PORT: Connection timed out)
Symptoms are not limited to these examples, additional errors may be observed. The key theme is that some communication-related error is observed.
Cause
NetWorker uses operating system mail commands: mailx (Linux) and smtpmail (Windows) to send standard output vial email. There is little NetWorker functionality involved in sending email messages other than indicating what conditions should be met in order to send an email, For example: Backup completes or fails, a device is full, and so forth.
This cause of emails not being sent or received could be due to lack of communication between the backup server and mail server.
Resolution
- Identify that you are using the correct mail server for your domain by running the following command from your NetWorker server (replacing domain.com with your domain name):
Linux:
dig +short domain.com MX
Windows:
nslookup set q=mx domain.com
- Ensure that the NetWorker server can correctly resolve the mail server address.
nslookup FULLY_QUALIFIED_DOMAIN_NAME nslookup SHORT_NAME nslookup IP_ADDRESS
- Check the NetWorker server's "hosts" file. If the mail server is entered in the hosts file, ensure that the correct IP address is specified.
Windows: C:\Windows\System32\drivers\etc\hosts
- Ensure that you can connect to the mail server's SMTP port (25) from the NetWorker server:
nsrports -t MAIL_SERVER_ADDRESS -p 25
- Linux Servers Only:
A. On Linux systems, additional services such as postfix or sendmail may be installed to relay the messages from the server to the mail server.
rpm -qa | grep "postfix\|sendmail"
systemctl enable postfix ; systemctl start postfix systemctl enable sendmail ; systemctl start sendmail
systemctl status postfix systemctl status sendmail
- From an elevated prompt on the NetWorker server, test the mail commands.
echo "this is a test email" | mail [-S smtp=smtp://your.mailserver.domain] -s "Test Email" recipient@example.domain
echo "this is a test email" | smtpmail [-h your.mailserver.domain] -s "Test Email" recipient@example.domain
Logs:
On Linux NetWorker servers, review the following log for insight on mail issues: /var/log/maillog
General mail log file if a message was sent and accepted for delivery. /var/spool/mqueue
Messages stuck in the mqueue are not being sent or accepted from the server. If email messages are being sent and delivered, the mqueue should be empty.
If the NetWorker server is Windows, the Exchange admin must review logs from the exchange side.
Conclusion:
Regardless of NetWorker server errors, the domain or mail admin must check the mail server logs for specific notification errors. Once the mail commands are successful from the NetWorker server's command line, enter your operating system mail command and options in the wanted notification field.