Hi willir9.1

Just a shot from left field; do all of the nodes have network connectivity and are able to reach the syslog server?  I have seen problems when not all nodes are network attached... (NANON).

Cheers,

Matt

In order to add a construct like the one below, you will need to edit your custom syslog file by hand.

!lsass

cifs.*                                          @10.1.10.20

The custom syslog config file is located in /etc/mcp/override/syslog.conf.  You edit this file on any node in the cluster and the changes will be updated automatically on all nodes.

Once you've made this change, you can no longer use the 'isi_log_server' utility as it doesn't understand this construct.

I've had to use this in order to configure certain auditing events to reach an external syslog server:

yfvm-7111-2# cat /etc/mcp/override/syslog.conf
!audit_protocol
*.* @192.168.5.10

This is the syntax I use in order for the message to appear in /var/log/lsassd.log:

logger -i -t lsass -p cifs.err "cifs_err_test_A `date`"

And results in this entry in /var/log/lsassd.log:

2014-11-10T13:27:42-05:00 <30.3> yfvm-7111-2(id2) lsass[37482]: cifs_err_test_A Mon Nov 10 13:27:41 EST 2014

However it's not showing up in my syslog remote server but I do see the message being sent when I run tcpdump so it's a config issue on my remote syslog server.  tcpdump output from my remote syslog server:

13:27:42.013914 IP 192.168.32.222.57835 > yfvm-deb1.yf-isilab.local.syslog: SYSLOG unknown (30).error, length: 89
        0x0000:  4500 0075 29e2 0000 3f11 aa5d c0a8 20de  E..u)...?..]....
        0x0010:  c0a8 050a e1eb 0202 0061 df21 3c32 3433  .........a.!<243
        0x0020:  3e32 3031 342d 3131 2d31 3054 3133 3a32  >2014-11-10T13:2
        0x0030:  373a 3432 2d30 353a 3030 206c 7361 7373  7:42-05:00.lsass
        0x0040:  5b33 3734 3832 5d3a 2063 6966 735f 6572  [37482]:.cifs_er
        0x0050:  725f 7465 7374 5f41 204d 6f6e 204e 6f76  r_test_A.Mon.Nov
        0x0060:  2031 3020 3133 3a32 373a 3431 2045 5354  .10.13:27:41.EST
        0x0070:  2032 3031 34                             .2014

Interesting, and thanks for the tip. I edited mine and it propagated out to the other nodes just fine.

cat /etc/mcp/override/syslog.conf

*.warn;*.notice;kern.*;ifs.info;istat.none;    @10.1.10.20

!lsass

cifs.*                                          @10.1.10.20

!lwio

cifs.*                                          @10.1.10.20

My last updates to those files are prior to my syslog changes and subsequent. I tried to send a logger message like this;

logger -i -p cifs.err "cifs_err_test_A"

...but it's not showing up in the local files or in the downstream rsyslogd log. I'm wondering if there's an error in my logger syntax or if maybe just the hamhanded approach of appending text to one of those two files are a practical test.

R

Same here. I can get data into those files but it looks like I'm not getting out via syslogd without a fight.

Is the data at least reaching your remote syslog server (i.e.: did you confirm with tcpdump)?

Ok, finally got this to work for my remote syslog server.  I'm using 'rsyslog' and I had to add this in /etc/rsyslog.conf (and restart rsyslog):

:programname, isequal, "lsass" /var/log/messages

Now I can see the messages appear when I run logger -i -t lsass -p cifs.err "cifs_err_test_A `date`" on the cluster.

See screenshot, cluster window is on the left and remote syslog on the right.

Well played sir. I was able to look up the syntax and understand it pretty readily and apply the same for lwiod.log.

Thanks for following up!

For anyone finding this thread, there's some good info on the solution Yan's showed in this link; http://www.sclug.org/sites/www.sclug.org/files/presentations/rsyslog_filtering.pdf

IMPORTANT: I've just stumbled on something today related to this config that might cause certain messages to be ignored by syslog.

If you are using a custom syslog configuration on the cluster with a '!progname' construct, you will probably need to update your custom config.  The custom syslog configuration will be in file /etc/mcp/override/syslog.conf, an example that uses !progname is pasted below:

!audit_protocol

*.*     @192.168.5.10

!lsass

cifs.*  @192.168.5.10

Such a custom config that has '!progname' must be updated to add a '!*' on a line by itself.

So the updated config looks like this:

!audit_protocol

*.*     @192.168.5.10

!lsass

cifs.*  @192.168.5.10

!*

What this extra line does is reset the progname so that the rest of the standard syslog config gets properly applied.  From the syslog.conf man page:

A program or hostname specification may be reset by giving the program or hostname as `*'.