Audit with syslog

Henrik,

The same protocol audit events are captured by Syslog Forwarding, as would be captured with CEE.

The syslog forwarding has a similar mechanism as the events forwarded by CEE, to move the pointer back to a previous point in time and resend the auditing events

The command to reset the pointer to a previous point in timed is outlined on page 17 of the following white paper

http://www.emc.com/collateral/white-papers/h12428-wp-best-practice-guide-isilon-file-system-auditing.pdf

In addition, on page 12 of the above white paper, I outline how to setup syslog forwarding.

Can you provide more information on what you are looking for as it relates to security?

Thanks,

-Scott

Hi Scott

I am considering to either use syslog + Graylog or CEE and one of the suggested products (DatAdvantage/StealthAudit/Data Insight/Change Auditor). We only need the audit part, so we don't need all the extra features.

We are having several file shares that we need to audit, because of sensitive data. The file shares are in a separate Access Zone.

One more challenge is that the audit data also can be sensitive The admin of file share A is not allowed to have access to audit data from file share B. All audit data is in one stream, and can't be split to several logservers, I assume.

We are already using Graylog, so it would be easy to choose this solution.

It is fine that we can reset the pointer, but I hope that resetting the pointer not will be necessary - what happens if the Graylog/StealthAudit is unavailable for a periode?

To my understandig the CEE and syslog data can always be found Isilon (if auditing and syslog is configured), unless we choose to delete the data, correct? And if we are using syslog, we can delete the CEE data?

Thanks for the reply,

Henrik

We now have Isilon syslog in Graylog. A line i Graylog could look like this:

Jul 29 11:29:57.000 xxx.xxx.xxx.xxx audit_protocol[7449]: User-SID|1|yyy.yyy.yyy.yyy|OPEN|SUCCESS|1179785|FILE|OPENED|/ifs/data/file...

Can anybody explain these elements:

7449: ?

1: ?

1179785: ?

It seems that Isilon keeps 10 syslog files and then roll over, so we will loose audit data if Graylog is offline for more than 40 minuttes - Isilon saves a syslog file every 4 minuttes. Can you configure Isilon so that it has more than 10 syslog files?

And last: Any advice on what you can configure in the syslog.conf?

Thanks

Henrik

As far as the following question

It seems that Isilon keeps 10 syslog files and then roll over, so we will loose audit data if Graylog is offline for more than 40 minuttes - Isilon saves a syslog file every 4 minuttes. Can you configure Isilon so that it has more than 10 syslog files?

The above is actually controlled by the /etc/newsyslog.conf file. The threshold for rolling the audit syslog file isn't based on time, but based on size. The /var partition on each node is relatively small, so you wouldn't want to keep to many logs here to avoid a scenario where /var fills up.

Here is the entry in /etc/newsyslog.conf

# logfilename          [owner:group]    mode count size when  flags [/pid_file] [sig_num]

/var/log/audit_protocol.log             664  10    10000  *   ZC

So the default permission on the file is 664 (rw-rw-r--)

We keep 10 of the logs

The size is 10000, but I have to look up to see what the unit used here is, as the size should be about 1Gb

The ZC indicates that we would Gzip Compress on log rotation.

Henrik,

The format for the OPEN Audit Syslog Messages is

Open Format

YYYY-MM-DDTHH:MM:SS-GG:GG clustername-n(id1) audit_protocol[#]:UserSID|ZoneID|ClientIP|OPEN|Result(NT_STATUS)|DesiredAccess|HandleID|Type|CreateResult|AbsolutePath

The 7449 is a value that increments for each request, it initially starts at 0

The 1 corresponds to the Access Zone

You can see the Access Zone IDs, by running 'isi zone zones list -v'
tme-sandbox-6# isi zone zones list -v |egrep "Name:|Zone" |grep -v NetBIOS

                    Name: System

                  Zone ID: 1

For the 1179785 value, please refer to the following Microsoft documentation, which explains the DesiredAccess Masks.

https://msdn.microsoft.com/en-us/library/cc246503.aspx

As it relates to the Syslog Rolling, you also have the option to reset the pointer in the Audit Logs to an earlier point in time to reforward those logs to syslog. The following command available in OneFS 7.2 will move the Syslog Audit specific pointer in the audit logs stored under /ifs/.ifsvar/audit/logs

Example: The following will update the pointer to forward events newer than Nov 19, 2014 at 2pm

isi audit settings modify --syslog-log-time "Protocol@2014-11-19 14:00:00"

I have been working on setting up the same thing. I'd be interested in what your extractors look like...

Hi Scott

Thanks for the explanation.

The timestamp in the syslog messages look rather random/wrong. Some of the operation I made on the Isilon is first reported hours later, and with the timestamp hours later.

Best regards

Henrik

Something like this:

my $ctr = 0;
while(defined(my $y = )) {
        next if -f '/tmp/noGL2';
        my %out;

        chomp $y;
        my @f = split /\s+\:*\s*/, $y;
        my $zulu = shift @f;
        my $zulu = shift @f;
        my $zulu = shift @f;
        my $fw  = shift @f;
        my $_msgid = shift @f;
        my $_mesg = join ' ', @f;

        my @g;
        my $gotit = 0;

        my $tim = time;

         my %out = ();

        $out{'version'} = "1.1";
        $out{'host'} = $fw;
        $out{'stream'} = $stream;

        my $ys = substr($y, 0, 1024); $ys .= "..." if $ys ne $y;
        $out{'short_message'} = $ys;
        $out{'full_message'} = $y;

        if($y =~/audit_protocol.\d+.: (.+)/) {
                my @fields = split /\|/, $1;

                my $sid  = shift @fields;  $out{'Isilon_SID'} =  $sid;
                my @sid = split /\-/,   $sid;
                my $uid = pop @sid;
                my $sid = join '-', @sid;
                my $cheat = $cheat{$sid};
                   $cheat = 'N/A' unless defined $cheat;
                $out{'Isilon_AD'} =  $cheat;
                $out{'Isilon_ADUID'} =  $uid;

                my $ruf1 = shift @fields; #$out{'Isilon_'} =

                my $ip   = shift @fields;
                   $out{'Isilon_Ip'} = $ip;

                my $op   = shift @fields;
                   $out{'Isilon_Op'} = $op; #XX

                my $res  = shift @fields;
                   $res=~s/\:\d+$//;
                   $out{'Isilon_Result'} = $res;

                my $ruf2 = shift @fields; #$out{'Isilon_'} =

                my $filetype = shift @fields;
                   $out{'Isilon_FileType'} = $filetype; #XX

                my $detop = shift @fields;
                   $out{'Isilon_Op'} = $detop    if $op eq 'OPEN';
                   $out{'Isilon_Op'} = 'RENAMED' if $op eq 'RENAME';
                   $out{'Isilon_Op'} = 'DELETED' if $op eq 'DELETE';

                my $fn   = pop @fields;
                   $out{'Isilon_PathName'} = $fn;

                my @fn   = split /\//, $fn;
                my $fn = pop @fn;
                my @ext= split /\./, $fn;
                my $ext= 'N/A';
                   $ext = pop @ext if $#ext > 0;
                $out{'Isilon_FileName'} = $fn;
                $out{'Isilon_DirName'} = join '/', @fn;
                $out{'Isilon_Extension'} = $ext;
        }
        #$ctr++;
        my $out = encode_json \%out;
        $socket[$ctr % 2]->send($out);
        $socket[$ctr % 2]->send("\000") || exit(1);
}

Henrik,

What version of OneFS are you running? I do see a bug open for the issue.

It may be best to open a support case and have it attached to bug 147392 or one of it variants depending on the version of OneFS.

-Scott

Hi Scott

OneFS 7.0.1. I will talk to support about the problem.

Henrik

@Henrik_Ravn​ 

Im trying to setup the same on my graylog. Possible you can guide how you did it. I have your code above. But where exactly this code goes. And does it use syslog to receive.