Hey Downhill,

I realize this is not exactly what your asking for as you mention only wanting 10gige interfaces and the following command captures for all interfaces but it will name the pcaps appropriately for the interface that it captures on and it will do it for the array by hostname and interface name you can always delete the unneeded interfaces or modify as needed.

1. mkdir -p /ifs/data/Isilon_Support/$(date +%m%d%Y)

2. isi_for_array 'for i in `ifconfig | grep -B2 ether | grep flags | cut -d: -f1`; do tcpdump -i ${i} -s0 -w /ifs/data/Isilon_Support/$(date +%m%d%Y)/`hostname`.${i}_$(date +%m%d%Y_%H%M%S).pcap &; done'

I hope that helps,

D_Tracy

thanks guys, that was exactly what I was looking for. I didn't realize the names of these interfaces is different in the UI than physically seen with ifconfig, plus my searching the knowledge base for that command kept turning up nothing. Got my captures now though.

thank you D_Tracy, have you tried isi_netlogger with vlanX interfaces (VLAN tagged interfaces, not physical)

Yan,

can isi_netlogger take -w parameter and if so, if you need to capture from vlanX interface on multiple nodes what syntax would i use ? Where would the pcap file reside ..on each node ?

Thank you

Once isi_netlogger is stopped any way you ran it, it will bundle all the pcaps up and delete the folders it created and the .pid file. You will find the bundle in /ifs/netlog/bundled (note, if netlogger created a lot of files and/or the files are very large, this process may take some time)

it will look like the following:

ls -l /ifs/netlog/bundled

total 102

-rw-r--r--    1 root  wheel  686 Aug 22 23:52 netlog-2013-08-22_23.52.02.tar.bz2

-rw-rw-rw-    1 root  wheel  542 Sep 20 16:01 netlog-2013-09-20_16.01.54.tar.bz2

No need to specify '-w' when using isi_netlogger, that is done by default.

To capture on vlan interface you would use isi_netlogger -i vlan1 for example.  This will simply pass the command to 'tcpdump -i '

Example:

yfvm-7201-2# isi_netlogger -c -i em3 -p
Netlogger keeping 3 logs of 10 minutes each...
running on nodes: ['yfvm-7201-1 192.168.71.2', 'yfvm-7201-2 192.168.71.3', 'yfvm-7201-3 192.168.71.4']
Running on node yfvm-7201-1 Command:/usr/sbin/tcpdump -U  -i  em3  not arp
Running on node yfvm-7201-2 Command:/usr/sbin/tcpdump -U  -i  em3  not arp
Running on node yfvm-7201-3 Command:/usr/sbin/tcpdump -U  -i  em3  not arp

If you look at output from 'ps' on a given node you will see how the '-w' switch is used:

yfvm-7201-1# isi_for_array -s "ps auxwww | grep 'cpdump' | grep -v isi_rdo"
  
yfvm-7201-1: root   14011  0.0  0.2 30772  4736  ??  D     4:20PM   0:00.56 /usr/sbin/tcpdump -U -w /ifs/netlog/yfvm-7201-1/netlog-yfvm-7201-1.em3.2016-05-19_16.20.01.pcap -i em3 not arp
yfvm-7201-2: root   14072  0.0  0.2 30772  4888  ??  S     4:20PM   0:00.06 /usr/sbin/tcpdump -U -w /ifs/netlog/yfvm-7201-2/netlog-yfvm-7201-2.em3.2016-05-19_16.20.01.pcap -i em3 not arp
yfvm-7201-3: root   13881  0.0  0.2 30772  4696  ??  S     4:20PM   0:00.11 /usr/sbin/tcpdump -U -w /ifs/netlog/yfvm-7201-3/netlog-yfvm-7201-3.em3.2016-05-19_16.20.01.pcap -i em3 not arp

And when you ctrl-c from the terminal where you launched isi_netlogger it will bundle the individual capture files as mentioned by D_Tracy.

Notice in the output below that I have 2 files per node since by default it captures for 10 minutes before rotating the file and it will keep 3 files per node (by default).  This can be adjusted with the -t (how many minutes before rotating) and -k (how many files to keep) flags.

yfvm-7201-1# tar ztvf /ifs/netlog/bundled/netlog-2016-05-19_16.32.02.tar.bz2
drwxr-xr-x  2 root   wheel       0 May 19 16:30 ifs/netlog/yfvm-7201-1/
-rw-r--r--  0 root   wheel   76185 May 19 16:29 ifs/netlog/yfvm-7201-1/netlog-yfvm-7201-1.em3.2016-05-19_16.20.01.pcap
-rw-r--r--  0 root   wheel    3718 May 19 16:32 ifs/netlog/yfvm-7201-1/netlog-yfvm-7201-1.em3.2016-05-19_16.30.01.pcap
drwxr-xr-x  2 root   wheel       0 May 19 16:30 ifs/netlog/yfvm-7201-2/
-rw-r--r--  0 root   wheel    2970 May 19 16:29 ifs/netlog/yfvm-7201-2/netlog-yfvm-7201-2.em3.2016-05-19_16.20.01.pcap
-rw-r--r--  0 root   wheel     713 May 19 16:30 ifs/netlog/yfvm-7201-2/netlog-yfvm-7201-2.em3.2016-05-19_16.30.01.pcap
drwxr-xr-x  2 root   wheel       0 May 19 16:30 ifs/netlog/yfvm-7201-3/
-rw-r--r--  0 root   wheel   12619 May 19 16:30 ifs/netlog/yfvm-7201-3/netlog-yfvm-7201-3.em3.2016-05-19_16.20.01.pcap
-rw-r--r--  0 root   wheel     318 May 19 16:30 ifs/netlog/yfvm-7201-3/netlog-yfvm-7201-3.em3.2016-05-19_16.30.01.pcap

excellent, thank you Yan

Yeah, thanks again Yan. Netlogger is slick for capturing things.

Yan,

i tried to use isi_netlogger with vlan interface and getting this. Interface is up and running, serving connections:

cluster-1# isi_netlogger -c -i vlan6 -p

Netlogger keeping 3 logs of 10 minutes each...

On Node cluster-1, Interface vlan6 is either unusable or down or inactive

ifconfig

vlan6: flags=8843 metric 0 mtu 1500

  options=103

  ether 00:07:43:0a:74:3f

  inet 10.224.14.130 netmask 0xffffff00 broadcast 10.224.14.255 zone 1

  inet 10.224.14.160 netmask 0xffffff00 broadcast 10.224.14.255 zone 2

Is this error thrown for every node or just for node 1?

i just get the error that i pasted, i looked on other nodes and tcpdump is not running.

You have to use the following workaround, found an internal bug around this.  So change your isi_netlogger syntax to this:

isi_netlogger -c -p -a -- -i vlan6

Unfortunately you can't specify any additional filters with the -- construct when you use this workaround; the above will result in the following tcpdump command:

/usr/sbin/tcpdump -U -a -i vlan6

thank you Yan, this makes is very crippled as i need to be able to filter on specific host or specific protocol. Do you know when this bug will be addressed ?

Thank you