Has anyone an iops drop every hour?

[tonin1998]

Monitoring our servers, i saw a extreme iowait using the unity time every hour, exactly 20 minutes past every hour. The unity stats reflects this with an iops drop like this:

In our SPB in service shell and looking /var/log/messages, we see this:

2018-02-05T15:20:01+00:00 self CRON[5470]: (root) CMD (/EMC/Platform/bin/check_disk_usage.sh &>/dev/null)

2018-02-05T15:20:01+00:00 self CRON[5471]: (root) CMD ([ -x /usr/lib64/sa/sa1 ] && exec /usr/lib64/sa/sa1 -S ALL 1 1)

2018-02-05T15:20:01+00:00 self CRON[5472]: (root) CMD (PATH="/sbin:$PATH" /etc/cron.hourly/monitor_ntp --log_time_stamps)

2018-02-05T15:20:01+00:00 self CRON[5469]: pam_unix(crond:session): session closed for user root

2018-02-05T15:20:01+00:00 self CRON[5467]: pam_unix(crond:session): session closed for user root

2018-02-05T15:20:02+00:00 self CRON[5468]: pam_unix(crond:session): session closed for user root

It's possible that one of this cron actions do the iops drop? Anyone has see this in their system?

Regards.

[kelleg]

I'd recommend that you open a service request with support. Be sure to include which code version is currently running, there may have been fixes in the latest releases (but I'm not aware of any issues like this). I notice that it appears to be the Reads that are dropping, have you checked at the hosts are not performing any activity at those times? Also, please include the types of hosts that are experiencing this issue - what OS, applications, each. Also, get a new Data Collect just after (say about 5 minutes after a drop) you see the drop. We'll also need a couple of the performance archives - see KB 491175 for additonal information about collecting the data needed for a performance review.

glen

[paranoidandroi1]

Which code level is it ?

I'm not seeing this in 4.1.1 and 4.1.2

[tonin1998]

It's 4.2.1.9535982, the last target I think.

[paranoidandroi1]

Maybe the cron jobs are not wrongly niced. Did you raise a SR for that issue ?

[kelleg]

There are a couple of things that "might" be causing this, but it would be best that you open a case with support.

Glen

[kelleg]

Did you open a service request with EMC support. If so, what is the SR number - I'd like to track this.

glen