Unsolved
This post is more than 5 years old
4 Posts
1
1386
February 5th, 2018 10:00
Has anyone an iops drop every hour?
Monitoring our servers, i saw a extreme iowait using the unity time every hour, exactly 20 minutes past every hour. The unity stats reflects this with an iops drop like this:
In our SPB in service shell and looking /var/log/messages, we see this:
2018-02-05T15:20:01+00:00 self CRON[5470]: (root) CMD (/EMC/Platform/bin/check_disk_usage.sh &>/dev/null)
2018-02-05T15:20:01+00:00 self CRON[5471]: (root) CMD ([ -x /usr/lib64/sa/sa1 ] && exec /usr/lib64/sa/sa1 -S ALL 1 1)
2018-02-05T15:20:01+00:00 self CRON[5472]: (root) CMD (PATH="/sbin:$PATH" /etc/cron.hourly/monitor_ntp --log_time_stamps)
2018-02-05T15:20:01+00:00 self CRON[5469]: pam_unix(crond:session): session closed for user root
2018-02-05T15:20:01+00:00 self CRON[5467]: pam_unix(crond:session): session closed for user root
2018-02-05T15:20:02+00:00 self CRON[5468]: pam_unix(crond:session): session closed for user root
It's possible that one of this cron actions do the iops drop? Anyone has see this in their system?
Regards.
kelleg
6 Operator
•
4.5K Posts
0
February 6th, 2018 08:00
I'd recommend that you open a service request with support. Be sure to include which code version is currently running, there may have been fixes in the latest releases (but I'm not aware of any issues like this). I notice that it appears to be the Reads that are dropping, have you checked at the hosts are not performing any activity at those times? Also, please include the types of hosts that are experiencing this issue - what OS, applications, each. Also, get a new Data Collect just after (say about 5 minutes after a drop) you see the drop. We'll also need a couple of the performance archives - see KB 491175 for additonal information about collecting the data needed for a performance review.
glen
paranoidandroi1
20 Posts
0
February 6th, 2018 08:00
Which code level is it ?
I'm not seeing this in 4.1.1 and 4.1.2
tonin1998
4 Posts
0
February 7th, 2018 04:00
It's 4.2.1.9535982, the last target I think.
paranoidandroi1
20 Posts
0
February 7th, 2018 05:00
Maybe the cron jobs are not wrongly niced. Did you raise a SR for that issue ?
kelleg
6 Operator
•
4.5K Posts
0
February 7th, 2018 07:00
There are a couple of things that "might" be causing this, but it would be best that you open a case with support.
Glen
kelleg
6 Operator
•
4.5K Posts
0
February 9th, 2018 09:00
Did you open a service request with EMC support. If so, what is the SR number - I'd like to track this.
glen