This post is more than 5 years old
1 Rookie
•
8 Posts
0
7358
October 23rd, 2018 19:00
Sense key: 3 Sense code: 11 Sense qualifier: 0: Disk 0:0:0
Hello,
Our T310 server with raid 5 running on perc h700 started displaying some errors recently. It is just 2 months our of warranty and not allowed to be extended. I wish we could. I suspect a drive is starting to go bad. I wondered if I someone could look over the info below and confirm my thoughts. And to also recommend the next course of action. The primary concerning errors I will post first, then secondary errors that might also be a concern, then system, controller and drive info. Sorry if the formatting is less than ideal.
Please let me know if you need anything else. Thank you very much for your help.
---------- most concerning errors
2095 Sun Oct 21 21:11:57 2018 Storage Service Unexpected sense. SCSI sense data: Sense key: 3 Sense code: 11 Sense qualifier: 0: Physical Disk 0:0:0 Controller 0, Connector 0
2095 Sun Oct 21 21:11:57 2018 Storage Service Unexpected sense. SCSI sense data: Sense key: 6 Sense code: 29 Sense qualifier: 0: Physical Disk 0:0:0 Controller 0, Connector 0
2095 Sun Oct 21 21:11:57 2018 Storage Service Unexpected sense. SCSI sense data: Sense key: 3 Sense code: 11 Sense qualifier: 0: Physical Disk 0:0:0 Controller 0, Connector 0
2266 Sun Oct 21 21:11:57 2018 Storage Service Controller log file entry: Physical Disk 0:0:0 Controller 0, Connector 0
2095 Sun Oct 21 21:11:54 2018 Storage Service Unexpected sense. SCSI sense data: Sense key: 6 Sense code: 29 Sense qualifier: 0: Physical Disk 0:0:0 Controller 0, Connector 0
--------------- other errors that are also slightly concerning
2095 Sun Oct 21 21:11:57 2018 Storage Service Unexpected sense. SCSI sense data: Sense key: 3 Sense code: 11 Sense qualifier: 0: Physical Disk 0:0:0 Controller 0, Connector 0
2095 Sun Oct 21 21:11:57 2018 Storage Service Unexpected sense. SCSI sense data: Sense key: 6 Sense code: 29 Sense qualifier: 0: Physical Disk 0:0:0 Controller 0, Connector 0
2095 Sun Oct 21 21:11:57 2018 Storage Service Unexpected sense. SCSI sense data: Sense key: 3 Sense code: 11 Sense qualifier: 0: Physical Disk 0:0:0 Controller 0, Connector 0
2266 Sun Oct 21 21:11:57 2018 Storage Service Controller log file entry: Physical Disk 0:0:0 Controller 0, Connector 0
2095 Sun Oct 21 21:11:54 2018 Storage Service Unexpected sense. SCSI sense data: Sense key: 6 Sense code: 29 Sense qualifier: 0: Physical Disk 0:0:0 Controller 0, Connector 0
2243 Sat Oct 20 04:24:14 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Oct 20 03:28:07 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2243 Sat Oct 13 04:27:07 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Oct 13 03:27:57 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2358 Fri Oct 12 16:54:20 2018 Storage Service The battery charge cycle is complete.: Battery 0 Controller 0
2199 Fri Oct 12 13:28:30 2018 Storage Service The virtual disk cache policy has changed.: Virtual Disk 0 (Virtual Disk 0) Controller 0 (PERC H700 Adapter)
2189 Fri Oct 12 13:28:29 2018 Storage Service The controller write policy has been changed to Write Back.: Battery 0 Controller 0
2247 Fri Oct 12 12:41:54 2018 Storage Service The controller battery is charging.: Battery 0 Controller 0
2177 Fri Oct 12 12:41:40 2018 Storage Service The controller battery Learn cycle has completed.: Battery 0 Controller 0
2188 Fri Oct 12 11:49:10 2018 Storage Service The controller write policy has been changed to Write Through.: Battery 0 Controller 0
2199 Fri Oct 12 11:49:10 2018 Storage Service The virtual disk cache policy has changed.: Virtual Disk 0 (Virtual Disk 0) Controller 0 (PERC H700 Adapter)
2176 Fri Oct 12 10:18:05 2018 Storage Service The controller battery Learn cycle has started.: Battery 0 Controller 0
2180 Mon Oct 08 09:46:30 2018 Storage Service The controller battery Learn cycle will start in 4 days.: Battery 0 Controller 0
2243 Sat Oct 06 04:23:47 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Oct 06 03:27:50 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2243 Sat Sep 29 04:26:51 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Sep 29 03:27:43 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2243 Sat Sep 22 04:25:57 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Sep 22 03:27:35 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2358 Mon Sep 17 11:35:22 2018 Storage Service The battery charge cycle is complete.: Battery 0 Controller 0
2334 Mon Sep 17 11:14:13 2018 Storage Service Controller event log: Unexpected sense: Encl PD 20 Path 5882b0b04b4bf400, CDB: 1c 01 a0 00 04 00, Sense: 5/24/00: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:12 2018 Storage Service Controller event log: Current capacity of the battery is above threshold: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:12 2018 Storage Service Controller event log: Battery started charging: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:12 2018 Storage Service Controller event log: Unexpected sense: Encl PD 20 Path 5882b0b04b4bf400, CDB: 1c 01 a0 00 04 00, Sense: 5/24/00: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:11 2018 Storage Service Controller event log: Inserted: PD 01(e0x20/s1): Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:11 2018 Storage Service Controller event log: Inserted: PD 02(e0x20/s2): Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:11 2018 Storage Service Controller event log: Time established as 09/17/18 11:09:52; (65 seconds since power on): Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:11 2018 Storage Service Controller event log: Battery temperature is normal: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:10 2018 Storage Service Controller event log: Shutdown command received from host: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:10 2018 Storage Service Controller event log: Firmware initialization started (PCI ID 0079/1000/1f16/1028): Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:10 2018 Storage Service Controller event log: Battery Present: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:10 2018 Storage Service Controller event log: Package version 12.10.7-0001: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:10 2018 Storage Service Controller event log: Board Revision A04: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:10 2018 Storage Service Controller event log: Enclosure PD 20(c None/p0) communication restored: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:10 2018 Storage Service Controller event log: Inserted: Encl PD 20: Controller 0 (PERC H700 Adapter)
2334 Mon Sep 17 11:14:10 2018 Storage Service Controller event log: Inserted: PD 00(e0x20/s0): Controller 0 (PERC H700 Adapter)
1000 Mon Sep 17 11:14:02 2018 Instrumentation Service Server Administrator starting
1012 Mon Sep 17 11:14:02 2018 Instrumentation Service IPMI status Interface: OS
1001 Mon Sep 17 11:14:02 2018 Instrumentation Service Server Administrator startup complete
1008 Mon Sep 17 11:14:02 2018 Instrumentation Service Systems Management Data Manager Started
2243 Sat Sep 15 04:27:15 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Sep 15 03:29:21 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2085 Thu Sep 13 16:50:27 2018 Storage Service Virtual disk Check Consistency completed: Virtual Disk 0 (Virtual Disk 0) Controller 0 (PERC H700 Adapter)
2058 Thu Sep 13 16:01:54 2018 Storage Service Virtual disk Check Consistency started: Virtual Disk 0 (Virtual Disk 0) Controller 0 (PERC H700 Adapter)
2243 Sat Sep 08 04:26:10 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Sep 08 03:29:14 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2243 Sat Sep 01 04:28:10 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Sep 01 03:29:08 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2243 Sat Aug 25 04:30:11 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Aug 25 03:29:01 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2243 Sat Aug 18 04:29:10 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Aug 18 03:28:50 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2243 Sat Aug 11 04:26:08 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Aug 11 03:28:44 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2243 Sat Aug 04 04:30:35 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Aug 04 03:28:35 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2243 Sat Jul 28 04:28:15 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Jul 28 03:28:30 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2243 Sat Jul 21 04:26:26 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Jul 21 03:28:24 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2358 Sat Jul 14 13:57:59 2018 Storage Service The battery charge cycle is complete.: Battery 0 Controller 0
2189 Sat Jul 14 10:32:10 2018 Storage Service The controller write policy has been changed to Write Back.: Battery 0 Controller 0
2199 Sat Jul 14 10:32:10 2018 Storage Service The virtual disk cache policy has changed.: Virtual Disk 0 (Virtual Disk 0) Controller 0 (PERC H700 Adapter)
2247 Sat Jul 14 09:46:40 2018 Storage Service The controller battery is charging.: Battery 0 Controller 0
2177 Sat Jul 14 09:46:25 2018 Storage Service The controller battery Learn cycle has completed.: Battery 0 Controller 0
2188 Sat Jul 14 08:53:55 2018 Storage Service The controller write policy has been changed to Write Through.: Battery 0 Controller 0
2199 Sat Jul 14 08:53:55 2018 Storage Service The virtual disk cache policy has changed.: Virtual Disk 0 (Virtual Disk 0) Controller 0 (PERC H700 Adapter)
2176 Sat Jul 14 07:22:36 2018 Storage Service The controller battery Learn cycle has started.: Battery 0 Controller 0
2243 Sat Jul 14 04:24:56 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Jul 14 03:28:14 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
2085 Tue Jul 10 17:10:52 2018 Storage Service Virtual disk Check Consistency completed: Virtual Disk 0 (Virtual Disk 0) Controller 0 (PERC H700 Adapter)
2058 Tue Jul 10 16:21:28 2018 Storage Service Virtual disk Check Consistency started: Virtual Disk 0 (Virtual Disk 0) Controller 0 (PERC H700 Adapter)
2180 Tue Jul 10 05:59:02 2018 Storage Service The controller battery Learn cycle will start in 4 days.: Battery 0 Controller 0
2243 Sat Jul 07 04:19:48 2018 Storage Service The PTRL Read has stopped.: Controller 0 (PERC H700 Adapter)
2242 Sat Jul 07 03:28:08 2018 Storage Service The PTRL Read has started.: Controller 0 (PERC H700 Adapter)
------------------------ system info
server t310
Status ID Name Slot ID State Firmware Version Storport Driver Version
0 PERC H700 Adapter PCI Slot 1 Ready 12.10.7-0001 6.1.7601.18386
Firmware Version 12.10.7-0001
Driver Version 4.31.01.64
Storport Driver Version 6.1.7601.18386
server 2008 r2 sp1
ID 0:0:0
Status OK
Name Physical Disk 0:0:0
State Online
Power Status Spun Up
Bus Protocol SATA
Media HDD
Revision MA08
T10 PI Capable No
Certified Yes
Capacity 232.25GB
Used RAID Disk Space 232.25GB
Available RAID Disk Space 0.00GB
Hot Spare No
Vendor ID DELL(tm)
Product ID ST3250310NS
Serial No. 9SF18CHB
Part Number TH0F420T2123398L001WA0
Negotiated Speed 3.00 Gbps
Capable Speed 3.00 Gbps
Sector Size 512B
SAS Address 4433221107000000
Non-RAID HDD Disk Cache Policy Not Applicable
Physical Disk 0:0:1 Online Spun Up Available TasksBlinkUnblinkOffline ... Execute SATA HDD 02.03B05
ID 0:0:1
Status OK
Name Physical Disk 0:0:1
State Online
Power Status Spun Up
Bus Protocol SATA
Media HDD
Revision 02.03B05
T10 PI Capable No
Certified Yes
Capacity 232.25GB
Used RAID Disk Space 232.25GB
Available RAID Disk Space 0.00GB
Hot Spare No
Vendor ID DELL(tm)
Product ID WDC WD2502ABYS-18B7A0
Serial No. WD-WCAT1J281714
Part Number TH0H962F1255216R036CA0
Negotiated Speed 3.00 Gbps
Capable Speed 3.00 Gbps
Sector Size 512B
SAS Address 4433221106000000
Non-RAID HDD Disk Cache Policy Not Applicable
Physical Disk 0:0:2 Online Spun Up Available TasksBlinkUnblinkOffline ... Execute SATA HDD 02.03B05
ID 0:0:2
Status OK
Name Physical Disk 0:0:2
State Online
Power Status Spun Up
Bus Protocol SATA
Media HDD
Revision 02.03B05
T10 PI Capable No
Certified Yes
Capacity 232.25GB
Used RAID Disk Space 232.25GB
Available RAID Disk Space 0.00GB
Hot Spare No
Vendor ID DELL(tm)
Product ID WDC WD2502ABYS-18B7A0
Serial No. WD-WCAT1J280891
Part Number TH0H962F1255216R032DA0
Negotiated Speed 3.00 Gbps
Capable Speed 3.00 Gbps
Sector Size 512B
SAS Address 4433221105000000
Non-RAID HDD Disk Cache Policy Not Applicable
virtual disk
Status Name State Hot Spare Policy violated Tasks Layout Size T10 Protection Information Status Device Name Bus Protocol Media Read Policy Write Policy Stripe Element Size Disk Cache Policy
Virtual Disk 0 Ready Not Assigned Available TasksReconfigure ...Delete ...Check ConsistencyAssign/Unassign Dedicated Hot Spare ...BlinkUnblinkRename ...Change Policy ...Slow Initialize ...Fast Initialize ...Replace Member Disk ... Execute RAID-5 464.50GB No Windows Disk 0 SATA HDD Adaptive Read Ahead Write Back 64 KB Enabled


Daniel My
12 Elder
•
6.2K Posts
0
October 24th, 2018 09:00
Hello
I put the logs in a spoiler tag to make the thread easier to read. In the future, if you want to add logs to a thread then please either attach a file or provide a URL to a text sharing site instead of copying and pasting logs into a post. Log snippets are fine, but large snippets or full logs don't belong in the post body.
These look like bad block errors to me. Bad blocks happen with drives, they are fairly common. If multiple bad blocks are found on a drive then it could be a sign that the drive is going to fail. Once the disk crosses a threshold for bad blocks the SMART on the disk will report the drive as predictive failure. If I had several drives that were reporting bad blocks then I would be proactive about replacing them. If it was just one drive and I had good backups then I would wait for it to go predictive failure before replacing. I didn't notice any of the drives being predictive failure in the logs.
I would also run a consistency check. It is good to run consistency checks at regular intervals or when disk issues are encountered. A consistency check will verify the data is intact. You can find more information in the controller manual.
http://www.dell.com/storagecontrollermanuals/
Thanks
joedelledge
1 Rookie
•
8 Posts
0
October 24th, 2018 13:00
Hi Daniel,
Thank you so much for the speedy response. Thanks for changing the log portion to a spoiler. I will be cautious in the future with long logs. I didn't see an attach file option, but will look for that next time.
I will keep an eye on the drive, and now understand that an occasional error as such is just a bad block being found.
Did you also see the error about :
2334 Mon Sep 17 11:14:12 2018 Storage Service Controller event log: Unexpected sense: Encl PD 20 Path 5882b0b04b4bf400, CDB: 1c 01 a0 00 04 00, Sense: 5/24/00: Controller 0 (PERC H700 Adapter)
^ it looks to be associated with a reboot. I am guessing it was just slow to find the controller on reboot?
I will run a consistency check. I faithfully run a consistency check about once per month.
Again thank you very much for the speedy and expert response.
Very pleased.
Daniel My
12 Elder
•
6.2K Posts
0
October 24th, 2018 14:00
Yes, it appears to have occurred during startup. Illegal requests can typically be ignored unless they occur frequently or issues are experienced when they occur. It appears the same command was attempted twice and failed both times. It may have been successful on the next attempt.
Illegal requests can occur under normal operating conditions, one common reason is a device being requested may not be in a ready state. It may occur during startup and the device has not completed initialization, or the device may have entered a power saving state. If illegal requests occur often or issues are experienced when they occur then it could be faulty hardware, incompatible hardware, or a firmware communication issue. I would start by making sure all firmware and drivers are updated.
Thanks