ECS: OBS: xDoctor: RAP095: Symptom Code: 2060 Detected TCP-InCsumErrors count exceeds

Summary: xDoctor has detected that the TCP In-CsumErrors count has exceeded the configured threshold.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

The TCP InCsumErrors (Incoming Checksum Errors) count has exceeded 20,000 on the node identified in the alert.

xDoctor has detected that the TCP InCsumErrors count has exceeded the configured threshold. This is an indication that the reported node is experiencing Ethernet corruption. xDoctor's default threshold for this check is 20,000.

xDoctor ERROR Example: 
Timestamp    = 2019-09-17_213225
    Category = OS
    Source   = SOFTWARE
    Severity = ERROR
    Node     = 169.254.6.1
    Message  = Detected TCP InCsumErrors count exceeds threshold
    Extra    = {'169.254.6.6': '200000', '169.254.6.3': '100000'}
    RAP      = RAP095
    Solution = 537292 

Cause

This is often related to a failing hardware component or network interruption in the TCP stack to increase CRC inconsistency to the ECS nodes.

Resolution

IMPORTANT! A new feature has been released in xDoctor 4-8.106.0 and above. This knowledge base (KB) is now automated with xDoctor to quickly attempt to discover the potential cause of xDoctor RAP095 errors. If the script is unable to correct the xDoctor RAP095 errors, it provides a detailed summary of its findings.

For more information, follow ECS: ObjectScale: How to run KB Automation Scripts (Auto Pilot)

Automated Solution: 
To find the master node of the rack:
Command:

ssh master.rack

This is a VDC wide script, hence the target should be the local VDC name. xDoctor topology has the VDC name to use as target when running the script. 

Command: 

# sudo xdoctor --top --vdc

Example: 

admin@ecsnode1:~> sudo xdoctor --top --vdc
 
ECS
|
|- CLOUD - ID:[21a7111a45e4a9dbca00000000000000]
   |
   |- Local VDC - ID:[8af5b9c3-9c0c-43b5-9402-000000000000] Name:[VDC1]
   |- Local SP  - ID:[52576f30-f8f3-493a-9999-000000000000] Name:[SP1]
   |  |
   |  |- Local RACK - Name:[red] Primary:[169.254.1.1] PSNT:[CKM00000000000] SWID:[CKM00000000000]
   |  |  |
   |  |  |- Node  1, [   provo], NAN.IP:[    169.254.1.1], Public.IP:[   10.60.ccc.ddd], DNS:[10.174.ccc.ddd], NTP:[10.18.ccc.ddd , 10.174.ccc.ddd]
   |  |  |- Node  2, [   sandy], NAN.IP:[    169.254.1.2], Public.IP:[   10.60.ccc.ddd], DNS:[10.174.ccc.ddd], NTP:[10.18.ccc.ddd , 10.174.ccc.ddd]
   |  |  |- Node  3, [    orem], NAN.IP:[    169.254.1.3], Public.IP:[   10.60.ccc.ddd], DNS:[10.174.ccc.ddd], NTP:[10.18.ccc.ddd , 10.174.ccc.ddd]
   |  |  |- Node  4, [   ogden], NAN.IP:[    169.254.1.4], Public.IP:[   10.60.ccc.ddd], DNS:[10.174.ccc.ddd], NTP:[10.18.ccc.ddd , 10.174.ccc.ddd]
   |  |  |- Node  5, [  layton], NAN.IP:[    169.254.1.5], Public.IP:[   10.60.ccc.ddd], DNS:[10.174.ccc.ddd], NTP:[10.18.ccc.ddd , 10.174.ccc.ddd]
   |  |  |- Node  6, [   logan], NAN.IP:[    169.254.1.6], Public.IP:[   10.60.ccc.ddd], DNS:[10.174.ccc.ddd], NTP:[10.18.ccc.ddd , 10.174.ccc.ddd]
   |  |  |- Node  7, [    lehi], NAN.IP:[    169.254.1.7], Public.IP:[   10.60.ccc.ddd], DNS:[10.174.ccc.ddd], NTP:[10.18.ccc.ddd , 10.174.ccc.ddd]
   |  |  |- Node  8, [  murray], NAN.IP:[    169.254.1.8], Public.IP:[   10.60.ccc.ddd], DNS:[10.174.ccc.ddd], NTP:[10.18.ccc.ddd , 10.174.ccc.ddd]

 

  1. Run the automation command from the master node with xDoctor 4-8.106.0 and above.  

 

NOTE: Only --target-node is supported for this action.

 

This script was incorporated into xDoctor4/8/106.0 release which features version 3.2 of the Ansible automation scripts.

#sudo xdoctor autopilot --kb 48156 --target-node 169.254.1.x

Example:
Using /etc/ansible/ansible.cfg as config file
VERSION: 3.2.1
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ Starting Automation: 48156
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[WARNING]: Found variable using reserved name: run_once
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ 2025-09-29 08:31:50 TASK OK: hw_check role
│ 2025-09-29 08:31:50 [169.254.1.3] => OK
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ 2025-09-29 08:31:50 TASK OK: Set configuration file variables
│ 2025-09-29 08:31:50 [169.254.1.3] => OK

 

  1. Review summary:
Example: Active issue found that indicates errors are incrementing. 
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ 2025-09-29 09:35:49 TASK OK: Print consolidated summary
│ 2025-09-29 09:35:49 [169.254.1.7] => OK
│ ************************************************
│ RAP095 Diagnostic Summary:
│ ************************************************
│ Issue Status:  FAIL: RAP095 Issue ACTIVE - InCsumErrors still incrementing — ongoing network or maintenance activity may be causing errors. Please investigate.
│ Initial InCsumErrors Count: 65183
│ Recheck After 5 Minutes: Incrementing
│ Threshold: 20000
│ ************************************************
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ NODE AUTOMATION SUMMARY:
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ HOST                   ok    changed   unreachable   failed   skipped   rescued   ignored
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ 169.254.1.7            33          2             0        0        30         0         0
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ Status: PASS
│ Time Elapsed: 0h 0m 5s
│ Debug log: /tmp/autopilot/log/autopilot_48156_20250929_093541.log
│ Message:  Issue is active - ongoing network or maintenance related TCP InCsumErrors detected.
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Example: Historical issue found with errors not incrementing. Threshold increased to avoid future xDoctor notifications. 
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ 2025-09-29 08:36:56 TASK OK: Print consolidated summary
│ 2025-09-29 08:36:56 [169.254.1.3] => OK
│ ************************************************
│ RAP095 Diagnostic Summary:
│ ************************************************
│ Issue Status:  PASS: RAP095 historical issue - InCsumErrors stable — historical issue. Threshold increased by 20,000 and /usr/local/xdoctor/config/pacemaker/bit_flip_check.conf updated.
│ Initial InCsumErrors Count: 4358017
│ Recheck After 5 Minutes: Not required (<20k)
│ Threshold: 20000
│ ************************************************
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ NODE AUTOMATION SUMMARY:
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ HOST                   ok    changed   unreachable   failed   skipped   rescued   ignored
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ 169.254.1.3            34          3             0        0        29         0         0
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ Status: PASS
│ Time Elapsed: 0h 5m 6s
│ Debug log: /tmp/autopilot/log/autopilot_48156_20250929_083136.log
│ Message:  Issue is historical - threshold increased to avoid false alerts.

Example: No issue found

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ 2025-09-29 08:38:42 TASK OK: Print consolidated summary
│ 2025-09-29 08:38:42 [169.254.1.8] => OK
│ ************************************************
│ RAP095 Diagnostic Summary:
│ ************************************************
│ Issue Status:  PASS: No RAP095 issue detected.
│ Initial InCsumErrors Count: 0
│ Recheck After 5 Minutes: Not required (<20k)
│ Threshold: 20000
│ ************************************************
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ NODE AUTOMATION SUMMARY:
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ HOST                   ok    changed   unreachable   failed   skipped   rescued   ignored
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ 169.254.1.8            28          1             0        0        35         0         0
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ Status: PASS
│ Time Elapsed: 0h 0m 5s
│ Debug log: /tmp/autopilot/log/autopilot_48156_20250929_083835.log
│ Message:  No RAP095 issue detected - InCsumErrors below threshold.

    Additional Information

    Reach out to Dell Technical Support and quote this KB if none of the offered workarounds apply to your specific use case. 
    Gather and attach the xDoctor Report Archive.
     

    NOTE: This article is enhanced with new workarounds when new use cases are identified.

     

    Affected Products

    Elastic Cloud Storage

    Products

    ECS Appliance, ECS Appliance Hardware Series, Elastic Cloud Storage
    Article Properties
    Article Number: 000048156
    Article Type: Solution
    Last Modified: 15 يونيو 2026
    Version:  6
    Find answers to your questions from other Dell users
    Support Services
    Check if your device is covered by Support Services.