ECS: [ERROR] RAP081 - All NTP servers are NOT suitable for synchronization

When testing the NTP server if it is capable of synchronizing the time, it shows server dropped. But when running the command again it shows the server is synchronizing the time.

1- Get the list of NTP Servers on each of the listed nodes:
Command:
# getrackinfo -r | grep NTP

Example:

admin@node1:~> getrackinfo -r | grep NTP
        NTPServer =  xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx

2- Test if it is capable of synchronizing time. 

Command:

# sudo ntpdate -p 2 -d <NTP IP Address / NTP FQDN>
or
# sudo ntpdate -p 2 -d `getrackinfo -r | grep NTP |grep -oP "(?:[0-9]{1,3}\.){3}[0-9]{1,3}"`
 

Example:

admin@node1:~> sudo ntpdate -p 2 -d 123.x.xx.xx
10 Mar 11:06:15 ntpdate[69998]: ntpdate 4.2.8p15@1.3728-o Thu Jun 25 14:17:29 UTC 2020 (1)
Looking for host 123.x.xx.xx and service ntp
123.x.xx.xx reversed to <NTP hostname>
host found : <NTP hostname>
transmit(123.x.xx.xx)
transmit(123.x.xx.xx)
123.x.xx.xx: Server dropped: no data

10 Mar 11:06:19 ntpdate[69998]: no server suitable for synchronization found

Wait a few seconds, then run the same command again:

admin@node1:~> sudo ntpdate -p 2 -d 123.x.xx.xx
10 Mar 11:06:24 ntpdate[70526]: ntpdate 4.2.8p15@1.3728-o Thu Jun 25 14:17:29 UTC 2020 (1)
Looking for host 123.x.xx.xx and service ntp
123.x.xx.xx reversed to <NTP hostname>
host found : <NTP hostname>
transmit(123.x.xx.xx)
receive(123.x.xx.xx)
transmit(123.x.xx.xx)
receive(123.x.xx.xx)

server 123.x.xx.xx, port 123
stratum 1, precision -29, leap 00, trust 000
refid [NIST], root delay 0.000244, root dispersion 0.000488
reference time:      e7b58d80.00000000  Fri, Mar 10 2023 11:05:36.000
originate timestamp: e7b58db2.9661f6d5  Fri, Mar 10 2023 11:06:26.587
transmit timestamp:  e7b58db2.89fd47bf  Fri, Mar 10 2023 11:06:26.539
filter delay:  0.13205    0.13127    ----       ----
               ----       ----       ----       ----
filter offset: -0.004465  -0.004421  ----       ----
               ----       ----       ----       ----
delay 0.13127, dispersion 0.00002, offset -0.004421

10 Mar 11:06:26 ntpdate[70526]: adjust time server 123.x.xx.xx offset -0.004421 sec

The NTP server has a rate limitation for NTP requests. 
When xDoctor tries to sync all Nodes simultaneously while the NTP Server has a rate limit, it only responds to the first Node.
xDoctor sends all requests without a delay in between them, therefore they get blocked from the server.

xDoctor Team has confirmed to address this in a future xDoctor release. (Current release: 4.8-89.0) 
Request Info: Reduce the number of ntpdate commands used as it can cause NTP rate limiting which produces false positives as it works normally when xDoctor is not running this check.
 

For Troubleshooting, please check KB 000064221
ECS: xDoctor: RAP081: SymptomCode: 2048: "NTP daemon not running" or "All servers not suitable for synchronization found" | Dell US