PowerFlex: Gateway cannot connect to MDM after OS upgrade when timezone set to Europe/Kyiv

The upgrade stalls because the GW could not log in to the MDM. The following behaviour is observed on every GW node.

An MDM login API call returns a timeout and an HTTP 428 error:

curl -k -v --basic --user admin:<password> https://<MDM IP>:4445/api/login

curl -k -v --basic --user admin:********** https://10.1.1.1:4445/api/login

* Connected to 10.1.1.1 (10.1.1.1) port 4445 (#0)
{"message":"Failed connecting to cluster: 10.1.1.1 192.168.1.1 10.1.1.2", "httpStatusCode":428,"errorCode":0}

In contrast, the GW login API call succeeds:

curl -k -v --basic --user admin:<password> https://<GW IP>:4445/api/gatewayLogin

curl -k -v --basic --user admin:********** https://10.1.1.1:4445/api/gatewayLogin

* Connected to 10.1.1.1 (10.1.1.1) port 4445 (#0)
{"message":"Login successful"}

GW logs show repeated reconnection attempts and “Connection reset” errors on port 6611:

2025-08-10 05:58:48,747 [command-service-0-ReceiveLoop-450941627] WARN  c.e.e.c.service.CommandService - Connection error connection failed
2025-08-10 05:58:48,749 ERROR c.e.e.n.c.LayeredProtocolConnection - unexpected problem during connection opening to 192.168.1.1: Connection refused (Connection refused)

Port 6611 is reachable with a raw telnet test, confirming that the network itself is not blocked:

curl -v telnet://192.168.1.1:6611
Trying 192.168.1.1:6611 ...
* Connected to 192.168.1.1 (192.168.1.1) port 6611 (#0)

All other components, SDS, SDC, etc., report normal status, indicating a GW‑specific failure.

Impact

The failed GW‑MDM communication prevents the automated upgrade, leaving the cluster in a mixed‑version state (GW is on a new version, MDM is on an old version).

During the OS upgrade on the GW, the TZ database was updated from the historic identifier Europe/Kiev to the newer Europe/Kyiv (IANA rename in 2022).

PowerFlex components generate and validate time‑based tokens during the SSL handshake. The MDM's old version code does not recognise the new identifier, causing a mismatch between the token timestamp and the expected value. The result is a failed authentication (HTTP 428) and an immediate reset of the TCP connection on port 6611.

The problem appears on any GW that runs on a system where the timezone file contains Europe/Kyiv, even when the MDM runs on an older node with Europe/Kiev. The issue is isolated to the GW‑MDM communication path; direct REST calls to /api/gatewayLogin succeed because they bypass the token validation step.

Workaround: Revert the timezone

1. Check the current timezone:

cat /etc/timezone

2. If the output is Europe/Kyiv, change it to the legacy name:

ln -sf /usr/share/zoneinfo/Europe/Kiev /etc/localtime
echo "Europe/Kiev" > /etc/timezone
systemctl restart chronyd
systemctl restart scaleio-gateway

3. Validate that the Gateway can now log in:

The command should return a successful JSON payload instead of HTTP 428.

curl -k -v --basic --user admin:<password> https://<MDM IP>:4445/api/login

Permanent fix

Upgrade all PowerFlex components to a version that includes the timezone fix, mentioned in the Fixed In Version section, below.