VxRail Dashboard - The provided vCenter credentials are not valid

Hey there,

I'm running into an issue with a VxRail cluster when trying to access any details about it. On the main dashboard page I get a "The provided vCenter credentials are not valid" message on the System Health, while none of the views under the Monitor/Configure VxRail in the web client are returning any information, some return the "The provided vCenter credentials are not valid" message. Initially, I thought this has something to do with vCenter machine ssl certificate being replaced a few months ago and I was planning to go through this KB: VxRail: How to manually import vCenter SSL certificate on VxRail Manager | Dell US , however when going through with that fix I got an error - URLError(gaierror(-2, 'Name or service not known'). Looking through the logs I did find this in the short.term.log:

"socket.gaierror: [Errno -2] Name or service not known"
2023-09-20-09:23:46 microservice.do-cluster "[2023-09-20 09:23:46,897: INFO/MainProcess] Input url is qualified for certificate verification: vcenter.domain enable_m2m:False"
2023-09-20-09:23:46 microservice.configservice "2023-09-20 09:23:46,892 [INFO] <MainThread:139673557267776> service.py get_configuration() (117): read from db: certificate_verification_enable = true"
2023-09-20-09:23:46 microservice.do-cluster "[2023-09-20 09:23:46,898: INFO/MainProcess] Input url is qualified for certificate verification: vcenter.domain enable_m2m:False"
2023-09-20-09:23:46 microservice.do-cluster "[2023-09-20 09:23:46,901: INFO/MainProcess] Task worker.celery_task.rest_cluster.login_by_session_cookie[4cc52718-cd9c-4ee6-9393-e8b3aa4e12c3] succeeded in 0.03073218999998062s: {'result': 'LOGIN_BY_SESSION_COOKIE_FAIL'}"
2023-09-20-09:23:46 microservice.do-cluster "[2023-09-20 09:23:46,901: ERROR/MainProcess] Failed to login by session cookie!"
2023-09-20-09:23:46 microservice.do-cluster "Traceback (most recent call last):"
2023-09-20-09:23:46 microservice.do-cluster "  File ""/home/app/worker/task/vcenter.py"", line 414, in login_by_session_cookie"

And this:

2023-09-20-09:23:47 microservice.auth-service "2023-09-20 09:23:47 [ERROR] <> verify.go TokenVerify() (89): Failed to get result state"
2023-09-20-09:23:47 microservice.auth-service "2023-09-20 09:23:47 [ERROR] <> authsvc.go TokenAuth() (152): Unauthorized"

The strange part about that first message is the fact that the vcenter FQDN is incomplete. I did edit it for this post, but in the original log entry the fqdn is missing the last part of the domain (.com).

Is it possible that the vCenter fqdn became corrupted in the VxRail database? Any easy way to check that? I did try to get into the VxRail database, but things are different than they were in version 4.x. in postgres.

VxRail version is 7.0.241 and this is an external vCenter deployed outside of VxRail.

Thanks,