PowerScale OneFS: Active Directory Provider Goes Missing After Join Attempt for Same Domain

After performing an AD join with the same name of an AD provider that the cluster is already joined to, the provider may go missing. Administrators may see an error related to the join attempt, though this error could vary depending on the cause.

In the example below, we reproduce the issue with the provider TESTDOMAIN.LOCAL by using the incorrect account to facilitate the join.

Outputs from isi auth status and isi auth ads list -v show that the provider is present prior to running the command:

corebuddy-1# isi auth status
ID                                            Active Server                Status
----------------------------------------------------------------------------------
lsa-activedirectory-provider:TESTDOMAIN.LOCAL winserv2019.testdomain.local online
lsa-activedirectory-provider:DOMAIN2.LOCAL    domain2DC1.domain2.local     online
lsa-local-provider:System                     -                            active
lsa-local-provider:test                       -                            active
lsa-local-provider:domain2                    -                            active
lsa-file-provider:System                      -                            active
----------------------------------------------------------------------------------
Total: 6

corebuddy-1# isi auth ads list -v | grep -i testdomain
                     Name: TESTDOMAIN.LOCAL
           Primary Domain: TESTDOMAIN.LOCAL
                   Forest: testdomain.local
           NetBIOS Domain: TESTDOMAIN
                 Hostname: corebuddy.testdomain.local

Below we run the command. Notice that the incorrect username listed (for our example, we used the group name for 'Administrators' instead of the administrator account):

corebuddy-1# isi auth ads create --name=TESTDOMAIN.LOCAL --user=administrators
password:
Failed to join domain 'TESTDOMAIN.LOCAL' account 'corebuddy' user 'administrators@TESTDOMAIN.LOCAL': (LW_ERROR_KRB5KDC_ERR_C_PRINCIPAL_UNKNOWN) Client not found in Kerberos database

Now the provider is missing. In WebUI, this might show up as unstyled, but in CLI it is not in the output for isi auth status.

orebuddy-1# isi auth status
ID                                         Active Server            Status
---------------------------------------------------------------------------
lsa-activedirectory-provider:DOMAIN2.LOCAL domain2DC1.domain2.local online
lsa-local-provider:System                  -                        active
lsa-local-provider:test                    -                        active
lsa-local-provider:domain2                 -                        active
lsa-file-provider:System                   -                        active
---------------------------------------------------------------------------
Total: 5

Because isi auth status can show that a provider is missing for other reasons, we must validate this further. The command isi auth ads list -v does not display any output matching the provider TESTDOMAIN.LOCAL below. This means that the configuration contains no entries matching the missing domain.

corebuddy-1# isi auth ads list -v |grep -i test
corebuddy-1#

LSASS logs show similar errors observed when running the initial command:

2024-06-11T21:25:21.298318+00:00 <30.4> corebuddy-1(id1) lsass[5467]: [LwKrb5GetTgtImpl /b/mnt/src/isilon/fsp/lwadvapi/threaded/krbtgt.c:247] KRB5 Error code: -1765328378 (Message: Client 'administrators@TESTDOMAIN.LOCAL' not found in Kerberos database)
2024-06-11T21:25:21.299207+00:00 <30.4> corebuddy-1(id1) lsass[5467]: [lsass] Error occurred while joining domain: Error code: 41737 (symbol: LW_ERROR_KRB5KDC_ERR_C_PRINCIPAL_UNKNOWN)

The above scenario is not limited to the usage of incorrect credentials. If there is any condition present that would induce a failure to join the AD provider, then the failure results in the provider being deleted. For example, if the same name for the machine account is used elsewhere within the forest or in a trusted domain, that induces a failure. The only difference may be the error that is provided.

When performing an AD join against an already joined domain, the presence of conditions that would induce failure results in the AD provider being deleted. This is by design as OneFS would treat this as a "rejoin" attempt. When the failure is encountered, any related configuration to that specified auth provider is deleted.

corebuddy-1# isi auth status
ID                                         Active Server            Status
---------------------------------------------------------------------------
lsa-activedirectory-provider:DOMAIN2.LOCAL domain2DC1.domain2.local online
lsa-local-provider:System                  -                        active
lsa-local-provider:test                    -                        active
lsa-local-provider:domain2                 -                        active
lsa-file-provider:System                   -                        active
---------------------------------------------------------------------------
Total: 5

Names of authentication providers are global, so having two providers of the same name is not supported. If there are no other outstanding conditions present that would prevent a join operation, rejoining the domain resolves the issue.

If there is a requirement to join the same AD domain provider with separate machine accounts, then the multi-instancing feature should be used. This is elaborated on in the "Zone-specific authentication providers" section of the OneFS Administration Guides. Remember that the rejoined AD provider must be added back to any access zones that previously had it to restore access.

See article PowerScale OneFS Info Hubs for documentation on your version of OneFS.