PowerScale: 'The authentication request could not be handled' error with Centrify LDAP

Examples:

# isi auth groups members list dellgroup --zone dellzone --provider="lsa-ldap-provider:Centrify LDAP Proxies"
Failed to get members for group GROUP:dellgroup: The authentication request could not be handled

OR

isi auth groups members list --gid 30118 
Failed to get members for group GID:30118: The authentication request could not be handled

In packet captures, Dell Technologies sees the following errors:
 

errorMessage: cdcLdapSearch :Bad parameter (cdcRC=4), errSystem=Ldap, errCode=-7, errString=Bad search filter

Full example of packet capture:

# tshark -r file.cap

1060 736.218674 10.193.45.68 â 10.193.3.52  LDAP 1194 searchRequest(4) "ou=North America,dc=acf,dc=dell,dc=com" wholeSubtree
1061 736.218828  10.193.3.52 â 10.193.45.68 TCP 66 389 â 15206 [ACK] Seq=3557 Ack=1413 Win=261656 Len=0 TSval=1645667785 TSecr=662636581
1062 736.258022  10.193.3.52 â 10.193.45.68 LDAP 175 searchResDone(4) other (cdcLdapSearch :Bad parameter (cdcRC=4), errSystem=Ldap, errCode=-7, errString=Bad search filter)  [2 results]
1063 736.258142  10.193.3.52 â 10.193.45.68 LDAP 175 searchResDone(4) other (cdcLdapSearch :Bad parameter (cdcRC=4), errSystem=Ldap, errCode=-7, errString=Bad search filter)  [2 results]
1064 736.258150 10.193.45.68 â 10.193.3.52  TCP 66 15206 â 389 [ACK] Seq=1413 Ack=3775 Win=131648 Len=0 TSval=662641755 TSecr=1645667785
1065 736.261570 10.193.45.68 â 10.193.3.52  LDAP 73 unbindRequest(5)

Within the frame, Support can see the following:
 

Lightweight Directory Access Protocol
    LDAPMessage searchResDone(4) other (cdcLdapSearch :Bad parameter (cdcRC=4), errSystem=Ldap, errCode=-7, errString=Bad search filter) [2 results]
        messageID: 4
        protocolOp: searchResDone (5)
            searchResDone
                resultCode: other (80)
                matchedDN:
                errorMessage: cdcLdapSearch :Bad parameter (cdcRC=4), errSystem=Ldap, errCode=-7, errString=Bad search filter
        [Response To: 1060]
        [Time: 0.039348000 seconds]

Starting in OneFS version 8.2.2.0, LDAP member enumeration requests intentionally group the queries to improve response times. OneFS changed from an iterative UID query, user-by-user, to a batched query, for performance reasons. 
The error in the paps suggests the proxy does not support multiple `uid` entries in a search filter. If the proxy does not support multiple 'uid' entries, then Support needs the proxy vendor (that is. Centrify) to fix this issue.

The proxy appears to not like what OneFS sends over the wire, which is in the form of (&(objectClass=posixAccount)(|(uid=user1)(uid=user2)(uid=user3))) to resolve the user members from the memberUid query for the group to fully resolve the bid>name and the members, names>uid.

In addition, Centrify has acknowledged that our behavior changed at some point, from an iterative uid query, user-by-user, to a batched query, as above. According to Centrify knowledge base articles and documentation, the error suggests the proxy does not support multiple `uid` entries in a search filter.

Centrify has acknowledged that the syntax PowerScale is now using is not supported. There is a Request for Enhancement (RFE) to add this to Centrify, and our name has been added to that RFE.

As there is nothing wrong with the search filter, and our software code is not reverting back to a nonbatched query (for performance reasons), Centrify must accommodate our valid search query filter querying multiple users for uidNumber.

Contact the vendor (Centrify) for resolution. 

To confirm if a customer is experiencing the issue, Dell Technologies Support can also collect packet captures while replicating the error:

Here are the steps for packet captures:


1. Generate a list of LDAP servers configured on the PowerScale cluster:
 

# isi auth ldap ls
Name  Base DN       Server Uris         Status
-----------------------------------------------
LDAP  DC=amd,DC=com ldap://isilon02 online
                    ldap://isilon0404

2. Resolve the IP addresses of each of the LDAP servers, above.
 

#nslookup <ldap host>

Make note of the LDAP IP addresses for Support.

For example:
 

powerscale-2-1# nslookup isilon02
Server:         127.42.0.1
Address:        127.42.0.1#53

Non-authoritative answer:
Name:   isilon02.dell.com
Address: 10.178.35.1

3.  Create a directory to contain the packet capture data: 
 

# mkdir -p /ifs/data/Isilon_Support/<SR number>

Substitute the SR number above with the proper PowerScale Support service request number, for example, for service request number 1234567:
 

# mkdir -p /ifs/data/Isilon_Support/1234567

4. Ssh to a node as the 'root' user.

5. Start node packet captures:

i.e., for service request # 1234567, the command would be as follows:
 

# for i in $(ifconfig | grep flags= | cut -f1 -d':'|egrep -v "ib0|ib1|lo0"); do tcpdump -i ${i} -s0 -C 200 -W 3 -w /ifs/data/Isilon_Support/1234567/${HOST}_${i}.pcap port 389 &; done

Type Ctrl-C to break out and let tcpdump run in the background.

6. Next, re-create the issue/error on the node.
7. Once you replicate the issue/error stop the pcap:
 

# pkill -9 tcpdump

Ensure the pcaket capture is not running anymore:
 

# ps -auxwww|grep tcpdump

You should see no tcpdump processes running.

8. Upload the pcaps to Support for review.