1. Preparation
2. LDAP Provider Removal and cleanup
3. LDAP Provider Rebuild
4. Narrow LDAP user and group scope application

 

 

Background: 

 

By default, PowerFlex Management Platform (PFMP) directory providers import all LDAP objects within the configured search scope, including non-user accounts and other objects not required by our RBAC.

In large environments, this behavior can result in significant resource utilization across PFMP management VMs (MVMs), the supporting Postgres database, and the overall platform.

Therefore, LDAP synchronization may fail to complete within acceptable timeframes, causing job backlogs, memory leaks, and concurrency issues.

 

1. Preparation

 

Requirements:

• Global Catalog
• Finding the DN of the AD Groups for PFMP
• Bind User Requirements
• PFMP LDAP General Requirements

 

 

Using the Active Directory Global Catalog

Note:  We require use of port 3268 for standard LDAP or 3269 for LDAPS in AD environments. This allows LDAP group members from child domains, or other trusted domains to be usable in PFMP.

An AD group for example:

 

 

The child domain above na.powerflex.lab.domain.com must be visible by PFMP's LDAP User Federation Keycloak, using the global catalog ports allow them to be reached. 

This is also why we use objectGUID in the PFMP Directory Provider, it ties accounts to their global unique UUID.

 

If both child domains are listed under the Trusts tab, it seems sub domains trust each other by default.

In a Microsoft Active Directory (AD) environment, a parent domain, and its child (sub) domains inherently trust each other in a hierarchical structure. This is known as a transitive trust.

By default, these trusts are two-way and transitive, meaning that a parent domain trusts its child domains and vice versa. This transitive nature extends the trust to all domains within the parent-child hierarchy.

 

Finding available LDAP or AD GC servers from DNS entries

 

Usually the customer provides LDAP address to use.  These settings can be copied from the PFMP Directory Provider configuration before removing it as part of the next steps of this guide.

Alternatively, available servers can be retrieved from DNS for a specific domain:

 

# listing LDAP server
$ host -t srv _ldap._tcp.powerflex.lab.domain.com
Server:  nameserver.domain.com
Address:  10.8.8.8

Non-authoritative answer:
_ldap._tcp.powerflex.lab.domain.com       service = 0 100 636 hcihopad1.powerflex.lab.domain.com. _ldap._tcp.powerflex.lab.domain.com       service = 0 100 389 hcihopad1.powerflex.lab.domain.com. _ldap._tcp.powerflex.lab.domain.com       service = 0 100 389 hcick3ad2.powerflex.lab.domain.com. _ldap._tcp.powerflex.lab.domain.com       service = 0 100 636 hcick3ad1.powerflex.lab.domain.com. _ldap._tcp.powerflex.lab.domain.com       service = 0 100 389 hcihopad3.powerflex.lab.domain.com.

# listing AD GC servers (Active Directory Global Catalog) 
host -t srv _gc._tcp.powerflex.lab.domain.com
Server:  nameserver.domain.com
Address:  10.1.1.1

Non-authoritative answer:
_gc._tcp.powerflex.lab.domain.com service = 0 100 3268 hcihopad3.powerflex.lab.domain.com.
_gc._tcp.powerflex.lab.domain.com service = 0 100 3268 hcick3ad3.powerflex.lab.domain.com.
_gc._tcp.powerflex.lab.domain.com service = 0 100 3268 hcick3ad2.powerflex.lab.domain.com.

 

 

How to find the distinguishedName of an OU or Group.

First, we are going to need the  Distinguished Names (DN)  of the OUs and Groups where we want to read users into PFMP.  These paths do not strictly follow DNS naming and contain OUs inside an x.500 LDAP path which we can extract manually.

We can find these DNs using different utilities depending on OS, one easy way is using curl:

How to test PFMP LDAP settings using curl utility on Linux.

 

The newer versions of curl which is installed on MVMs can be used to find the full distinguished names you need, for example this searches for the full DN of the LabAdmins Group CN:

#Get the DN of your groups for use in the LDAP User filter while also validating your LDAP Group Filter syntax
# Set this to the group filter you intend to use in keycloak
GROUPFILTER='(CN=LabAdmins)'
# Use the LDAP svc bind creds to confirm the user can in fact call LDAP and see the group(s)
BINDUSER="user@powerflex.lab.domain.com"
# LDAP server URL.  For TLS use ldaps:// and port 3269
LDAPSERVER='ldap://ldap.powerflex.lab.domain.com:3268'
# Use the same search base as you intend for the group path, generally just the DC components
SEARCHBASE=DC=powerflex,DC=lab,DC=domain,DC=com
# Call curl which will request the password of the bind user and return results curl -s sub --user "$BINDUSER" "$LDAPSERVER/$SEARCHBASE?distinguishedName?sub?$GROUPFILTER" 

Enter host password for user 'name@powerflex.lab.domain.com':

 

DN:CN=LabAdmins,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
      distinguishedName:CN=LabAdmins,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com

DN: CN=LabAdmins,CN=Users,DC=na,DC=powerflex,DC=lab,DC=domain,DC=com  distinguishedName: CN=LabAdmins,CN=Users,DC=na,DC=powerflex,DC=lab,DC=domain,DC=com

 

 

Manual examples:

 

curl -s sub --user "FLEXLAB\name"
"ldap://ldap.powerflex.lab.domain.com:3268/DC=powerflex,DC=lab,DC=domain,DC=com?distinguishedName,objectGUID?sub?(|(CN=FLEXLAB-APP-PROD-SDS-Admin)(CN=LabAdmins)(CN=LabUsers)(CN=EMEALabAdmins))"

Enter host password for user 'FLEXLAB\name':

DN: CN=LabAdmins,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
       distinguishedName: CN=LabAdmins,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com

DN: CN=LabAdmins,CN=Users,DC=na,DC=powerflex,DC=lab,DC=domain,DC=com
       distinguishedName: CN=LabAdmins,CN=Users,DC=na,DC=powerflex,DC=lab,DC=domain,DC=com

DN: CN=LabUsers,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
       distinguishedName: CN=LabUsers,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com

Find all members of an LDAP group using filter properties that you can then place into the keycloak backend:

#check for all members of the group with CN=LabAdmins

curl -s sub
--user "name@powerflex.lab.domain.com"
"ldap://ldap.powerflex.lab.domain.com:3268/DC=powerflex,DC=lab,DC=domain,DC=com?member?sub?(&(objectClass=group)(cn=LabAdmins))"

DN:CN=LabAdmins,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
        member: CN=matrixadmin,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
        member: CN=matrixadmin,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
        member: CN=svc_adfsck3,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
        member: CN=svc_mcp_rw,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com         member: CN=adcertsvc,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com         member: CN=bobthebuilder,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com         member: CN=Ashish Rahangdale,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
        member: CN=Packer Admin,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
        member: CN=FDLabAdmins,OU=Foundations,DC=powerflex,DC=lab,DC=domain,DC=com
        member: CN=Tejas Wadekar,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com

 

 

How to test PFMP LDAP settings using ldapsearch or curl utility on Linux.

 

If a linux host is available on the network, using ldapsearch utility could be useful to test and validate LDAP settings used in PFMP/Keycloak.

Alternatively, curl utility can also be used for LDAP queries from one of the Management VMs/MVMs.

 

Quick introduction (commonly used options):

 

 

ldapsearch - LDAP search tool
ldapsearch opens a connection to an LDAP server, binds, and performs a search using specified parameters.

usage: ldapsearch [options] [filter
[attributes...]]

where:

filter       RFC 4515 compliant LDAP search filter
attributes   whitespace-separated list of attribute descriptions

Search options:
  -b basedn base dn for search
 -LLL      print responses in LDIF format without comments and version Common options:  -D binddn bind DN  -H URI    LDAP Uniform Resource Identifier(s)  -N        do not use reverse DNS to canonicalize SASL host name  -w passwd bind password (for simple authentication)  -W        prompt for bind password  -x        Simple authentication

 

 

 

Checking/installing the tool:

$ which ldapsearch
/usr/bin/ldapsearch

#RHEL/CentOS install
sudo yum install openldap-clients

# SLES install
sudo zypper install openldap2-client

 

 

Defining Base DN (-b) from LDAP root entry:

 

$ curl "ldap://ldap.powerflex.lab.domain.com:3268/?rootDomainNamingContext"
DN:
        rootDomainNamingContext: DC=powerflex,DC=lab,DC=domain,DC=com

# ldapsearch (with simple auth)
$ ldapsearch -LLL -H ldap://ldap.powerflex.lab.domain.com -x -D "FLEXLAB\ck3-user" -W -b "" -s base "dn=" rootDomainNamingContext

dn: rootDomainNamingContext: DC=powerflex,DC=lab,DC=domain,DC=com

 

 

The root entry provides information about directory server. On AD, rootDomainNamingContext attribute gives the top base DN which can be used for search.

 -W -b "

 

Testing LDAP URI, Bind DN  (user/password with read access) and User Search Settings:

 

$ ldapsearch -LLL -H ldap://powerflex.lab.domain.com:3268 -x -D "CN=ck3-user,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com" -W \ -b "dc=powerflex,dc=lab,DC=domain,dc=com" "(sAMAccountName=username)"
Enter LDAP Password: ********

dn: CN=Name,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
objectClass: person
cn: Name memberOf: CN=EMEALabAdmins,OU=LabAdmins,DC=powerflex,DC=lab,DC=domain,DC=com memberOf: CN=LabUsers,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com memberOf: CN=LabAdmins,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com objectGUID:: 6Z359eP5vky4P0Ye8iFp8g== sAMAccountName: username

# equivalent curl command:

$ curl -u 'CN=ck3-user,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com' \ 
"ldap://ldap.powerflex.lab.domain.com:3268/DC=powerflex,DC=lab,DC=domain,DC=com??sub?(sAMAccountName=username)"
# OR (with attributes selection)
$ curl -u 'FLEXLAB\ck3-user' \
"ldap://ldap.powerflex.lab.domain.com:3268/DC=powerflex,DC=lab,DC=domain,DC=com?cn,objectClass,memberOf,objectGUID,sAMAccountName?sub?(sAMAccountName=username)"

In this example, we are querying LDAP for an existing user account using sAMAccountName attribute (output has been truncated to only show attributes we need).

 

Corresponding settings in the PFMP UI:

Address (-H):  ldap://active.directory.domain.address.com:3268

Bind DN (-D): "CN=ck3-user,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com"   # ("FLEXLAB\ck3-user" or "ck3-user@powerflex.lab.domain.com" are equivalents)

(-x is required for simple authentication to avoid default SASL mode)

Bind DN Password (-W): prompt for password (or use –w '***' to provide it in command)

 

The output of the command validates the following PFMP User Search Settings:

Username LDAP Attribute: sAMAccountName

ID Attribute: objectGUID

Object Class: person

Search Path (-b): "dc=powerflex,dc=lab,DC=domain,dc=com"

 

 

Testing Group Search Settings:

 

$ ldapsearch -LLL -H ldap://powerflex.lab.domain.com:3268 -x -D "FLEXLAB\ck3-user" -W \ -b "dc=powerflex,dc=lab,DC=domain,dc=com" "(&(objectClass=group)(|(cn=FLEXLAB-APP-PROD-SDS-Admin)(cn=EMEALabAdmins)))" cn 

dn: CN=EMEALabAdmins,OU=LabAdmins,DC=powerflex,DC=lab,DC=domain,DC=com
sAMAccountName: Administrator

member: CN=ck3-builder,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
sAMAccountName: name member: CN=Name ,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com sAMAccountName: username member: CN=name,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com sAMAccountName: bobthebuilder

$ curl -u 'FLEXLAB\ck3-user' "ldap://ldap.powerflex.lab.domain.com:3268/DC=powerflex,DC=lab,DC=domain,DC=com?member?sub?(&(objectClass=group)(cn=EMEALabAdmins))"

 

 

In this example, we are querying LDAP for an existing group (objectClass) using its cn attribute and requesting member value(s).

 

It permits to validate following PFMP settings:

Group Member Attribute: member

Group ID Attribute: cn

Group Object Class: group

Group Search Path: "dc=powerflex,dc=lab,DC=domain,dc=com"

 

Testing Group LDAP Filter:

 $ ldapsearch -LLL -H ldap://powerflex.lab.domain.com:3268 -x -D "FLEXLAB\ck3-user" -W \ -b "dc=powerflex,dc=lab,DC=domain,dc=com" "(&(objectClass=group)(|(cn=FLEXLAB-APP-PROD-SDS-Admin)(cn=EMEALabAdmins)))" cn dn:
CN=EMEALabAdmins,OU=LabAdmins,DC=powerflex,DC=lab,DC=domain,DC=com 
cn:EMEALabAdmins

dn:CN=FLEXLAB-APP-PROD-SDS-Admin,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
cn:FLEXLAB-APP-PROD-SDS-Admin

$ curl -u 'FLEXLAB\ck3-user' "ldap://ldap.powerflex.lab.domain.com:3268/DC=powerflex,DC=lab,DC=domain,DC=com?cn?sub?(&(objectClass=group)(|(cn=FLEXLAB-APP-PROD-SDS-Admin)(cn=EMEALabAdmins)))"

 

Testing User LDAP Filter:

$ ldapsearch -LLL -H ldap://powerflex.lab.domain.com:3268 -x -D "FLEXLAB\ck3-user" -W -b "dc=powerflex,dc=lab,DC=domain,dc=com" \
"(&(objectCategory=Person)(sAMAccountName=*)(|(memberOf=CN=FLEXLAB-APP-PROD-SDS-Admin,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com)\(memberOf=CN=LabAdmins,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com)))" sAMAccountName

dn: CN=Administrator,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
sAMAccountName:Administrator

dn: CN=name,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
sAMAccountName:name

dn: CN=Name,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
sAMAccountName: username

dn: CN=bobthebuilder,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com
sAMAccountName: bobthebuilder

 

 

$ curl -u 'FLEXLAB\ck3-user' "ldap://ldap.powerflex.lab.domain.com:3268/DC=powerflex,DC=lab,DC=domain,DC=com?sAMAccountName?sub?(&(objectCategory=Person)(sAMAccountName=*)(|(memberOf=CN=FLEXLAB-APP-PROD-SDS-Admin,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com)(memberOf=CN=LabAdmins,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com)))"

 

Windows

Launch Active Directory Users and Computers. Click View and select Advanced Features.

The DN is the name that uniquely identifies an entry in the directory. The first component of the DN is referred to as the Relative Distinguished Name (RDN).

A DN is made up of attribute=value pairs, separated by commas, for example:

CN=PowerFlex Admin,OU=sio-group1,DC=powerflex,DC=lab,DC=domain,DC=com

Think of them as "FQDNs for LDAP," the complete x500 address 'URL'.

AD Users and Computers can be used to determine DNs and other attributes:

 

 

Navigate and right-click the OU where you want to read users, then select Properties.

In the OU Properties, select the Attribute Editor tab.

Click distinguishedName to highlight it, then click View.

Right-click the highlighted value and select Copy. 

Click Cancel and then OK to close the Attribute Editor and OU Properties windows.

 

 

 

 

Narrow scope LDAP User Filter example:

 

(&(objectCategory=Person)(sAMAccountName=*)(|(memberOf=CN=FLEXLAB-APP-PROD-SDS-Admin,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com)(memberOf=CN=LabAdmins,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com)))

 

 

Visual explanation:

 

Bind User Requirements.

 

This bind user must at least have permission to query users and groups that are defined in the LDAP server to allow storage system to authenticate these users.

 

By default, when a user is added to the security tab of the top-level domain in Active Directory Users and Computers, it is given the default read-only requirements needed for LDAP auth in PFMP. e.g.

 

 

 

 

FYI: We do not need our LDAP bind user actually 'in' the AD groups we must read in the directory, we need the Security tab permissions of "Read."

 

PFMP LDAP Provider General Requirements

 

Address: ldaps://ad.ldap.domain:3269 (SSL) OR ldap://ad.ldap.domain:3268 (PLAIN)

Bind DN: ADNTDOMAIN\serviceaccount or Bind DN Distinguished name.

Bind DN Password: service account password

Timeout 30000

Username LDAP attribute: sAMAccountName

ID attribute: objectGUID

Object Class: Person

Search Path: (Highest level that can query all groups, e.g. only the Domain Components/DCs: DC=powerflex,DC=lab,DC=domain,DC=com)

Group Member Attribute: member

Group ID Attribute: cn

Group Object Class: group

Group Search Path:(Same as User Search Path or more granular if needed.  Since we will be using a narrow filter, our scope can be wide without impact to performance.)

 

#The following entry provides at least one group to the LDAP Group filter (PFMP 4.6.x+), so that keycloak does not search for all groups from the entire directory:

Group LDAP Filter: (CN=GROUP1)

#Given the 64 char limitation in this field for PFMP UI only, we must use the keycloak to work around it if we want to add more than that, .e.g 3 different groups:
 

Group LDAP Filter: (|(CN=GROUP1)(CN=GROUP2)(CN=GROUP3))

Test Connection and if successful Click Submit.

E.g.

 

 

2. LDAP Provider Removal and Cleanup

 

Remove the LDAP users, groups, and then the Directory Provider from the PFMP UI as the admin user.  Do not try to do this as an LDAP user, only as admin.

 

WARNING: Make sure to record the current configuration for "Directory Provider" and "Remote Users/Groups" mapping from PFMP (in particular if you need to restore previous role mapping).

If group/role mapping is not displayed on PFMP (known issue), use following instructions:

# from any mvm
cmo_pri=$(kubectl get pods -n powerflex -l='postgres-operator.crunchydata.com/role=master,postgres-operator.crunchydata.com/instance-set' -o name)

# Group role
kubectl exec -it -n powerflex $cmo_pri -- psql -U postgres -d keycloak -c "SELECT keycloak_group.name AS group_name,keycloak_role.name AS role_name FROM group_role_mapping JOIN keycloak_group ON (group_role_mapping.group_id = keycloak_group.id) JOIN keycloak_role ON (group_role_mapping.role_id = keycloak_role.id);"

# Group attributes
kubectl exec -it -n powerflex $cmo_pri -- psql -U postgres -d keycloak -c "SELECT keycloak_group.name AS group_name,group_attribute.name AS attribute_name, value FROM group_attribute JOIN keycloak_group ON (group_attribute.group_id = keycloak_group.id);"

Example:
mvm01:~
# kubectl exec -it -n powerflex $cmo_pri -- psql -U postgres -d keycloak

keycloak=# SELECT keycloak_group.name AS group_name,keycloak_role.name AS role_name FROM group_role_mapping JOIN keycloak_group ON (group_role_mapping.group_id = keycloak_group.id) JOIN keycloak_role ON (group_role_mapping.role_id = keycloak_role.id);
 
 group_name  |   role_name
---------------+----------------
EMEALabAdmins | SuperUser
LabAdmins     | LifecycleAdmin
LabUsers      | Monitor (3 rows) keycloak=# SELECT keycloak_group.name AS group_name,group_attribute.name AS attribute_name, value FROM group_attribute JOIN keycloak_group ON (group_attribute.group_id = keycloak_group.id);  group_name  | attribute_name |  value ---------------+----------------+---------
EMEALabAdmins | remote_type    | Ldap EMEALabAdmins | SuperUser      | GLB:GLB EMEALabAdmins | is_remote      | true LabAdmins     | remote_type    | Ldap LabAdmins     | LifecycleAdmin | GLB:GLB LabAdmins     | is_remote      | true LabUsers      | remote_type    | Ldap LabUsers      | Monitor        | GLB:GLB LabUsers      | is_remote      | true (9 rows)
keycloak=# \q

 

 

If any stale LDAP groups remain in the keycloak database, they can be deleted manually using the following process:

 

Keycloak database backup and keycloak_group database table clean up commands.

 

Note: Manually cleanup keycloak is sometimes not needed. 

We should use the PFMP UI to remove the provider (always from PFMP first), then confirm it has been cleared from keycloak using /auth/ URL.

 

 

How to check the LDAP group count in keycloak from any one MVM:

 

# Run this to set the variable first.

cmo_pri=$(kubectl get pods -n powerflex -l='postgres-operator.crunchydata.com/role=master,postgres-operator.crunchydata.com/instance-set' -o name)

#check count of groups in keycloak
kubectl exec -it -n powerflex $cmo_pri -- psql -U postgres -d keycloak -c "SELECT COUNT (*) FROM keycloak_group;"

# We usually want 0 groups after removing the LDAP provider.
#Backup the current keycloak DB before any edits.
kubectl exec -it -n powerflex $cmo_pri -- bash -c 'pg_dump -U postgres -d keycloak' >/home/delladmin/keycloak.sql

#delete all entries inside of keycloak_group (and related roles and attributes) while preserving its structure
kubectl exec -it -n powerflex $cmo_pri -- psql -U postgres -d keycloak -c "DELETE FROM group_attribute;"
kubectl exec -it -n powerflex $cmo_pri -- psql -U postgres -d keycloak -c "DELETE FROM group_role_mapping;"
kubectl exec -it -n powerflex $cmo_pri -- psql -U postgres -d keycloak -c "DELETE FROM keycloak_group;"

#check that the group count has reduced:
kubectl exec -it -n powerflex $cmo_pri -- psql -U postgres -d keycloak -c "SELECT COUNT (*) FROM keycloak_group;"

 

This is only after first removing in PFMP UI, and only if the provider data remains in the backend.  If you are using  PFxM 4.6.0.1, you can also perform the following method to remove imported users in the Keycloak UI:

1. Log in using /auth/ as keycloak user
2. Keycloak password is unique for each install and can be found on any MVM using:

 

kubectl get secret -n powerflex keycloak-admin-credentials -o jsonpath="{.data.password}" | base64 --decode; echo

 

3. User federation -> LDAP -> Settings and in the upper right Actions drop-down select Remove Imported.

 

 

 

 

 

 

With the groups and directory providers cleaned up and removed, reboot the three MVM nodes. 

 

 - Note: This is only required when the environment is large, and LDAP queries stack up repeatedly as seen in this case in the logs below.

 - If the environment is not overloaded with sync issues and the UI is stable, the MVM reboots can generally be skipped. It is a good measure in general though.

 

Reboot KB https://www.domain.com/support/kbdoc/en-us/000225550 

MVM Graceful Reboot (Must be ran one node at a time) Do not move to the next node until you verify that the database is reporting Sync Standby.

 

#As a best practice identify the node that is the current Postgres leader and reboot this one last.

#Run the following command on any MVM to list Database status and database names associated.

kubectl exec -n powerflex -c database $(kubectl get pods -n powerflex -l='postgres-operator.crunchydata.com/role=master, postgres-operator.crunchydata.com/instance-set' | grep Running | cut -d' ' -f1) -- sh -c 'patronictl list'

#Run this command to identify which MVM node is hosting the postgres database pod.

 

for x in `kubectl get pods -n powerflex | grep "postgres-ha-cmo" |awk '{print $1}'` ; do echo $x; kubectl get pods -n powerflex $x -o json | grep '"nodeName"' | cut -d ':' -f2 ; echo " "; done

 

#On the MVM that is the Postgres Leader pod instance.   Run 'watch kubectl get nodes' to monitor the status of your reboots. This MVM must be the last node that will be rebooted.

#Example reboot order if pfmp-mgmt-1 is the Postgres Leader. Adjust the order of the commands below depending on which MVM is the Postgres Leader.

 

Open a terminal connected to the third MVM and run the first two commands.

 

kubectl label
node pfmp-mgmt-3 cmo.maintenance.mode=true
kubectl drain pfmp-mgmt-3 --ignore-daemonsets --delete-emptydir-data && sudo reboot

#When the node comes up from reboot. Run the following command.

kubectl uncordon pfmp-mgmt-3 ; kubectl label node pfmp-mgmt-3 cmo.maintenance.mode-

#Run the following command until the database associated with MVM3 reports Sync Standby. At that point you can move to the next MVM.

 

kubectl exec -n powerflex -c database $(kubectl get pods -n powerflex -l='postgres-operator.crunchydata.com/role=master, postgres-operator.crunchydata.com/instance-set' | grep Running | cut -d' ' -f1) -- sh -c 'patronictl list'

 

Open a terminal connected to the second MVM and run the first two commands.

 

kubectl label node pfmp-mgmt-2 cmo.maintenance.mode=true

kubectl drain pfmp-mgmt-2 --ignore-daemonsets --delete-emptydir-data && sudo reboot

#When the node comes up from reboot.  Run the following command.

kubectl uncordon pfmp-mgmt-2 ; kubectl label node pfmp-mgmt-2 cmo.maintenance.mode-

#Run the following command until the database associated with MVM2 reports Sync Standby. At that point you can move to the next MVM.

kubectl exec -n powerflex -c database $(kubectl get pods -n powerflex -l='postgres-operator.crunchydata.com/role=master, postgres-operator.crunchydata.com/instance-set' | grep Running | cut -d' ' -f1) -- sh -c 'patronictl list' 

#Run the following command to use this MVM as the node status monitoring MVM.

'watch kubectl get nodes'

 

Go to the terminal connected to the first MVM that is running 'watch kubectl get nodes'.  Press ctrl+c to go back to the terminal and run the first two commands.

kubectl label
node pfmp-mgmt-1 cmo.maintenance.mode=true

kubectl drain pfmp-mgmt-1 --ignore-daemonsets --delete-emptydir-data && sudo reboot

#When the node comes up from reboot.  Run the following command.

kubectl uncordon pfmp-mgmt-1 ; kubectl label node pfmp-mgmt-1 cmo.maintenance.mode-

#Run the following command until the database associated with MVM1 reports Sync Standby.  This MVM was the leader, but a new leader would take over when MVM1 was drained.  

kubectl exec -n powerflex -c database $(kubectl get pods -n powerflex -l='postgres-operator.crunchydata.com/role=master, postgres-operator.crunchydata.com/instance-set' | grep Running | cut -d' ' -f1) -- sh -c 'patronictl list'

END OF MVM GRACEFUL REBOOT PROCEDURE

Once rebooted and the UI is restored for PFMP, login as admin and add Settings -> Directory Provider using the steps below:

3. LDAP Provider Rebuild

 

As mentioned in the preparation section, If there are child AD domains, or AD domain trust accounts in any groups you which to utilize in PFMP, then the global catalog (GC) LDAP ports 3268 (plain) and 3269 (SSL) should be used.

 

E.g.

 

ldaps://ad.ldap.corporate.domain.com:3269
Users DN: DC=powerflex,DC=lab,DC=domain,DC=com
Bind DN:DOMAIN\svc_ldapbindaccount 
Username LDAP attribute: sAMAccountName
UUID LDAP attribute: objectGUID
User Object Classes: Person
This field has a 64 character limit in PFMP UI only.   You can put one group in to prevent 'full' directory sync, then use the keycloak UI to refine and add on to the group filter.
Group LDAP Filter: (|(CN=FLEXLAB-APP-PROD-SDS-Admin)(CN=LabAdmins))
Timeout 22222

4. Narrow LDAP user and group scope application

 

 

 

Log in using /auth/ as keycloak user

User Filter:

 

 

 

# User LDAP Filter is missing from PFMP UI so we need to use the following to change that.

#This filter must be applied in the Keycloak PowerFlex realm (PowerFlex --> User federation --> User LDAP filter)

 

 

 

User LDAP filter:

 

 

(&(objectCategory=Person)(sAMAccountName=*)(|(memberOf=CN=FLEXLAB-APP-PROD-SDS-Admin,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com)(memberOf=CN=LabAdmins,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com)(memberOf=CN=LabUsers,CN=Users,DC=powerflex,DC=lab,DC=domain,DC=com)))

 

 

 

Save the change.

 

 

 

Explanation of this narrowed filter:

 

 

 

objectCategory equals Person.

And

sAMAccountName exists

And

memberOf equals CN=PowerFlex Admin,OU=sio-group1,DC=powerflex,DC=lab,DC=domain,DC=com

Or

memberOf equals CN=FLEX-STOR-MON,OU=15G,OU=FLEXLAB,DC=powerflex,DC=lab,DC=domain,DC=com

From <https://ldap-builder.powerflex.lab.domain.com/>

User federation -> LDAP -> Settings and in the upper right Actions drop-down select remove imported.

 

4.6.x+ UI Method to delete group mapping.

 

In the LDAP Group Mapping setting for 'Membership User LDAP Attribute', we also need to ensure a correct value we ship as 'cn' and cannot change in PFMP UI is changed in keycloak UI to DN.

 

 

 

In the group LDAP filter, we can list an either/or operator |   and the CNs of each group, e.g.

(|(CN=FLEXLAB-APP-PROD-SDS-Admin)(CN=CN=PRV_US_SA_SRV_SVS))

 

PowerFlex rack, VxFlex Ready Nodes, PowerFlex custom node, PowerFlex appliance R650, PowerFlex appliance R6525, PowerFlex appliance R660, PowerFlex appliance R6625, Powerflex appliance R750, PowerFlex appliance R760, PowerFlex appliance R7625 , PowerFlex custom node, PowerFlex custom node R650, PowerFlex custom node R6525, PowerFlex custom node R660, PowerFlex custom node R6625, PowerFlex custom node R750, PowerFlex custom node R760, PowerFlex custom node R7625, PowerFlex custom node R860, PowerFlex rack connectivity, PowerFlex rack HW, PowerFlex rack RCM Software, PowerFlex Software, VxFlex Ready Node, VxFlex Ready Node R640, VxFlex Ready Node R740xd, PowerFlex appliance R640, PowerFlex appliance R740XD, PowerFlex appliance R7525, PowerFlex appliance R840, VxFlex Ready Node R840 ... View More View Less