Ask the Expert: AIMA: Everything you wanted to know but were afraid to ask.

what is AIMA ?

This Ask the Expert event is now open for comments and questions. We look forward to an interesting and informative discussion!

Hello Dynamox,

I'm glad you asked that. If you can give me about 15 minutes or so, I am about to post an intro which will, I hope, answer that question and possibly others and maybe provoke further questions.

Thanks,

Tim

It seems that lsass(d) in 6.5 never caches queries from an LDAP server.

Run (on the cluster) stat on 50 files, and 200 full queries, with one new connection per query,

are sent to the LDAP server within a second or so -- even if only a few distinct

user/group names appear listed as  file owners.

Sporadicly our cluster (and others have reported the same here)

complains about the LDAP server accepting no (more) connections.

Quite understandable with such flooding.

How about better caching? Any options, or am I missing something?

Are there improvements in 7.0?

Or is id caching regarded as a problem when looking at the whole cluster?

Could be possible, but not fully obvious. Considering that queries to the

LDAP server are asynchronous (from multiple nodes)  anyway,

a few seconds shouldn't do any harm I think; and would help a lot in

the above example.

-- Peter

Hi Peter,

sorry for the lack of clarity there. Yes, by 7.x I mean 7.0.1.x or 7.0.2.x or 7.1.x when it ships.

Current GA versions are, I believe 7.0.1.8 for the 7.0.1 maintenance release train and 7.0.2.1 for the 7.0.2 train. The caching was added in 7.0 and will exist in all future versions.

Regards,

Tim

Tim, thanks a lot!

7.x means starting from 7.0, or "some" x?

Cheers

-- Peter

Super -- thanks again!

Hello Peter,

you are quite correct. The LDAP provider in 6.5.x does not cache lookups. Happily, I can report that lsass performs caching for both NIS and LDAP in OneFS 7.x in addition to the caching it has always performed for Active Directory.

By default, the id cache size is 4.77MB and can be modified if required:

# isi zone zones modify --cache-size=

The cache lifetime for both NIS and for LDAP is 15 minutes by default. The duration can also be modified using:

isi auth ldap modify --cache-entry-expiry=

Your latter point is also important. As implemented today, each node has its own instance of lsass, and its own cache etc. The idmap database for mappings is shared, but there is no globally coherent cache. That obviously means that if you have a large cluster, each node will be performing lookups to the auth providers. Given that in general, a user will connect to only one node at a time, and that the lookups are cached, this should not cause a dramatic increase in traffic, but I do plan to talk to Engineering about where it would be feasible to have some degree of collaboration between the nodes. This would not be trivial to achieve but it certainly warrants consideration.>

Regards,

Tim

OK, today I'd like to talk a little about how the auth token is generated, and one tip for an idmapper rule that may be helpful when managing access by group.

As previously mentioned, AIMA allows you to configure authentication providers such as Active Directory, LDAP etc.

In OneFS 6.5, there was a single list of providers as a hidden config parameter. With the advent of authentication zones in OneFS 7.0, each zone has its own set of providers. They can be added in any order but, generally speaking, the default order should not be changed. In particular, on 6.5, the idmapper provider (ID) is at the start of the list and should never be moved (this was addressed in 7.0).

The default order in 7.0 is:

ActiveDirectory,Local,File,Ldap,Nis

Again, as previously mentioned, there is an assymmetry in the design because non-kerberized NFSv3 clients don't actually perform authentication, and so everything is tailored to allow raw uids and gids to give correct access. The ActiveDirectory provider at the head of the list is "special" insofar as it has an implicit callout to lookup the domain user as a "Unix" user in the other providers. This means that, if you do not disable lookup-users, a user "testuser1" who exists in AD and LDAP will receive a token that contains both the Windows and Unix identities, with the Unix uid as the on-disk identity:

isi7014-1# isi auth mapping token --user='VMTEST\testuser1'

        User

          Name : VMTEST\testuser1

            UID : 1001

            SID : S-1-5-21-4087762976-3323942327-742857095-1118

          ZID: 1

        Zone: System

  Privileges: -

Primary Group

          Name : VMTEST\domain users

            GID : 1000008

            SID : S-1-5-21-4087762976-3323942327-742857095-513

Other Groups

          Name : Users

            GID : 1545

            SID : S-1-5-32-545

          Name : testgroup1

            GID : 1001

            SID : S-1-22-2-1001

Note that for the User, I have both a valid SID and also a UID. However, my primary group is a SID. This is because my primary group in AD is "Domain Users" which doesn't exist in my LDAP provider. If I want the Windows user to pick up my Unix primary group when there is a matching user, I need to add a small mapping rule:

isi7014-1# isi zone zones modify --zone=System --add-user-mapping-rules='VMTEST\* += * [group]'

We will talk later about the idmapper. What that rule does is find users in the VMTEST domain, and insert (+=) the primary group ([group]) from the matching Unix user.

Now let's look at the token once more:

isi7014-1# isi auth mapping token --user='VMTEST\testuser1'

         User

           Name : testuser1

            UID : 1001

            SID : S-1-5-21-4087762976-3323942327-742857095-1118

          ZID: 1

         Zone: System

   Privileges: -

Primary Group

           Name : testgroup1

            GID : 1001

            SID : S-1-22-2-1001

Other Groups

           Name : VMTEST\domain users

            GID : 1000008

            SID : S-1-5-21-4087762976-3323942327-742857095-513

           Name : Users

            GID : 1545

            SID : S-1-5-32-545

Notice how my primary group is now Unix group 1001 *testgroup1) and the 'VMTEST\Domain Users' group has been pushed out as a secondary.

That's probably enough for one posting. I welcome comments, questions, complaints, etc. :-)

Do I get it correctly, [group] is not a placeholder to be filled in by the CLI admin user,

but a literal that pulls "the group" from any(?) later provider?

-- Peter

So yesterday, I mentioned that if you want to manage group access using the Unix groups, you can simply force the primary Unix group to be the primary (on-disk) group for the Windows user.

Conversely, if you want the AD groups to be used and you want NFS users to have access, there is a global or per-export NFS setting that will help. This setting is the "map-lookup-uid" setting that can be set in the UI of from the CLI for an individual export or for the default export.

When you enable "map-lookup-uid" on an NFS export, the way that incoming RPCs are treated changes dramatically. Instead of taking the uid, primary gid and up-to-16 supplementary groups that are sent over the wire, OneFS does the following:

• Takes the incoming uid and performs a reverse-lookup to find a name.
• Takes the name and looks up the name in the default/primary domain to get the Windows identities for that name.
• In doing the lookup in the AD provider, that also performs the implicit callout to lookup the name in the Unix auth providers.

The upshot of this is that the NFS server acting on behalf of the client is now running with an auth token that is the same as if the domain user was accessing the cluster via SMB. This actually has two interesting efffects:

1. The NFS user has all of the Windows groups in its access token and can this access data that e.g. is governed by an ACL that only permits access to a group that is found only in AD and not in LDAP/NIS.
2. The other side-effect. If the user is a member of more than 16 supplementary Unix groups, these will all be picked up and added to the token.

Both configurations are valid and it comes down to the end-user which one is preferable.

Hi Peter,

yes, the CLI syntax for the user mapper allows you to pick and choose which elements of the RHS of the map you wish to pull when doing an insert or append. In this case,"+=" is the insert operation and the '[group]' qualifier means it is pulling the primary group from a match.

When matching, an id of the form "DOMAIN\*" we will look in AD for a domain user, and when matching "*", we will lookup in what I tend to call a Unix provider, i.e. LDAP or NIS (or a local user from the cluster).

I will see if I can find when we expect to publish the usermapper guide as that covers it in a lot of detail.

Tim

Hi Tim,

your explanations is really superb. thanks a lot.

I would ask you about another szenario.

We asume we have local NFS Users (without kerberos or ldap) and on the same dir we would have access over smb with ad authentication for smb users. so we need an construct with one or more groups where the lokal nfs user and the ad user are put together to have access to the same data. (some user should able to read only, other users should be able to have rw access)

For my opinion, this could not be a AD group, because AD do not know about the local users? Could i add AD Users to a local group?

Is this possible, without matching user on both sides???

regards Uwe

Thank you, Tim. I'm looking forward to seeing the guide published.

-- Peter