this bit is so many times, especially with Apache websites, some PHP.

this caused problems for us with Acrobat Reader (acroread) and Firefox.  Both are distributed as 32-bit binaries.

Also, warning, do not change to 32-bit file IDs on an active export, or else (all? Most?) existing client filehandles will become very sad.  Guess how I figured this out.

Yes, i had clients actively accessing it.  duhhh.  i will definitely be shamed at our next staff meeting.  In an early-production/pre-production situation, hurting only computing staff home directories, thankfully. 

when you say active, you mean you had clients actively accessing it ? I had my NFS clients unmount the export, i made the change and only then they remounted it. I did not not have to re-create the export.

I should add a little additional clarification. It's not simply that the client is 32-bit, but more particularly, that the revision of the filesystem API (particularly the stat() system call) doesn't support 64-bit inode numbers on the client.

A frequent source of confusion is between fileids and filehandles. They are not the same. A filehandle is an opaque data type representing a file to an NFS client/server. A fileid is defined in the NFS RPC specifications and represents, effectively, and inode number. It is only returned by two of the NFS RPCs: GETATTR (stat) and READDIRPLUS (used by some clients).

Why all of the above details? Well, modern 32-bit Linux clients running a modern 32-bit C library (libc) have no issues with 64-bit inode numbers provided the code is compiled with "LARGEFILE" support enabled. However, much older versions of Linux code as well as earlier revisions of Unix OSes such as AIX are restricted to 32-bit inode numbers, and that is when you need to change the setting so we can return truncated fileid values.

Regards,

Tim

A typical symptom is the error message "Value too large for defined data type"

which is issued by applications running on the NFS client.

More specifically, this exact wording is provided by the UNIX/C library

when the 32-bit stat() system call fails on 64-bit replies from the server.

It took us some time to figure this out when an application did spit out

just this message without any context... But I've seen it on several

occasions since then...

-- Peter

Do you know if by using this setting there is a potential risk of accessing the wrong file giving the fact that the client app receives only a portion of the 64-bit inode number, which could cause two or more files report as having the same 32-bit portion of the inode number?  Or how does this really work?

We have a customer who has enabled the variable after having many files on the Isilon file system already.  Are the existing files affected somehow?

Appreciate your comments.

Thanks.

Hello sdrf,

that is a good question. The answer is no. The reason is the following.

Per my comment #6, setting that setting affects the returned inode number in two of the rpcs.

It doesn't affect the NFS filehandle that is returned by the server and used by the client to reference the file/directory. That value is still unique. You can't open a file by inode number over NFS, only by name, so there isn't any risk of accessing the wrong file. The only possible issue is if an application is coded to use the inode number returned in e.g. a stat() system call and where it expects that value to be unique across all files on the system at that time. I am not aware of any such application.

Regards,

Tim

Tim and Peter,

How to find out what does my client server support, 32 or 64 bit ?

Is it the same with celerra as well ? I really couldn't find a setting like this on celerra and i guess it goes with default value of 32 bit. I am moving data from celerra to Isilon and i came across couple of applications that needs export to have 32 bit fileid's, after spending lot of time

Here are two ways to check, both at the kernel and userspace.

1. For the kernel, run: uname -i

$ uname -i

x86_64 <=====64-bit. Some distros would say amd64

$ uname -i

i686 <=======32-bit. Other 32-bit labels include i386, i586, x86_32

2. For the userspace application, this example is for ls(1)

# file /bin/ls

/bin/ls: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, stripped

-vs-

/bin/ls: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), stripped

Keep in mind you could have a 64-bit kernel with a 32-bit userspace application, but not the other way around.

Thanks Mark,

I noticed this from server and application running the commands mentioned, though its a 64 bit OS and app, they dint work on 64 bit file ids and so export file ids has to be changed to 32 bit

This is the error message noticed by app team in logs

Running in PROCESS MODE.

Value Memory Info [Size = 7 KB ( 1 Pages @ 7936 Bytes/Page), Swapped? N, Max Memory = 144289 KB ( 18618 Pages)].

send STATUS 170 to job 972679

Session 972679 ended at Sat Aug 23 14:57:50 2014

        with error cd 170

972679  elapsed time(ms) real = 0 , user = 0, system=0.

If still confused, the answer is in Tim's post (6) above. A 64-bit application can use system calls that pass around *data* in a 32-bit layout, and one can't predict this from the OS and APP infos you found.

An experienced programmer might use tools like nm and trace to inspect the application binary and process to see what's going on.

The software manufacturer should also be able to provide this information (if the support hotline can grasp the essence of the question, that is). At least there should be some documentation on "error code 170".

-- Peter

Thanks Peter

Here is the issue i am facing with 9 bit inode values on server side. The inode value highlighted in red is of 9 bit which gets unwanted permissions (rw-rw----) and other files that has 6 bit inode values have right permissions (rw-rw-r--). Is this an issue from storage side as storage ?

Hello Rdamal,

I presume the listing above is from the client's perspective?

How do these files get created? Who/what is responsible for setting the permissions on the files?

Have you tried changing the permissions?

If you are seeing unexpected permissions on files from NFS and attempts and chmod do not result in the expected permissions, please check the permissions on the cluster. Specifically, can you run both "ls -led" and "ls -lend" on the file to view the permissions on the cluster. If the file has an ACL and you are using NFSv3, you're not going to be able to see the full permissions through NFS, and dependent on the "chmod on files with ACLs" ACL policy setting, the permissions after chmod may not be what you expect.

Regards,

Tim