Patrick_Kane
2 Bronze

32-bit Linux applications on Isilon question

Jump to solution

We have a site with some 32-bit Linux server running applications.

When they running applications will show error.

After we use 32-bit file IDs over NFSv3, those applications can run smoothly.

My question is…

What should we pay more attention when we use 32-bit file IDs over NFS?

Does it will limit all 64-bit file IDs environment? Connections or file size will be impact too?

Tags (2)
18 Replies
Highlighted
MRWA
2 Iron

Re: 32-bit Linux applications on Isilon question

Jump to solution

Isilon uses 64 bit by default.  Any NFS export that should use 32 bit would need to be specified to only use 32 bit.

You could setup a 32 bit Export and another with the default of 64 bit.

It's no longer a sysclt (sysctl vfs.nfsrv.do_32bit_fileid=1) setting in 6.5.X. You have use the isi nfs cmd and it's should be persistent on reboots unlike before you have to put it in the mcp override sysclt.conf file.

If you want to set the options for all exports, use the '-s' flag as follows. For a specific export, replace '-s' with '-i'. You can find the rule ID for a given export with 'isi nfs exports list'

# isi nfs exports modify -s --return-32bit-file-ids=yes

Hope this helps,

dynamox
6 Thallium

Re: 32-bit Linux applications on Isilon question

Jump to solution

this bit is so many times, especially with Apache websites, some PHP.

0 Kudos
ICPSR
1 Nickel

Re: 32-bit Linux applications on Isilon question

Jump to solution

this caused problems for us with Acrobat Reader (acroread) and Firefox.  Both are distributed as 32-bit binaries.

Also, warning, do not change to 32-bit file IDs on an active export, or else (all? Most?) existing client filehandles will become very sad.  Guess how I figured this out.

0 Kudos
dynamox
6 Thallium

Re: 32-bit Linux applications on Isilon question

Jump to solution

when you say active, you mean you had clients actively accessing it ? I had my NFS clients unmount the export, i made the change and only then they remounted it. I did not not have to re-create the export.

0 Kudos
ICPSR
1 Nickel

Re: 32-bit Linux applications on Isilon question

Jump to solution

Yes, i had clients actively accessing it.  duhhh.  i will definitely be shamed at our next staff meeting.  In an early-production/pre-production situation, hurting only computing staff home directories, thankfully. 

0 Kudos
tim_wright
1 Nickel

Re: 32-bit Linux applications on Isilon question

Jump to solution

I should add a little additional clarification. It's not simply that the client is 32-bit, but more particularly, that the revision of the filesystem API (particularly the stat() system call) doesn't support 64-bit inode numbers on the client.

A frequent source of confusion is between fileids and filehandles. They are not the same. A filehandle is an opaque data type representing a file to an NFS client/server. A fileid is defined in the NFS RPC specifications and represents, effectively, and inode number. It is only returned by two of the NFS RPCs: GETATTR (stat) and READDIRPLUS (used by some clients).

Why all of the above details? Well, modern 32-bit Linux clients running a modern 32-bit C library (libc) have no issues with 64-bit inode numbers provided the code is compiled with "LARGEFILE" support enabled. However, much older versions of Linux code as well as earlier revisions of Unix OSes such as AIX are restricted to 32-bit inode numbers, and that is when you need to change the setting so we can return truncated fileid values.

Regards,

Tim

Peter_Sero
4 Beryllium

Re: 32-bit Linux applications on Isilon question

Jump to solution

A typical symptom is the error message "Value too large for defined data type"

which is issued by applications running on the NFS client.

More specifically, this exact wording is provided by the UNIX/C library

when the 32-bit stat() system call fails on 64-bit replies from the server.


It took us some time to figure this out when an application did spit out

just this message without any context... But I've seen it on several

occasions since then...

-- Peter

0 Kudos
sdrf
1 Copper

Re: 32-bit Linux applications on Isilon question

Jump to solution

Do you know if by using this setting there is a potential risk of accessing the wrong file giving the fact that the client app receives only a portion of the 64-bit inode number, which could cause two or more files report as having the same 32-bit portion of the inode number?  Or how does this really work?

We have a customer who has enabled the variable after having many files on the Isilon file system already.  Are the existing files affected somehow?

Appreciate your comments.

Thanks.

0 Kudos
tim_wright
1 Nickel

Re: 32-bit Linux applications on Isilon question

Jump to solution

Hello sdrf,

that is a good question. The answer is no. The reason is the following.

Per my comment #6, setting that setting affects the returned inode number in two of the rpcs.

It doesn't affect the NFS filehandle that is returned by the server and used by the client to reference the file/directory. That value is still unique. You can't open a file by inode number over NFS, only by name, so there isn't any risk of accessing the wrong file. The only possible issue is if an application is coded to use the inode number returned in e.g. a stat() system call and where it expects that value to be unique across all files on the system at that time. I am not aware of any such application.

Regards,

Tim

0 Kudos