Patrick_Kane

updated

14 years ago

P

Patrick_Kane

1 Rookie

•

34 Posts

2

13297

January 10th, 2013 10:00

32-bit Linux applications on Isilon question

We have a site with some 32-bit Linux server running applications.

When they running applications will show error.

After we use 32-bit file IDs over NFSv3, those applications can run smoothly.

My question is…

What should we pay more attention when we use 32-bit file IDs over NFS?

Does it will limit all 64-bit file IDs environment? Connections or file size will be impact too?

  • MRWA

    1 Rookie

    •

    83 Posts

    4653

    1

    Posted January 10th, 2013 10:00

    Isilon uses 64 bit by default.  Any NFS export that should use 32 bit would need to be specified to only use 32 bit.

    You could setup a 32 bit Export and another with the default of 64 bit.

    It's no longer a sysclt (sysctl vfs.nfsrv.do_32bit_fileid=1) setting in 6.5.X. You have use the isi nfs cmd and it's should be persistent on reboots unlike before you have to put it in the mcp override sysclt.conf file.

    If you want to set the options for all exports, use the '-s' flag as follows. For a specific export, replace '-s' with '-i'. You can find the rule ID for a given export with 'isi nfs exports list'

    # isi nfs exports modify -s --return-32bit-file-ids=yes

    Hope this helps,

  • dynamox

    11 Legend

    •

    20419 Posts

    •

    87439 Points

    4653

    0

    Posted January 15th, 2013 14:00

    this bit is so many times, especially with Apache websites, some PHP.

  • ICPSR

    10 Posts

    4653

    0

    Posted October 21st, 2013 13:00

    this caused problems for us with Acrobat Reader (acroread) and Firefox.  Both are distributed as 32-bit binaries.

    Also, warning, do not change to 32-bit file IDs on an active export, or else (all? Most?) existing client filehandles will become very sad.  Guess how I figured this out.

  • ICPSR

    10 Posts

    4653

    0

    Posted October 21st, 2013 14:00

    Yes, i had clients actively accessing it.  duhhh.  i will definitely be shamed at our next staff meeting.  In an early-production/pre-production situation, hurting only computing staff home directories, thankfully. 

  • dynamox

    11 Legend

    •

    20419 Posts

    •

    87439 Points

    4653

    0

    Posted October 21st, 2013 14:00

    when you say active, you mean you had clients actively accessing it ? I had my NFS clients unmount the export, i made the change and only then they remounted it. I did not not have to re-create the export.

  • tim_wright

    29 Posts

    4653

    2

    Posted October 24th, 2013 08:00

    I should add a little additional clarification. It's not simply that the client is 32-bit, but more particularly, that the revision of the filesystem API (particularly the stat() system call) doesn't support 64-bit inode numbers on the client.

    A frequent source of confusion is between fileids and filehandles. They are not the same. A filehandle is an opaque data type representing a file to an NFS client/server. A fileid is defined in the NFS RPC specifications and represents, effectively, and inode number. It is only returned by two of the NFS RPCs: GETATTR (stat) and READDIRPLUS (used by some clients).

    Why all of the above details? Well, modern 32-bit Linux clients running a modern 32-bit C library (libc) have no issues with 64-bit inode numbers provided the code is compiled with "LARGEFILE" support enabled. However, much older versions of Linux code as well as earlier revisions of Unix OSes such as AIX are restricted to 32-bit inode numbers, and that is when you need to change the setting so we can return truncated fileid values.

    Regards,

    Tim

  • Peter_Sero

    6 Operator

    •

    1169 Posts

    4653

    0

    Posted October 24th, 2013 08:00

    A typical symptom is the error message "Value too large for defined data type"

    which is issued by applications running on the NFS client.

    More specifically, this exact wording is provided by the UNIX/C library

    when the 32-bit stat() system call fails on 64-bit replies from the server.


    It took us some time to figure this out when an application did spit out

    just this message without any context... But I've seen it on several

    occasions since then...

    -- Peter

  • sdrf

    1 Message

    4653

    0

    Posted November 25th, 2013 13:00

    Do you know if by using this setting there is a potential risk of accessing the wrong file giving the fact that the client app receives only a portion of the 64-bit inode number, which could cause two or more files report as having the same 32-bit portion of the inode number?  Or how does this really work?

    We have a customer who has enabled the variable after having many files on the Isilon file system already.  Are the existing files affected somehow?

    Appreciate your comments.

    Thanks.

  • tim_wright

    29 Posts

    4664

    0

    Posted November 25th, 2013 13:00

    Hello sdrf,

    that is a good question. The answer is no. The reason is the following.

    Per my comment #6, setting that setting affects the returned inode number in two of the rpcs.

    It doesn't affect the NFS filehandle that is returned by the server and used by the client to reference the file/directory. That value is still unique. You can't open a file by inode number over NFS, only by name, so there isn't any risk of accessing the wrong file. The only possible issue is if an application is coded to use the inode number returned in e.g. a stat() system call and where it expects that value to be unique across all files on the system at that time. I am not aware of any such application.

    Regards,

    Tim

  • Rdamal

    2 Intern

    •

    165 Posts

    1242

    0

    Posted October 1st, 2014 17:00

    Tim and Peter,

    How to find out what does my client server support, 32 or 64 bit ?

    Is it the same with celerra as well ? I really couldn't find a setting like this on celerra and i guess it goes with default value of 32 bit. I am moving data from celerra to Isilon and i came across couple of applications that needs export to have 32 bit fileid's, after spending lot of time