"Directory" file size and ability to list

Hello there,

I have a few queries on the behavior of a directory object created via the namespace interface.

What I am seeing in the current behavior is that the system metadata always reflects a size of 4096 for a directory object. I am guessing this is the size of soemthing like an inode for the object.

The reason this is important is that I want to get a listing of the objects under this directory; but unless I know how big the listing size is, how can I download this data? The run-time behavior is that a read on the directory object triggers a dump of the entire contents; this could be anywhere from a few hundred bytes to hundreds of MB! How much of a buffer should I allocate before I initiate a read on a directory object?

Further, the directory object does not seem to support a range read; that also triggers a dump of the entire object.

Also, the amount of extraneous data in a directory listing is large compared to the actual data; for every "filename" listed in the data, there is ~100 bytes of other data due to the XML format. Is there a more efficient interface to get the list of names in a directory?

To sum up: What is a good way to list the contents of directories that can span from a few bytes to very large sizes?

thanks,

Navneeth