Unsolved
This post is more than 5 years old
12 Posts
0
1979
Store a huge amount of small documents at once
Is there a way to collect, let's say 200 files of 50kb each, as one ZIP file and store it on Centera? It should be also possible to retrieve one single file from the ZIP later on. Maybe the single files inside the ZIP could be marked via an Index which is stored in the meta data of the ZIP.
Does the Centera already comes up with such or a similar solution?
clarkhodge
11 Posts
1
May 17th, 2010 06:00
Centera doesn't have this functionality 'built-in'. It's up to the application to present the data to the Centera. The app can send however it would like (single object, containers of objects, segmented objects..) Centera has two two issues with small files - performance, and object count issues.
StorageSwitch provides a file system gateway to Centera (we present Centera as an NFS/CIFS/WebDAV/FTP/... mount. We support 'containerization' of files - like you mention - to significanlty increase performance and to avoid object count issues. There are downsides to containers, in particular records management, and deletion management. Of course when you read a file back - you only get the one you want - not the entire container.
Other apps have this capability too, but are 'locked' to the app.
Feel free to give me a ring - if you'd like to discuss...
----
clark@storageswitch.com
303.859.3321
mckeown_paul
409 Posts
0
May 17th, 2010 06:00
Hi
there is nothing to stop you ziping up content and writing it as one big object container, you would need to understand the zip format yourself though to be able to read back individual files.
If you are not looking to compress the object then you could just build the index yourself and place it as a side xstream to the xstream that is your container. In the index put the filename offset from the BOF and size. You would then look this up when you need to access individual files.
If you need compression then you'll probably have to think of a way to do the same but also compress and decompress your objects
nishiichinoe
12 Posts
0
May 18th, 2010 02:00
Hi Paul,
Thank you for your reply. What do you mean with "side xstream"? My Java code looks basicly like this:
FPClip theClip = new FPClip(thePool, TAG_NAME);
FPTag topTag = theClip.getTopTag();
FPTag newTag = new FPTag(topTag, TAG_NAME);
topTag.Close();
FPFileInputStream inputStream = new FPFileInputStream(oFile);
// write the binary data for this tag to the Centera
newTag.BlobWrite(inputStream, FPLibraryConstants.FP_OPTION_CLIENT_CALCID);
// close the input stream
inputStream.close();
// get the new clip id
String clipID = theClip.Write();
// close the tag and the clip
newTag.Close();
theClip.Close();
How can I attach at the FPFileInputStream a side stream?
Best Regards,
Andreas