This post is more than 5 years old
6 Posts
0
10824
Centera file storage Clip Id
Hello,
I have a set of files in a folder.
Each folder has subfolder and in each subfolder there is pdf and xml
xml file has a metadata information like users who can access the pdf file.
how do i store the pdf files from each subfolder into centera extracting the metadata and linking user to the file.
Do i need to store file by file and get the clip id and store it in the local database as a reference to the particular user..
Please provide me with some sample code if any...
pls help me i am really confused..
Ashok
mckeown_paul
409 Posts
0
August 16th, 2010 03:00
As far as the Centera SDK is concerned it is just writing an object with some meta data associated. Any asscoiation it may have had with other objects e.g. its location within a hierarchical file system would need to be stored as metadata within the clip.
Once a clip has been written tot he centera there is no way ro "roll back" the clip other than deleting it and if the clip has retention applied to it then you will not be able to delete it until the retention period is deleted
gstuartemc
417 Posts
1
July 26th, 2010 02:00
The Centera SDK includes C and Java code samples which show how to perform all the common tasks associated with managing content on a Centera.
You would store the pdf file as a blob on an tag within a Centera clip. You can also store the associated xml metadata file as a blob on another tag within the clip, or alternatively you can parse it to extract the required metadata attributes and store these as attributes within the clip.
You would then store the resulting clipID in a database or include it within a "stub" file on your existing file system directory entry.
ashokkumar2
6 Posts
0
August 16th, 2010 02:00
Thank you for response...it really help me to analyse more in detail
I have a project folder (application folder) in which there are 100's of sub folders(requests within the application).
each sub folder will have a .doc file and .xml file.
.xml file has the metadata like user,creation date,request no.
archived data should be accessible only by the respectie users specified in the .xml of each subfolder.
I am really confused ,how can i push the data to centera..
while archiving the application into centera..do i need to go to each subfolder extract metadata and push the .doc file and save the clip Id in local database for the users specified in the xml. In this case if parsing of of xml file fails after processing 5 requests, is it possible to roll back the archived requests?
or can i do a batch process the whole application folder?
Please suggest me with a solution..
ashokkumar2
6 Posts
0
October 18th, 2010 03:00
Thank you for your input.
As i mentioned above that, I have a project folder (application folder) in which there are 100's of sub folders(requests within the application).
each sub folder will have a .doc file and .xml file.
Each time i browse thru subfolder i need to store the file in centera, so for every sub folder i need to open a centera connection and close after writing the files to centera? Is it recommend? will be there any performance problems?
gstuartemc
417 Posts
1
October 18th, 2010 12:00
It is certainly not recommended and you will get very poor performance by doing so! You should open the pool connection once and keep it open until your application shuts down (or errors cause it to be invalidated).
ashokkumar2
6 Posts
0
November 4th, 2010 04:00
Thank you, i implemented as you suggested.
but i think i have one more problem with respect performance.
Archiving process is something like below
I have a application folder in which there are many subfolders ,each subfolder will have few files and metadata related to the files created by.
when i started storing the files, my logic is like open the pool connection.
browse thru each subfolder and store each file by creating a new clip ,get the top tag and finally close the clip.
Here each file is stored in one clip, will it create more object count?
What would be the best way to implement my requirements?
one more problem is : the metadata in subfolder will tell which user has access to which files ,so it clip ids have to maintained user specific in my local database.
Please provide me with some solution
Thanks in advance
gstuartemc
417 Posts
1
November 4th, 2010 04:00
Each clip or blob that you create takes up one object, so storing a single blob in a clip will take up more object than multiple blobs within a clip. This is only an issue if the objects are small (<200KB), but this can be eased by using Embedded Blobs. In this case, objects under the threshold (that you set in your code or via environment variables) are base64 encoded as attribuites of the clip (rather than linked blobs) and thus do not take up an additional object.
This is fully explained in the Programmer's Guide and API Reference Guide. The alternative technique (blob aggregation / containerization) is also discussed.
Regrading the user folder access / metadata issue, this should be handled within a local database (as you seem to suggest that you currently do).
emcstone
9 Posts
0
November 4th, 2010 18:00
I have a case. There are 1 million files(1-10M each file) or more needing to be stored in Centera Cluster. And scheduled a full backup job for them each week.
How should I arrange the storage struct on your rich experience, for example there are two demos:
1. store single file in one clip.
2. store finite number files in one clip, 1000 files each clip for example.
How much difference on reading performance between above?
If the second is better, how many files each clip should i store? In another words, the most suitable count of the blobs in one clip.
Thank you.