I've written a performance test harness to insert, retrieve and delete items using the XAM api with .Net. This is being compared to carrying out similar actions against a Sql database (i.e.storing data as datatype image in Sql Server). The harness cycles through a range of file sizes, carrying out each action a specified number of times. I'm finding that each time a stream is created, memory usage increases by approximately that stream size. All stream creation and usage is bounded by a Using statement (i.e. the stream is being disposed), however I find that memory usage grows constantly. I put a manual call to the garbage collector at each filesize change, but it didn't appear to make much difference. I also find that once memory grows to a certain size, the process maxes out the cpu, and has to be killed. This doesn't occur all of the time however.Are the streams being disposed of correctly in the XAM Api?

Also, what other tweaks would you suggest to improve read/write performance? I have set the buffer size and embedded data threshold on the XSystem. Is there a buffer size setting at which performance improvement stops?

I have just run this again, and this time received the error:

Unhandled Exception: XAM Error <1002>: "xam/out of memory" [XSystem_OpenXSet]

Edit: I should point out that this is running on a windows server, and while the Peak Mem Usage stays within reasonable limits, the VM Size continues to grow quite massively - after inserting, retrieving and deleting 100 x 10K and 100 x 40K items, VM size is at ~477MB.

Edit: An additional question - where is file duplication detection set - I've seen in the Centera SDK API ref that there used to be a value FP_OPTION_ENABLE_DUPLICATE_DETECTION which is deprecated, and that duplicate detection is always enabled. Does this then mean that after 100 inserts of the same file in my tests, only 1 file will actually be stored, and all other CDFs will point to the same blob, and therefore this is not a true test?