FPClip open performance

Question

Hi, I am using the 3.2 SDK against a legacy Gen2 Centera server, which is running on an old 2.3 OS. I need to open clips to gather the metadata ( just the clip attributes, I don't care about the Tag/blob data at this point ). I am currently using the API: FPClip fpClip = new FPClip( fpPool, clipId, FPLibraryConstants.FP_OPEN_FLAT ); I am seeing some pretty inconsistent response time from the server. I tested fetching about 100000 clips ( with 10 threads shared a single Pool Connection ). In one test it took about 50 mins, but in the other one with same set of clipIds, it took 6 hours to finish... So just want to see if you can comment/advise on if this is the efficient way to open the clip to get the metadata? In my other tests with more clipIds to fetch, I noticed the response time degrades over time... that is, the first 50000 clips-open returned pretty fast, but after 100000 clips ( which got fetched in under an hour ), the clip open response time degrades quite significantly... that is, it took about 2 hours to open the next 60, 000 clips, and another 6 HOURS to open the next 150000 clips... So in total, it took 9 hours to open ~300,000 clips ( with 10 threads ) thanks for any advice/comments. P.S., the clip processing is fast and trivial. It just prints out some attributes like name, creationDate, etc.. and a clip is always closed after processed. I'd been monitoring the memory usage too, and it's very stable...

mckeown_paul · Answer

A couple of points

THe rev of CentraStar that the GEN2 cluster is running is EOSL and should be upgraded. There are many improvements internally that will make the location and opening of the CDF much more efficient.

I havent tried doiong what you are doing but I would guess a single thread could open maybe 5 - 10 CDFs per second (for sure I am not out by an order of magnitude) however I don't know why with time you are seeing performance slow.

At the moment I would recommend you get this cluster up graded to the latest rev of C*. A service call should be able to get this done for you. It pointless looking for credible performance stats out of your configuration with unsupported revs of C*, particularly when there are many improvements in C* 4.x

wo2 · Answer

Hi Paul,

thanks for the comments. it's a legacy system and unfortunately upgading the system is not an option for us...but yeah, we'd involved the service to do some health check on the system.

Thanks again.

Centera

Was this post helpful?