Start a Conversation

Unsolved

This post is more than 5 years old

2458

January 12th, 2016 00:00

Single Access Profile, Multiple Pools

Hi,

My dev guys are having a problem with a new app they're developing, and I'm hoping someone else has already solved this particular issue.

We have an new app that is writting images to a Centera cluster, and that cluster is replicated to a second Centera.  The app will also read legacy data written by an old application into the Default Pool, and the Default Pool is replicated bi-directionally.  We need to make sure we are able to write new images even in the event that the primary Centera is not available.  To achieve this, my understanding is that we need to create two separate virtual pools.

Pool1 will be replicated from Primary to Secondary, and will be the home pool for AppProfile1 on the Primary Centera.

Pool2 will be replicated from Secondary to Primary, and will be the home pool for AppProfile1 on the Secondary Centera.

AppProfile1 will be given read privileges to Poo2 on the Primary Centera

AppProfile1 will be given read privileges to Pool1 on the Secondary Centera

During normal operations, all writes will hit Pool1.  If the Primary Centera is offline, we change the connection string for the app to point it to the Secondary Centera, and now all writes hit Pool2.  Once the Primary becomes available again, Pool2 replicates the changes back to the Primary, and we change the connection string to point to the Primary once more.

EMC Support tell me that this should work, and is the recommended configuration.  So far, so good, although if anyone has any other suggestions, please feel free to share them.

Now the problem.  In our Dev environment, we are using the community edition of ECS to emulate the Centera (as we don't have a spare Centera for Test/Dev purposes).  We can read & write just fine to the home pool (default bucket) on the ECS using the CAS API/Centera SDK, but we can't seem to access C-Clips that reside in a pool other than the home pool of the Access Profile.  If I change the home pool of the Access Profile, we are able to read & write C-Clips written to the new pool, but are then unable to access C-Clips we had previously written to the original home pool.

We’re using the .NET wrapper for the Centera SDK and are reading content with the following code (abbreviated for simplicity).

var pool = new FPPool("10.x.x.3?dev.pea");

var clipRef = GetCenteraContentClipRef(pool, contentAddress, FPMisc.OPEN_ASTREE)

If the content is in the same pool that we are writing to (ie the home pool / default bucket) it can be retrieved without issue. If the content is in a different pool we get a Clip not found error, even if our profile specified in the PEA file has read access to that pool.  We have also tried prefixing the contentAddress with the Pool name in the / format as mentioned here ( https://www.emc.com/techpubs/ecs/ecs_create_bucket-1.htm) but we get a fp_parameter_error.

From what I have read, the application should not need to know which pool a clip has been written to, as that would imply some need for the application to manage the storage.  The whole point of Content Addressing is that the appilcation doesn't know or care where the object actually is, right?  I would also assume that the ECS implementation of the CAS API will behave in the same way as the actual Centera, as otherwise we would have a whole bunch of applications needing to be re-written.  If this assumption is wrong, please let me know.

So, has anyone had any success in accessing multiple pools from a single access profile?  If yes, how?

FYI, we're using CentraStar 4.2.0 on the Centeras, ECS 2.1 Community Edition for the Dev CAS, and version 3.3.718 of the Centera SDK on Windows 2012 R2.

Cheers,

Ben

208 Posts

January 12th, 2016 07:00

Hi Ben -

Your design decisions are very sound (optimal even) for 'classic' Centera.  But, as you are experiencing, multi-pool access on ECS doesn't quite work the same way as it does with Centera.  As you may end up migrating to ECS some day, perhaps shrinking things down to a single pool on each cluster would make things simpler.  I believe it would still work fine, as all of the content would be replicated eventually to both systems, no matter where it was written. The only drawback I see is that it would be impossible to tell where a piece of content was originally committed (A or B) since there would be no separation of pools on ingest, but I'm not really sure this matters from a practical standpoint.

Best of luck!

Mike Horgan

9 Posts

January 12th, 2016 08:00

Thanks for your reply Mike, and for taking the time to look at the issue.

What I'm taking away from your post is that what we're trying, and failing, to do in Dev should actually work in our Prod environment?  Specifically the ECS software doesn't allow multi-pool, single profile (user) access, but Centera does?
  That would be a good outcome for us, if not ideal.  Using ECS instead of pure Centera was always a risk, in terms of environment matching etc, so this kind of issue is not unexpected.

As for merging everything down to one pool...  Unfortunately the legacy app that has written all of its data to the default pool also did so with the anonymous profile, so migrating that data to a single virtual pool on the Centera is out of the question.  Or so I'm led to believe, in any case.

A single pool also creates problems for the new app in a DR situation, as I've described in the original post.  The Centera doco is pretty clear on the fact that we can't replicate a single pool in both directions, so writes to the secondary during an outage would not get back the primary in a single pool configuration.

Anyway, sounds like my way forward is to do some dev testing with our prod Centera.  Hope that doesn't break.

Cheers,

Ben

208 Posts

January 12th, 2016 08:00

Hi Ben -

Yes, I do think this will work fine in your prod environment.   I was not aware that bidirectional replication of a single pool is not (any longer?) supported on Centera.

As far as I am aware (have not tested it myself), Centera content migrated to an ECS ignores the metadata embedded in the CDFs and allows you to move content between pools at migration time.   So you would be able to combine your app's data from both the default pool and the multiple app pools into an app-dedicated pool on ECS. Of course, this presumes your migration tool/process is sufficiently sophisticated and flexible, like Interlock's .

Best Regards,

Mike Horgan

1 Message

January 12th, 2016 23:00

Hi Mike,

Thanks again for your time on this.

The Centera doco is pretty clear on the bidirectional replication issue.  From the online help Replication Topologies page:

A pool can be replicated in one direction only.  Do not replicate a replicated pool on the target cluster back to the source cluster.

and

If the source cluster fails, an application can be configured to write to a pool on the target cluster but it must not write to its replicated pool on the target cluster, because the data cannot be repolicated back to the original pool.

That seems clear-cut to me, but perhaps there's somthing I'm missing?

Cheers,

Ben

208 Posts

January 13th, 2016 06:00

Hi Ben -

Clear as a summer day.  Just news to me

Best Regards,

Mike Horgan

No Events found!

Top