Unsolved
This post is more than 5 years old
2 Intern
•
718 Posts
3
11904
Ask the Expert: Object Storage into the Cloud
Welcome to the EMC Support Community Ask the Expert conversation. We will be covering EMC Cloud Object storage. Our SME’s will be available to answer questions on ATMOS, CTA, Centera and ECS. Among many areas, our SME’s will be on hand to answer your questions on the benefits and best practices of ECS CAS and Centera to ECS migrations.
Meet Your Experts:
Technical Support Engineer Level IV During the past 11 years in the Object space, Dean has been involved in Remote Tech Support, TCE, and Testing. He is currently working in Centera to ECS Migration. |
|
Technical Support Engineer III Christian has been working in remote tech support in EMC Object space (CAS, CTA and CUA) for 8 years. Before that he was working as CE responsible for NAS storage. Christian has been employed at EMC for 15 years. |
|
ECS/Atmos Tech Support EMC Object Storage support Engineer since 2006. |
|
Technical Support Engineer III Ragy has been working within the Technology field for 6 years. Currently a Technical support engineer in the ETD Object Storage team. Currently, Cloud Tiering Appliance is his main focus. Ragy is passinate about innovation, and this has allowed him to become a winner of the EMC innovation conference 2015. |
This discussion takes place from Jun 20th - July 1st. Get ready by bookmarking this page or signing up for e-mail notifications.
Share this event on Twitter or LinkedIn:
>> Ask the Expert: Object Storage into the Cloud http://bit.ly/1YuGLxd #EMCATE <<
RobertoAraujo1
2 Intern
2 Intern
•
718 Posts
1
June 20th, 2016 07:00
This Ask the Expert session is now open for questions. For the next couple of weeks our Subject Matter Experts will be around to reply to your questions, comments or inquiries about our topic.
Let’s make this conversation useful, respectful and entertaining for all. Enjoy!
mfh2
208 Posts
1
June 20th, 2016 08:00
Robert, thanks for setting this up.
Can someone outline the Best Practices for Centera to ECS Migrations? Also, is there a formula for calculating how quickly the migration will complete?
Thanks,
Mike Horgan
Anonymous
5 Practitioner
5 Practitioner
•
274.2K Posts
0
June 20th, 2016 08:00
I will certainly contact Engineering and see if any test data can be made available.
This may just take a little time so please bear with me for a few days.
For Ecs transformation (so all reads and writes are just to ECS), the indexing side of things is quite quick and only needs to populate tables with queries to Centera.
For actually migrating the data I will revert as soon as I can
regards
Dean
mfh2
208 Posts
0
June 20th, 2016 08:00
Excellent, thanks Dean. I'd be very interested to know a baseline ECS Transformation duration expectation for something like 100M (100,000,000) small clips, say 50KB each from an imaging application. Lets assume the Centera has 16 nodes and 4 access nodes which would seem pretty typical.
Best Regards,
Mike Horgan
Anonymous
5 Practitioner
5 Practitioner
•
274.2K Posts
0
June 20th, 2016 08:00
Hello Mike
There are a number of tools to migrate Centera to ECS but ECS Transformation has the most advantages.
With ECS Transformation, (built into ECS from V2.2) Centera data can be indexed within ECS before the data itself is actually migrated.
Transformation is initiated from ECS and once the data to be migrated is indexed, the application can cutover at that point.
The application would read and write directly to ECS. ECS can retrieve indexed data from Centera with minimal performance cost.
At any time the migration of the data itself can then be started in the background.
There are no official figures as to how long a migration would take as there are so many variables. We can certainly ask to see if the Engineering test figures can be made.
Dean
simonmagennis
7 Posts
0
June 23rd, 2016 02:00
Timely thread for me!
Can someone explain the technical difference between Atmos and ECS or Point me to a detailed comparision? I have looked at a number of "technical" presentations and everything I see suggests that ECS is an update/rewrite of Atmos with a few changes such as containerization, xfs and a more modern stoage algorithm. Conceptually, I see no major difference and the HW is basically identical.
Thanks.
Simon
sullim6
16 Posts
0
June 23rd, 2016 04:00
Hi Simon,
Though the hardware of ECS and Atmos can be similar and both have support multi protocol access (Note: ECS also has support for Hadoop and OpenStack Swift, refer to the ECS 2.2.1 Data Access Guide for details on ECS API support) , the underlying Architecture layers in each product work very differently. Comparing the documents below should help you here I think. Let me know if you need anything clarified.
ATMOS
Atmos 2.4 Administrators Guide -> https://support.emc.com/docu68366_Atmos-2.4-Administrator%E2%80%99s-Guide.pdf?language=en_US
Atm,os 2.4.1 Relesae Notes -> https://support.emc.com/docu70762_Atmos-2.4.1-Release-Notes.pdf?language=en_US
ECS
ECS 2.2.1 Planning Guide -> https://support.emc.com/docu70102_ECS_2.2.1_Planning_Guide.pdf?language=en_US
ECS 2.2.1 Data Access Guide -> https://support.emc.com/docu70101_ECS_2.2.1_Data_Access_Guide.pdf?language=en_US
ECS 2.2.1 Administrators Guide ->
https://support.emc.com/docu70100_ECS-2.2.1-Administrator's-Guide.pdf?language=en_US
emma_keaveny
3 Posts
0
June 23rd, 2016 04:00
Good morning guys,
I have currently set up an ECS 2.2 Single Node Docker. I have been using this for archiving data to this endpoint, my problem is, I have used up all the space on the endpoint, after seeing this I then deleted the archives that I had sent to this endpoint but I dont see the available space increasing. I use the EMC2 Dashboard.to monitor the space and efficiency of the endpoint.
Is there a way to reclaim the space?
I appreciate any help that can be given.
Thanks
Emma
sullim6
16 Posts
0
June 23rd, 2016 04:00
Hi Mike,
While Dean looks into your query there is a Centera to ECS migration White Paper which you may find useful available @ http://www.emc.com/collateral/white-papers/h15115-centera-to-ecs-migration.pdf
Anonymous
5 Practitioner
5 Practitioner
•
274.2K Posts
0
June 23rd, 2016 05:00
Hi Emma
Thanks for your question.
When data is written to ECS, it gets appended into chunks (like containers)
When the whole chunk is empty the space is eligible for space reclamation by background Garbage Collection tasks.
How long ago was the data deleted and what would be the average file size?
regards
Dean
Gotsche_Helmut
48 Posts
0
June 23rd, 2016 05:00
The biggest difference is the policy engine on ATMOS.
If application simply need to set/update the proper metadata of an object. With the corresponding policy definition on ATMOS, these user- or system metadata are used as trigger for different protection/distribution of objects. This also covers retention enforcement, erasure service or migration to “public” cloud (if this is an ATMOS or ECS system).
--
With kind regards/Mit freundlichen Grüßen
Helmut Gotsche
Advisory Systems Engineer, EMEA
Object Storage
Software Defined Solutions@Emerging Technology Division
EMC² Computer Systems Austria GmbH
(Firmenbuch Gericht: HG Wien, FN 160018x, VATno: ATU43027201)
Am Euro Platz 1/4,
A-1120 Wien / Austria
Phone: +43 1 599 52 490
Fax: +43 1 253 67 22 1830
Mobile: +43 664 411 7466
e-Mail: Helmut.Gotsche@emc.com
More about ViPR: http://www.emc.com/cloud/vipr/index.htm
More about Atmos: http://www.emc.com/storage/atmos/atmos.htm
More about Centera: http://www.emc.com/archiving/centera.htm
For Developers: https://community.emc.com/community/edn/
The information contained in this communication is confidential.
It is intended solely for the person(s) to whom this email message is addressed.
If you have received this email message in error, please notify the sender immediately by telephone or email
and destroy the original message without making a copy.
P Please don't print this e-mail unless you really need to.
Thank you.
mfh2
208 Posts
1
June 23rd, 2016 05:00
Thanks Michael. I browsed the paper and it looks very well done, kudos to the author.
Best Regards,
Mike Horgan
emma_keaveny
3 Posts
0
June 23rd, 2016 06:00
Hi Dean,
The data was deleted 2 days ago (Tues 21st). Hard to say what the average size was to be honest. We were archiving projects that were between 2GB and 240GB.
I was testing archiving a project with 0.5TB to the endpoint that only had 300GB space available, I wanted to see what would happen and how it would handle the large file. It failed at 240GB and the archive was deleted (if an archive fails with our software, the archive is then deleted from the destination) however it still registered on the dashboard that the space was used and we were unable to archive large projects.
I hope this helps.
Thanks
Emma
Anonymous
5 Practitioner
5 Practitioner
•
274.2K Posts
0
June 28th, 2016 00:00
Hi Emma,
ECS Free and Frictionless is unfortunately outside the scope of our expertise here, and I was going to repost your issue on the main ViprECS community, but I see you have already done so here
Best regards
Dean
https://community.emc.com/thread/230083
simonmagennis
7 Posts
0
July 1st, 2016 02:00
A question concerning XOR recovery.
Assume that we have 8 DataCenters/VDCs (as shown in the ECS Storage Overhead illustrations in various places), how many sites can we lose and still be able to recover all data?
Thanks.
Simon