Unsolved
This post is more than 5 years old
2 Posts
0
2082
Bulk Uploads
Hi Jason,
I see it's a while since you posted this; I was just interested in knowing if there is now an 'official' tool for performing
bulk uploads of data to the ATMOS environment either by the customer themselves or by a client administrator working on the customers behalf ?
thanks Ian.
mortoman
2 Posts
0
July 12th, 2012 07:00
Thanks Jason,
this is very helpful.
cheers Ian.
JasonCwik
281 Posts
0
July 12th, 2012 07:00
Hi Ian,
I branched your comment into its own topic since it probably deserves more visibility.
Since requirements for bulk ingest vary widely by use case, we have no "official" solution. However, there are many options available.
For low sophistication end-users looking to ingest hundereds to thousands of files into Atmos, the browser plugins are the easiest to use utilities:
Chrome:
https://chrome.google.com/webstore/detail/befofpjhdclgccimklkaheifjajbcmnk
Firefox:
https://addons.mozilla.org/en-US/firefox/addon/atmosfox/
For users looking to migrate larger amounts of data, my bulk ingest utility (AtmosSync) can efficiently transfer from a filesystem into Atmos:
https://community.emc.com/docs/DOC-11009
For complex scenarios, I have a new "AtmosSync2". This version has a pluggable architecture and supports multiple sources (including JDBC and Amazon S3), destinations, and intermediate plugins. I have not had time to do a full write-up for it, but you can download it here:
https://api.atmosonline.com/rest/namespace/A130672722730429efbb/share/AtmosSync2.jar?uid=6c91fae4fc1f427a9b101ae92d35b0b7%2FA130672722730429efbb&expires=1373639320&signature=nMtiA0SuDQc4coDJBMvHS0Gc3P0%3D
Run the following for command-line help:
java -jar AtmosSync2.jar --help
I also have a slide deck describing the updates and some sample commands to run:
https://api.atmosonline.com/rest/namespace/A130672722730429efbb/share/AtmosSync_v2_external.pptx?uid=6c91fae4fc1f427a9b101ae92d35b0b7%2FA130672722730429efbb&expires=1405175364&signature=m7v34ivnQfFC8IEIE9M1P9RRxwM%3D
Both AtmosSync and AtmosSync2 are open source. You can find them in the "samples" directory of the Atmos Java connector:
http://code.google.com/p/atmos-java/source/browse/#svn%2Ftrunk%2Fsamples%2FAtmosSync
rjanakan1
19 Posts
0
July 17th, 2012 12:00
Looks like a great tool. Will this tool retain the original format and integrity of imported data say if the data is encrypted? I can see you can uplaod the file from local directory. Does it work/have any impact if the source file is on network mounted/presented through NFS/CIFS or iSCSI? Ideally, can it be used as a data migration utility from current storage to Atmos?
JasonCwik
281 Posts
0
July 17th, 2012 14:00
Yes, the utility transfers your metadata and binary content as-is, so any encryption, compression, etc, will be retained.
You are correct, the utility's main purpose is migrating data to Atmos. When using the Filesystem Source, any local filesystem on the client machine should work. This includes CIFS, NFS, and filesystems hosted on iSCSI LUNs.
rjanakan1
19 Posts
0
July 17th, 2012 15:00
Great, thanks!