Bulk Uploads

Thanks Jason,

this is very helpful.

cheers Ian.

Hi Ian,

I branched your comment into its own topic since it probably deserves more visibility.

Since requirements for bulk ingest vary widely by use case, we have no "official" solution.  However, there are many options available.

For low sophistication end-users looking to ingest hundereds to thousands of files into Atmos, the browser plugins are the easiest to use utilities:

Chrome:

https://chrome.google.com/webstore/detail/befofpjhdclgccimklkaheifjajbcmnk

Firefox:

https://addons.mozilla.org/en-US/firefox/addon/atmosfox/

For users looking to migrate larger amounts of data, my bulk ingest utility (AtmosSync) can efficiently transfer from a filesystem into Atmos:

https://community.emc.com/docs/DOC-11009

For complex scenarios, I have a new "AtmosSync2".  This version has a pluggable architecture and supports multiple sources (including JDBC and Amazon S3), destinations, and intermediate plugins.  I have not had time to do a full write-up for it, but you can download it here:

https://api.atmosonline.com/rest/namespace/A130672722730429efbb/share/AtmosSync2.jar?uid=6c91fae4fc1f427a9b101ae92d35b0b7%2FA130672722730429efbb&expires=1373639320&signature=nMtiA0SuDQc4coDJBMvHS0Gc3P0%3D

Run the following for command-line help:

java -jar AtmosSync2.jar --help

I also have a slide deck describing the updates and some sample commands to run:

https://api.atmosonline.com/rest/namespace/A130672722730429efbb/share/AtmosSync_v2_external.pptx?uid=6c91fae4fc1f427a9b101ae92d35b0b7%2FA130672722730429efbb&expires=1405175364&signature=m7v34ivnQfFC8IEIE9M1P9RRxwM%3D

Both AtmosSync and AtmosSync2 are open source.  You can find them in the "samples" directory of the Atmos Java connector:

http://code.google.com/p/atmos-java/source/browse/#svn%2Ftrunk%2Fsamples%2FAtmosSync

Looks like a great tool. Will this tool retain the original format and integrity of imported data say if the data is encrypted? I can see you can uplaod the file from local directory. Does it work/have any impact if the source file is on network mounted/presented through NFS/CIFS or iSCSI? Ideally, can it be used as a data migration utility from current storage to Atmos?

Yes, the utility transfers your metadata and binary content as-is, so any encryption, compression, etc, will be retained.

You are correct, the utility's main purpose is migrating data to Atmos.  When using the Filesystem Source, any local filesystem on the client machine should work.  This includes CIFS, NFS, and filesystems hosted on iSCSI LUNs.

Great, thanks!