Limit listDirectory()

mirando wrote: Is there a way to limit listDirectory() ? I'm using atmos-php and there is a cloud directory which has the contents of files uploaded by Atmos Geodrive. I then am trying to list the contents via a php script on a webpage, but the file list is too long (I am watching the TCP traffic and the XML is being returned but it will probably take forever to list the files). Considering there is the potential for (and probably are) thousands of files, I'd like to paginate the directory listing using atmos-php for the PHP script. So, is there some sort of helper or way to paginate the directory listing? Thanks.

Yes, Atmos supports pagination of the directory listing using the x-emc-limit and x-emc-token headers.  Unfortunately, I have not had the chance yet to update the PHP wrapper to add this support. If you would like to add it, you can take a look at the Java code and add the ListOptions object to the listDirectory function.

http://code.google.com/p/atmos-java/source/browse/trunk/api/src/com/emc/esu/api/rest/EsuRestApi.java#930

Hi Jashon,

We have been working under Atmos-c wrappers and tried pagination of the directory listing using the x-emc-limit and x-emc-tag headers.  Unfortunately we couldn't succeed. Could you please suggest what could be the problem?

For sample here i have sent the request/response headers. There are more than 7 files like myindex.txt, myindex_1.txt hence we give it as tag.

Request:

> GET /rest/namespace/vembu-test4-bucket/ HTTP/1.1

Host: accesspoint.emccis.com

Accept: */*

x-emc-date:Fri, 11 Nov 2011 06:01:59 GMT

x-emc-groupacl:other=NONE

x-emc-include-meta: 0

x-emc-limit: 5

x-emc-tags: myindex

x-emc-uid:9e21feebb9f5455f99cd1ad8c76ebe1e/EMC0056CC2B8DDD13D60

content-type:application/octet-stream

X-Emc-Signature:S6gVbwE0dlTfKpP38Y8zFjNXeaA=

Response:

< HTTP/1.1 403 Forbidden

< Server: Apache

< Content-Type: text/xml

< Date: Fri, 11 Nov 2011 06:02:48 GMT

< Connection: Keep-Alive

< Content-Length: 195

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

100   195  100   195    0     0     19      0  0:00:10  0:00:10 --:--:--     0* Connection #0 to host accesspoint.emccis.com left intact

    1032

    There was a mismatch between the signature in the request and the signature computed by the server.

Hi Lenin,

I see you're connecting to Atmos Online.  Unfortunately, listing a directory with x-emc-limit is only supported with Atmos 1.4, and Atmos Online is running 1.3.4.

If you download the Atmos Evaluation Edition you can have your own VMs running 2.0.0:

https://community.emc.com/docs/DOC-8252

Hi Jason,

The StoreGrid product is an online backup service where the clients will backup data to StoreGrid backup server which will be hosted  in partner's own data centers and offer online backup services to their customers.  Besides we have initiated StoreGrid Cloud  Virtual Appliance which can be instantiated as a backup or a replication server and run on Cloud Computing Infrastructure . The StoreGrid Cloud API has been designed to store data at Online cloud storage like EMC Atmos Online Storage, Amazon Simple Storage Service (S3) and MicroSoft Azure and so.

From your previous post we could know some of the functionality not yet available on EMC Atmos Online WebServices but existing in Atmos VMs deployment 2.0.0. We believe EMC Atmos VMs deployed in our own data center could not be act like S3 like online storage. We just need to replicate the data which has been backed up in backup server(partner's own data centers) into an online Storage service like S3,Azure. If so what are the things we need to follow in EMC Atmos API. Does the Atmos VMs meet our below requirement?

Client(having resource to be backed up) -> Backingup to Backup server(Where backup data is kept and sent to Online Cloud Storage using StoreGrid Cloud API) -> Replicating to Online Storage(like Atmos,S3,Azure).

EMC Atmos, which provides powerful and efficient management of globally distributed cloud storage which enhance our online cloud backup service.We believe we could use Atmos Online Storage same that of S3 infrastructure.

Awaiting your reply.

Regards,

Prabu RM

Vembu Technologies.

Hi Prabu,

The Atmos Online service is targeted at application developers to develop and test their applications.  You may also use the Evaluation Edition (EE) to develop your application locally.  Once your application has been developed, you may deploy it in one of the following production configurations:

1. Create account(s) with service providers running Atmos: https://community.emc.com/docs/DOC-7450
2. Atmos Virtual Edition (VE) running in your private datacenters.
3. Physical Atmos hardware running in your private datacenters.
4. Combination of 2 and 3
5. Private Atmos (2 or 3) federating to a public service provider (1).

In general, the answer to your question is yes, you can use Atmos storage like S3, but you have more options on how and where it is run including the option of running it yourself.

Thanks,

Jason

Hi Jason

Thanks for your info.. We would like to have a Atmos Private Cloud setup locally to test this scenario. We downloaded the AEE (Atmos Evaluation Edition from the below location

https://developer-content.emc.com/downloads/aee/aee_process.htm

and we follows the install pdf available with the download. We configured those 4 vms in ESX4 server. For storage we would like to use FreeNas or Openfiler iSCSCI common storage. Please confirm is this requirement enough to have a Private Cloud to implement our testing.

I believe steps 2,3 and 4 are enough to test our Application Atmos API locally, Please confirm.

Regards

Prabu 

Hi Prabu,

The Evaluation Edition ships with about 20GB of storage for testing your application and is not expandable beyond this limit.  If you require a larger system for testing, we can get you in touch with a local sales representative to coordinate the deployment of an AtmosVE system for your Proof-Of-Concept.

In general, AtmosVE will support whatever storage is supported by VMWare.  So, if you can connect your NAS or iSCSI storage to VMWare and create virtual disks on it then Atmos should support it.