- Notes, cautions, and warnings
- S3
- Amazon S3 API support in ECS
- S3 API supported and unsupported features
- Bucket policy support
- Object Tagging
- S3 Object Lock
- Object lifecycle management
- S3 Extensions
- Metadata Search
- S3 and Swift Interoperability
- Create and manage secret keys
- Authenticating with the S3 service
- Using s3curl with ECS
- Use SDKs to access the S3 service
- ECS S3 error codes
- Hadoop S3A for ECS
- Enabling data2 IP in ECS S3
- Cloud DVR
- ECS IAM for S3
- OpenStack Swift
- EMC Atmos
- CAS
- Setting up CAS support in ECS
- Cold Storage
- Compliance
- CAS retention in ECS
- Advanced retention for CAS applications: event-based retention, litigation hold, and the min/max governor
- Set up namespace retention policies
- Create and set up a bucket for a CAS user
- Set up a CAS object user
- Set up bucket ACLs for CAS
- ECS Management APIs that support CAS users
- Content Addressable Storage (CAS) SDK API support
- ECS CAS error codes
- Enabling data2 IP in CAS
- ECS Management REST API
- ECS HDFS
- ECS HDFS Introduction
- Configuring Hadoop to use ECS HDFS
- Hadoop authentication modes
- Migration from a simple to a Kerberos Hadoop cluster
- File system interaction
- Supported Hadoop applications
- Integrate a simple Hadoop cluster with ECS HDFS
- Install Hortonworks HDP using Ambari
- Create a bucket for HDFS using the ECS Portal
- Plan the ECS HDFS and Hadoop integration
- Obtain the ECS HDFS installation and support package
- Deploy the ECS HDFS Client Library
- Configure ECS client properties
- Set up Hive
- Verify Hadoop access to ECS
- Secure the bucket
- Relocate the default file system from HDFS to an ECS bucket
- Integrate a secure Hadoop cluster with ECS HDFS
- Verify that AD/LDAP is correctly configured with a secure Hadoop cluster
- Pig test fails: unable to obtain Kerberos principal
- Permission denied for AD user
- Permissions errors
- Failed to process request
- Enable Kerberos client-side logging and debugging
- Debug Kerberos on the KDC
- Eliminate clock skew
- Configure one or more new ECS nodes with the ECS service principal
- Workaround for Yarn directory does not exist error
- Set up the Kerberos KDC
- Configure AD user authentication for Kerberos
- Secure bucket metadata
- Hadoop core-site.xml properties for ECS HDFS
- Hadoop core-site.xml properties for ECS S3
- External key management
- Document feedback
ECS 3.6.2 Data Access Guide
Supported EMC Atmos REST API Calls
ECS supports a subset of the EMC Atmos API.
| Method | Path | Description |
|---|---|---|
| Service Operations | ||
| GET | /rest/service | Get information about the system |
| Object Operations | ||
| POST | /rest/objects /rest/namespace/<path> | Create an object (See notes below) |
| DELETE | /rest/objects/<ObjectID> /rest/namespace/<path> | Delete object |
| PUT | /rest/objects/<ObjectID> /rest/namespace/<path> | Update object (See notes below) |
| GET | /rest/objects/<ObjectID> /rest/namespace/<path> | Read object (or directory list) |
| POST | /rest/namespace/<path>?rename | Rename an object |
| MetaData Operations | ||
| GET | /rest/objects/<ObjectID>?metadata/user /rest/namespace/<path>?metadata/user | Get user metadata for an object |
| POST | /rest/objects/<ObjectID>?metadata/user /rest/namespace/<path>?metadata/user | Set user metadata |
| DELETE | /rest/objects/<objectID>?metadata/user /rest/namespace/<path>?metadata/user | Delete user metadata |
| GET | /rest/objects/<ObjectID>?metadata/system /rest/namespace/<path>?metadata/system | Get system metadata for an object |
| GET | /rest/objects/<ObjectID>?acl /rest/namespace/<path>?acl | Get ACL |
| POST | /rest/objects/<ObjectID>?acl /rest/namespace/<path>?acl | Set ACL |
| GET | /rest/objects/<ObjectID>?metadata/tags /rest/namespace/<path>?metadata/tags | Get metadata tags for an object |
| GET | /rest/objects/<ObjectID>?info /rest/namespace/<path>?info | Get object info |
| Head | /rest/objects/<ObjectID> /rest/namespace/<path> | Get all object metadata |
| Object-space Operations | ||
| GET | /rest/objects | List objects |
| GET | /rest/objects?listabletags | Get listable tags |
| Anonymous Access | ||
| GET | /rest/objects/<ObjectId>?uid=<uid>&expires=<exp>&signature=<sig> /rest/namespace/<path>?uid=<uid>&expires=<exp>&signature=<sig> | Shareable URL |
NOTE:
- The x-emc-wschecksum header is supported in ECS.
- The Atmos objects do not inherit ACL from the group ACL that is set at a bucket level.
- If there is no user ACL provided, the ACL is inherited from the x-emc-useracl header.
- If there is no group ACL provided, the Read ACL is used by default.
- GET /rest/objects does not support different response types with x-emc-accept. For example, text/plain is not supported.
- Read, Write, and Delete ACLs work in ECS the same as Atmos.
- POST /rest/objects supports the x-emc-object-id header to enable legacy (44 character) object Ids.
Atmos listable tags
Listable tags are special user-defined tags used to list or filter objects. For example, an application could enable the user to tag a group of illustrations (objects) with a tag like "Vacation2016". Later the application can respond to a query of "Vacation2016" by listing only the objects tagged with this listable tag.
Using the Atmos protocol with ECS, a user cannot delete or modify another user's listable tags. Under some conditions, this ability is enabled in native Atmos.
Listable tags are indexed in ECS, increasing the performance and scalability of the retrieval of tagged objects.
In ECS, the EMC_TAGS metadata tag is used to persist listable tags. This tag name should not be used in user-defined metadata tags.
Object ID length
Support for the Atmos API in ECS expands the length of the object Id from 44 characters to 101 characters. Hence, when moving applications from Atmos to ECS you need to be aware that the object Id length will be different.
To create an object with the legacy 44 character Id length, you can use the x-emc-object-id header. This enables objects to be migrated to Atmos.
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please select whether the article was helpful or not.
Comments cannot contain these special characters: <>()\