ECS: PUT or GET or DELETE requests on bucket with life-cycle policy (that has no prefix) returns 500 error

摘要: Unable to write or read or delete objects on bucket after created life-cycle policy with no prefer ()

本文适用于 本文不适用于 本文并非针对某种特定的产品。 本文并非包含所有产品版本。
查看其他资源

症状

After create life-cycle policy (that has no prefix) on the bucket, GET or PUT or DELETE object requests return 500 internal error. List bucket works fine.

Checking existing policy for object xxx with prefix null shows this in dataheadsvc.log. 
2018-04-07T03:30:48,744 [qtp597531831-21452-0a20a979:162908dae63:2fef:17-s3-xx.xx.xx.xx]  INFO  ObjectExpirationUtil.java (line 33) Checking existing policy  for object testfile1 with prefix null
2018-04-07T03:30:48,745 [qtp597531831-21452-0a20a979:162908dae63:2fef:17-s3-xx.xx.xx.xx] ERROR  GeneralExceptionMapper.java (line 37) internal error
java.lang.NullPointerException
        at java.lang.String.startsWith(String.java:1405)
        at java.lang.String.startsWith(String.java:1434)
        at com.emc.storageos.data.object.expiration.ObjectExpirationUtil.getObjectExpirationDetails(ObjectExpirationUtil.java:42)
        at com.emc.storageos.data.object.RESTAccess.DataAccessLayer.getObjectInfo(DataAccessLayer.java:870)
        at com.emc.storageos.data.api.service.impl.resource.s3.operation.S3ObjectOperations.getObject(S3ObjectOperations.java:926)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
... ...
2018-04-07T03:30:48,751 [qtp597531831-21452]  INFO  RequestLog.java (line 83) xx.xx.xx.xxx GET //xx.xx.xx.xx:9020/test/testfile1 HTTP/1.1 500 9 - 111



 

原因

For AWS, each rule has <Filter> element which has children element: <Prefix>, <Tag>, <Filter> is required XML element, <Prefix>, and <Tag> is optional (apply to all objects in buckets).

For ECS, we only support filter <prefix> and do not have <Filter> element designed in request XML. <prefix> is not required parameter, but some place code uses prefix value directly(null point) which cause issue here. 

解决方案

If seeing error log like show in "Issue" field, confirm whether life cycle is set on the bucket and no prefix configured. See the ECS Data Access Guide for further reference of using s3curl.pl. 

# ./s3curl.pl --id=test -- http://<node data ip>:9020/<bucket>/?lifecycle |xmllint --format - 
#  ./s3curl.pl --id=test -- http://<node data ip>:9020/<bucket>/?lifecycle |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   285  100   285    0     0  23538      0 --:--:-- --:--:-- --:--:-- 23750
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Rule>
    <ID/>
    <Status>Enabled</Status>
    <Expiration>
      <Days>10</Days>
    </Expiration>
    <NoncurrentVersionExpiration/>
  </Rule>
</LifecycleConfiguration>

If you confirm that hit this issue, the workaround prior to 3.2 is to add <prefix/> in lifecycle.
This issue is fixed in 3.2.

Workaround:

1. Get current lifecycle setting and save to local file
#  ./s3curl.pl --id=test -- http://<data node ip>:9020/<bucket>/?lifecycle -o lifecycle.xml


#  ./s3curl.pl --id=test -- http://<data node ip>:9020/<bucket>/?lifecycle -o lifecycle.xml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   268  100   268    0     0  11926      0 --:--:-- --:--:-- --:--:-- 12181

2. Modify the lifecycle.xml and add prefix
# vi lifecycle.xml


<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Rule>
    <ID/>
    <Prefix/>                                           -->add this line
    <Status>Enabled</Status>
    <Expiration>
      <Days>10</Days>
    </Expiration>
  </Rule>
</LifecycleConfiguration>

3. Update lifecycle
#  ./s3curl.pl --id=test --put=lifecycle.xml http://xx.xx.xx.xx:9020/<bucket>/?lifecycle --calculateContentMd5 --debug


#  ./s3curl.pl --id=test --put=lifecycle.xml http://xx.xx.xx.xx:9020/fog-bbr-backup/?lifecycle --calculateContentMd5 --debug
s3curl: Found the url: host=xx.xx.xx.xx; port=9020; uri=/fog-bbr-backup/; query=lifecycle;
s3curl: replaced string: lifecycle
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\na4EIw7LnRPKpiAfSy/YHIQ==\n\nSat, 07 Apr 2018 05:11:32 +0000\n/fog-bbr-backup/?lifecycle'
s3curl: exec curl -v -H 'Date: Sat, 07 Apr 2018 05:11:32 +0000' -H 'Authorization: AWS pcf-fog-sys:4NFcmpx/3i0FBoJsFUJSabAlmGI=' -L -H 'content-type: ' -H 'Content-MD5: a4EIw7LnRPKpiAfSy/YHIQ==' -T lifecycle.xml http://xx.xx.xx.xx:9020/fog-bbr-backup/?lifecycle
* Hostname was NOT found in DNS cache
*   Trying xx.xx.xx.xx...
* Connected to xx.xx.xx.xx (xx.xx.xx.xx) port 9020 (#0)
> PUT /fog-bbr-backup/?lifecycle HTTP/1.1
> User-Agent: curl/7.37.0
> Host: xx.xx.xx.xx:9020
> Accept: */*
> Date: Sat, 07 Apr 2018 05:11:32 +0000
> Authorization: AWS pcf-fog-sys:4NFcmpx/3i0FBoJsFUJSabAlmGI=
> Content-MD5: a4EIw7LnRPKpiAfSy/YHIQ==
> Content-Length: 284
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Sat, 07 Apr 2018 05:11:32 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a23a338:1628835dec8:5cb3:19
< x-amz-id-2: a52c5da7cb5555375cef4f9172b03532652afc6cf67528d42451b0a3a20aad4a
< Content-Length: 0
<
* Connection #0 to host xx.xx.xx.xx left intact


4. Confirm lifecycle is updated.
#  ./s3curl.pl --id=test -- http://xx.xx.xx.xx:9020/<bucket>/?lifecycle |xmllint --format -


#  ./s3curl.pl --id=test -- http://xx.xx.xx.xx:9020/fog-bbr-backup/?lifecycle |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   285  100   285    0     0  23538      0 --:--:-- --:--:-- --:--:-- 23750
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Rule>
    <ID/>
    <Prefix/>
    <Status>Enabled</Status>
    <Expiration>
      <Days>10</Days>
    </Expiration>
    <NoncurrentVersionExpiration/>
  </Rule>
</LifecycleConfiguration>

5. Retry read/write/delete.

受影响的产品

ECS Appliance Hardware Gen1 U-Series

产品

ECS Appliance Hardware Gen1 U-Series, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, Elastic Cloud Storage
文章属性
文章编号: 000057163
文章类型: Solution
上次修改时间: 09 2月 2023
版本:  3
从其他戴尔用户那里查找问题的答案
支持服务
检查您的设备是否在支持服务涵盖的范围内。