ECS: PUT or GET or DELETE requests on bucket with life-cycle policy (that has no prefix) returns 500 error

摘要: Unable to write or read or delete objects on bucket after created life-cycle policy with no prefer ()

本文章適用於 本文章不適用於 本文無關於任何特定產品。 本文未識別所有產品版本。
查看其他資源

症狀

After create life-cycle policy (that has no prefix) on the bucket, GET or PUT or DELETE object requests return 500 internal error. List bucket works fine.

Checking existing policy for object xxx with prefix null shows this in dataheadsvc.log. 
2018-04-07T03:30:48,744 [qtp597531831-21452-0a20a979:162908dae63:2fef:17-s3-xx.xx.xx.xx]  INFO  ObjectExpirationUtil.java (line 33) Checking existing policy  for object testfile1 with prefix null
2018-04-07T03:30:48,745 [qtp597531831-21452-0a20a979:162908dae63:2fef:17-s3-xx.xx.xx.xx] ERROR  GeneralExceptionMapper.java (line 37) internal error
java.lang.NullPointerException
        at java.lang.String.startsWith(String.java:1405)
        at java.lang.String.startsWith(String.java:1434)
        at com.emc.storageos.data.object.expiration.ObjectExpirationUtil.getObjectExpirationDetails(ObjectExpirationUtil.java:42)
        at com.emc.storageos.data.object.RESTAccess.DataAccessLayer.getObjectInfo(DataAccessLayer.java:870)
        at com.emc.storageos.data.api.service.impl.resource.s3.operation.S3ObjectOperations.getObject(S3ObjectOperations.java:926)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
... ...
2018-04-07T03:30:48,751 [qtp597531831-21452]  INFO  RequestLog.java (line 83) xx.xx.xx.xxx GET //xx.xx.xx.xx:9020/test/testfile1 HTTP/1.1 500 9 - 111



 

原因

For AWS, each rule has <Filter> element which has children element: <Prefix>, <Tag>, <Filter> is required XML element, <Prefix>, and <Tag> is optional (apply to all objects in buckets).

For ECS, we only support filter <prefix> and do not have <Filter> element designed in request XML. <prefix> is not required parameter, but some place code uses prefix value directly(null point) which cause issue here. 

解析度

If seeing error log like show in "Issue" field, confirm whether life cycle is set on the bucket and no prefix configured. See the ECS Data Access Guide for further reference of using s3curl.pl. 

# ./s3curl.pl --id=test -- http://<node data ip>:9020/<bucket>/?lifecycle |xmllint --format - 
#  ./s3curl.pl --id=test -- http://<node data ip>:9020/<bucket>/?lifecycle |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   285  100   285    0     0  23538      0 --:--:-- --:--:-- --:--:-- 23750
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Rule>
    <ID/>
    <Status>Enabled</Status>
    <Expiration>
      <Days>10</Days>
    </Expiration>
    <NoncurrentVersionExpiration/>
  </Rule>
</LifecycleConfiguration>

If you confirm that hit this issue, the workaround prior to 3.2 is to add <prefix/> in lifecycle.
This issue is fixed in 3.2.

Workaround:

1. Get current lifecycle setting and save to local file
#  ./s3curl.pl --id=test -- http://<data node ip>:9020/<bucket>/?lifecycle -o lifecycle.xml


#  ./s3curl.pl --id=test -- http://<data node ip>:9020/<bucket>/?lifecycle -o lifecycle.xml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   268  100   268    0     0  11926      0 --:--:-- --:--:-- --:--:-- 12181

2. Modify the lifecycle.xml and add prefix
# vi lifecycle.xml


<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Rule>
    <ID/>
    <Prefix/>                                           -->add this line
    <Status>Enabled</Status>
    <Expiration>
      <Days>10</Days>
    </Expiration>
  </Rule>
</LifecycleConfiguration>

3. Update lifecycle
#  ./s3curl.pl --id=test --put=lifecycle.xml http://xx.xx.xx.xx:9020/<bucket>/?lifecycle --calculateContentMd5 --debug


#  ./s3curl.pl --id=test --put=lifecycle.xml http://xx.xx.xx.xx:9020/fog-bbr-backup/?lifecycle --calculateContentMd5 --debug
s3curl: Found the url: host=xx.xx.xx.xx; port=9020; uri=/fog-bbr-backup/; query=lifecycle;
s3curl: replaced string: lifecycle
s3curl: ordinary endpoint signing case
s3curl: StringToSign='PUT\na4EIw7LnRPKpiAfSy/YHIQ==\n\nSat, 07 Apr 2018 05:11:32 +0000\n/fog-bbr-backup/?lifecycle'
s3curl: exec curl -v -H 'Date: Sat, 07 Apr 2018 05:11:32 +0000' -H 'Authorization: AWS pcf-fog-sys:4NFcmpx/3i0FBoJsFUJSabAlmGI=' -L -H 'content-type: ' -H 'Content-MD5: a4EIw7LnRPKpiAfSy/YHIQ==' -T lifecycle.xml http://xx.xx.xx.xx:9020/fog-bbr-backup/?lifecycle
* Hostname was NOT found in DNS cache
*   Trying xx.xx.xx.xx...
* Connected to xx.xx.xx.xx (xx.xx.xx.xx) port 9020 (#0)
> PUT /fog-bbr-backup/?lifecycle HTTP/1.1
> User-Agent: curl/7.37.0
> Host: xx.xx.xx.xx:9020
> Accept: */*
> Date: Sat, 07 Apr 2018 05:11:32 +0000
> Authorization: AWS pcf-fog-sys:4NFcmpx/3i0FBoJsFUJSabAlmGI=
> Content-MD5: a4EIw7LnRPKpiAfSy/YHIQ==
> Content-Length: 284
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Sat, 07 Apr 2018 05:11:32 GMT
* Server ViPR/1.0 is not blacklisted
< Server: ViPR/1.0
< x-amz-request-id: 0a23a338:1628835dec8:5cb3:19
< x-amz-id-2: a52c5da7cb5555375cef4f9172b03532652afc6cf67528d42451b0a3a20aad4a
< Content-Length: 0
<
* Connection #0 to host xx.xx.xx.xx left intact


4. Confirm lifecycle is updated.
#  ./s3curl.pl --id=test -- http://xx.xx.xx.xx:9020/<bucket>/?lifecycle |xmllint --format -


#  ./s3curl.pl --id=test -- http://xx.xx.xx.xx:9020/fog-bbr-backup/?lifecycle |xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   285  100   285    0     0  23538      0 --:--:-- --:--:-- --:--:-- 23750
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Rule>
    <ID/>
    <Prefix/>
    <Status>Enabled</Status>
    <Expiration>
      <Days>10</Days>
    </Expiration>
    <NoncurrentVersionExpiration/>
  </Rule>
</LifecycleConfiguration>

5. Retry read/write/delete.

受影響的產品

ECS Appliance Hardware Gen1 U-Series

產品

ECS Appliance Hardware Gen1 U-Series, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, Elastic Cloud Storage
文章屬性
文章編號: 000057163
文章類型: Solution
上次修改時間: 09 2月 2023
版本:  3
向其他 Dell 使用者尋求您問題的答案
支援服務
檢查您的裝置是否在支援服務的涵蓋範圍內。