Isilon CloudPools: Amazon AWS is changing certificate providers which affects CloudPools access
Summary: Amazon is changing their certificate of authority for their web services, which affects CloudPools access.
Symptoms
Cause
See the below link for details on the Amazon certificate change:
https://aws.amazon.com/blogs/networking-and-content-delivery/reminder-amazon-s3-and-amazon-cloudfront-service-certificates-migrating-to-amazon-trust-services-starting-march-23-2021/
OneFS is built with certificates that are preinstalled to allow for ease of connection to our supported public CloudPools storage providers. Dell is working on a patch which adds this new certificate for future CloudPools configuration, but for existing customers using CloudPools archiving to Amazon AWS, the change that is shown in the Resolution section is required in order to continue with CloudPools connectivity.
Resolution
OneFS 8.1 and earlier:
Cluster download of .pem files:
# cd /ifs/.ifsvar/modules/cloud/cacert # curl -k https://www.amazontrust.com/repository/AmazonRootCA1.pem > AmazonRootCA1.pem; curl -k https://www.amazontrust.com/repository/AmazonRootCA2.pem > AmazonRootCA2.pem; curl -k https://www.amazontrust.com/repository/AmazonRootCA3.pem > AmazonRootCA3.pem; curl -k https://www.amazontrust.com/repository/AmazonRootCA4.pem > AmazonRootCA4.pem; curl -k https://www.amazontrust.com/repository/SFSRootCAG2.pem > SFSRootCAG2.pem
User download of the .pem files:
Retrieve the appropriate self-signed PEM certificate from Amazon’s official certificate site:
https://www.amazontrust.com/repository/
Using a client, download the following files:
- https://www.amazontrust.com/repository/AmazonRootCA1.pem
- https://www.amazontrust.com/repository/AmazonRootCA2.pem
- https://www.amazontrust.com/repository/AmazonRootCA3.pem
- https://www.amazontrust.com/repository/AmazonRootCA4.pem
- https://www.amazontrust.com/repository/SFSRootCAG2.pem
Move these files onto the cluster.
Making certificates available to OneFS:
As the root user on the command line on the cluster, starting in the directory in which the .pem files were moved, follow these steps (replace <cert.pem> with the .pem filename):
- Move the certs to the appropriate directory:
# cp <cert.pem> /ifs/.ifsvar/modules/cloud/cacert
- Move into the certs directory:
# cd /ifs/.ifsvar/modules/cloud/cacert
- Follow this process for each of the five .pem files downloaded. Get the hash for the cert:
# openssl x509 -hash -noout -in <cert.pem>
- Create a symlink to the <cert.pem> using the output from above <hash-val>:
# ln -s /ifs/.ifsvar/modules/cloud/cacert/<cert.pem> /ifs/.ifsvar/modules/cloud/cacert/<hash-val>.0
This completes adding the certificate on OneFS 8.1 and earlier versions.
For OneFS 8.2 and later:
Because the cacert directory does not exist, the downloads should be done in another directory, such as /ifs/data/Isilon_Support, that works.
Cluster-side download of .pem files:
# cd /ifs/data/Isilon_Support/ # curl -k https://www.amazontrust.com/repository/AmazonRootCA1.pem > AmazonRootCA1.pem; curl -k https://www.amazontrust.com/repository/AmazonRootCA2.pem > AmazonRootCA2.pem; curl -k https://www.amazontrust.com/repository/AmazonRootCA3.pem > AmazonRootCA3.pem; curl -k https://www.amazontrust.com/repository/AmazonRootCA4.pem > AmazonRootCA4.pem; curl -k https://www.amazontrust.com/repository/SFSRootCAG2.pem > SFSRootCAG2.pem
User download of the .pem files:
Retrieve the appropriate self-signed PEM certificates from Amazon’s official certificate site:
https://www.amazontrust.com/repository/
Using a client, download the following files:
- https://www.amazontrust.com/repository/AmazonRootCA1.pem
- https://www.amazontrust.com/repository/AmazonRootCA2.pem
- https://www.amazontrust.com/repository/AmazonRootCA3.pem
- https://www.amazontrust.com/repository/AmazonRootCA4.pem
- https://www.amazontrust.com/repository/SFSRootCAG2.pem
Move these file onto the cluster.
Making certificates available to OneFS:
As a root user on the command line on the cluster, starting in the directory in which the .pem file was moved, follow these steps (replace <cert.pem> with the .pem filename):
- Move the cert to the appropriate directory:
# cp <cert.pem> /ifs/data/Isilon_Support/
- Move to that directory:
# cd /ifs/data/Isilon_Support/
To import the certificate into our certificate management system:
# isi certificate authority import --certificate-path=<cert.pem> --description=”<description>” --name=<cert_name>
To do this automatically:
# isi certificate authority import --certificate-path=AmazonRootCA1.pem --description="Amazon CA1" --name=amazon_cert1; isi certificate authority import --certificate-path=AmazonRootCA2.pem --description="Amazon CA2" --name=amazon_cert2; isi certificate authority import --certificate-path=AmazonRootCA3.pem --description="Amazon CA3" --name=amazon_cert3; isi certificate authority import --certificate-path=AmazonRootCA4.pem --description="Amazon CA4" --name=amazon_cert4; isi certificate authority import --certificate-path=SFSRootCAG2.pem --description="Starfield Services Root CA" --name=Starfield_cert
Verification of successful import:
# isi certificate authority list
The output should show the newly added certificate names.
Regarding fix using RUP:
Update AWS S3 cert to Amazon Trust Services (8.2.2).
https://jira.cec.lab.emc.com/browse/PSCALE-58298
PATCH: [8.2.2_GA-RUP_2021-07][Multiple Userspace and Kernel Fixes](July 2021)
ttps://jira.cec.lab.emc.com/browse/PSP-1250
The below command must be run in addition to applying RUP in order to complete the import process of the target Amazon certificates:
# python -m isi.certs.provision
------------------
- Applying only the patch does not complete the import process.
i8220s-1# isi upgrade patches list
Patch Name Description Status
---------------------------------------------------------------------------
8.2.2_GA-RUP_2021-07_PSP-1250 Multiple Userspace and Kernel Fixes Installed
---------------------------------------------------------------------------
i8220s-1# isi certificate authority list | grep Amazon
i8220s-1#
- The below python-command must be run for the import of the target Amazon certificates to complete.
i8220s-1# python -m isi.certs.provision
i8220s-1#
i8220s-1# isi certificate authority list | grep Amazon
18ce6cf AmazonTrustServices_Root_CA3 valid 2040-05-26T09:00:00
1ba5b2a AmazonTrustServices_Root_CA2 valid 2040-05-26T09:00:00
8ecde68 AmazonTrustServices_Root_CA1 valid 2038-01-17T09:00:00
e35d284 AmazonTrustServices_Root_CA4 valid 2040-05-26T09:00:00