dkintanar
1 Copper

Atmos Jets3t Compatibility

Hello all,

We were hoping to leverage Atmos as our back end storage for the Liferay document library (www.liferay.com) - given Atmos' compatibility with the Amazon S3 API, the thought was that we would be able to easily point Liferay from Amazon to Atmos easily. 

Liferay uses the Jets3t toolkit for S3 connectivity and, using available properties (JetS3t - Advanced Configuration), we are able to point Jets3t at our instance of Atmos.  Unfortunately, when Liferay attempts to run it's migration into the S3 store, we receive the following error:

14:57:08,927 ERROR [liferay/convert_process-1][S3Store:63] Expected XML document response from S3 but received content type text/html; charset=utf-8


I'm posting here to see if anyone has had success using the JetS3t toolkit to connect to Atmos.  Is this something that should be doable?

Thanks,

Dustin

Labels (1)
0 Kudos
4 Replies
Highlighted
JasonCwik
2 Iron

Re: Atmos Jets3t Compatibility

Can you post your configuration?  What version of Atmos?

The s3 connector runs on port 8080 and/or 8443. You'll also need a recent version of jets3t so you can configure the endpoint in jets3t.properties.

Finally, there's an issue with jets3t creating zero-byte objects.  Jets3t computes the checksum in the content sending handler hooked to Expect: 100 continue, but Atmos is currently sending 200 OK immediately since there's no content to receive.  This causes jets3t to skip computing its checksum and then throws an error saying the checksum doesn't match.  This issue will be fixed in the next Atmos service pack.

dkintanar
1 Copper

Re: Atmos Jets3t Compatibility

Hi Jason,

I have the following in my jets3t properties:

#

# Configure jets3t to point at Atmos

#

s3service.s3-endpoint=servername.company.com

s3service.s3-endpoint-http-port=80

s3service.s3-endpoint-https-port=443

Endpoint link changed to anonymous values, but it seemed to pick up the configuration correctly.  Setting the endpoint to a random value yielded "unknown host" exceptions and, before we had a trusted cert setup, we were getting "javax.net.ssl.SSLPeerUnverfiedException: peer not authenticated" errors.  The Liferay properties hold the access/secret key along with the bucket name. 

Liferay seems to ship with the latest 0.9.0 version, but I downloaded and attempted to use the latest version from the Jets3t site anyhow. 

Let me see if I can try running through port 8080 and 8443 and see if that does the trick.

0 Kudos
dkintanar
1 Copper

Re: Atmos Jets3t Compatibility

Hi Jason,

Thank you for your help - using port 8080 and 8443 seems to have cleared up the connection issue, but Liferay still seems to be failing when trying to create or update objects within Atmos.  The error being encountered now is the following:

16:58:45,414 DEBUG [http-bio-8080-exec-8][MaintenanceUtil:64] Executing com.liferay.portal.convert.ConvertDocumentLibrary

16:58:45,423 INFO [liferay/convert_process-1][ConvertProcess:41] Starting conversion for com.liferay.portal.convert.ConvertDocumentLibrary

16:58:45,929 DEBUG [liferay/convert_process-1][MaintenanceUtil:64] Migrating 20 document library files

16:58:45,949 ERROR [liferay/convert_process-1][ConvertDocumentLibrary:188] Migration failed for 1

  1. com.liferay.portal.kernel.exception.SystemException: org.jets3t.service.S3ServiceException: The action Create Object in bucket cannot be performed with an invalid bucket: null

                at com.liferay.portlet.documentlibrary.store.S3Store.addFile(S3Store.java:92)

                at com.liferay.portal.convert.ConvertDocumentLibrary.migrateFile(ConvertDocumentLibrary.java:180)

                at com.liferay.portal.convert.ConvertDocumentLibrary.migrateDLFileEntry(ConvertDocumentLibrary.java:167)

                at com.liferay.portal.convert.ConvertDocumentLibrary.migrateDL(ConvertDocumentLibrary.java:143)

                at com.liferay.portal.convert.ConvertDocumentLibrary.migratePortlets(ConvertDocumentLibrary.java:238)

                at com.liferay.portal.convert.ConvertDocumentLibrary.doConvert(ConvertDocumentLibrary.java:102)

                at com.liferay.portal.convert.ConvertProcess.convert(ConvertProcess.java:44)

                at com.liferay.portal.convert.messaging.ConvertProcessMessageListener.doReceive(ConvertProcessMessageListener.java:47)

                at com.liferay.portal.convert.messaging.ConvertProcessMessageListener.receive(ConvertProcessMessageListener.java:32)

                at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(InvokerMessageListener.java:63)

                at com.liferay.portal.kernel.messaging.SerialDestination$1.run(SerialDestination.java:110)

                at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask._runTask(ThreadPoolExecutor.java:671)

                at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask.run(ThreadPoolExecutor.java:582)

                at java.lang.Thread.run(Unknown Source)

Caused by: org.jets3t.service.S3ServiceException: The action Create Object in bucket cannot be performed with an invalid bucket: null

                at org.jets3t.service.S3Service.putObject(S3Service.java:2267)

                at com.liferay.portlet.documentlibrary.store.S3Store.addFile(S3Store.java:84)

I have tried both pointing to an existing bucket and a bucket that does not yet exist.  I tried with both the version of jets3t that comes with Liferay and the latest version from the Jets3t site.  On the Liferay side, here is the general code path that is taken (just snippets of code):

// Getting the RestVerified that the Access/Secret Keys passed in work with AtmosChrome.  Per the

// Above comment, have tried with both the bucket missing and created.

AWSCredentials credentials = new AWSCredentials(_ACCESS_KEY, _SECRET_KEY)

_s3Service = RestS3Service(credentials);

_s3Bucket = _s3Service.getBucket(_BUCKET_NAME);

// The "key" and "is" objects are passed in as String and InputStream parameters to an "AddFile" method. 

S3Object s3Object = new S3Object(_s3Bucket,key);

s3Object.setDataInputStream(is);

_s3Service.putObject(_s3Bucket, s3Object);

is.close();

0 Kudos
dkintanar
1 Copper

Re: Atmos Jets3t Compatibility

Hello all - I was able to resolve the issues by creating the bucket under the S3 folder that was created in the Atmos repository.  I wasn't aware of it, but it appears that when connecting to Atmos through S3, the default root folder is S3 and anything uploaded or read is made to that folder.

By creating the bucket Liferay/Jets3t was looking for under S3, Liferay was able to read/write to the folder successfully.

Thanks,

Dustin

0 Kudos