Unsolved

1110

July 23rd, 2020 08:00

408 Request Timeout when sending data to Isilon WebHDFS

Greetings,

I'm currently deploying an hortonworks lab with OneFS for HDFS stack. I have the OneFS 8.2.1 simulator release.

I installed with success HDP 3.0.1, but I have un problem for initiate module like Spark2 which send at the first startup a 100MB tar file to Isilon. 

The command is :

curl -sS -L -w '%{http_code}' -X PUT --data-binary @/tmp/spark2/spark2-hdp-yarn-archive.tar.gz -H 'Contentpe: application/octet-str' 'http://isilonsczone-hdp.positiveyes.com:8082/webhdfs/v1/apps/spark2/spark2-hdp-yarn-archive.tar.gz?op=CREATE&user.name=hdfs&overwrite=True&permission=444

I tried to play with curl option as max-time but it's not working yet. I have the following answer :



408 Request Timeout

Request Timeout


Server timeout waiting for the HTTP request from the client.


So I transfer the file with hadoop fs -copyFromLocal :

hadoop fs -copyFromLocal /tmp/spark2/spark2-hdp-yarn-archive.tar.gz hdfs://isilonsczone-hdp:8020/hdp/apps/3.0.1.0-187/spark2/

The operation is successfull but it took at least 2 minutes (non production environment...)

To be sure it was not another problem than transfer time I tried curl transfer with a simple text file and it worked  like a charm :

curl -sS -L -w '%{http_code}' -X PUT --data-binary @/tmp/spark2/test/txt -H 'Content-Type: application/octet-str' 'http://isilonsczone-hdp.positiveyes.com:8082/webhdfs/v1/apps/spark2/test.txt?op=CREATE&user.name=hdfs&overwrite=True&permission=444'

I tried to change apache conf on my Isilon Cluster (/etc/mcp/template/apache24.conf + webui_httpd.conf) with :

TimeOut 600

+

          # RequestReadTimeout header=10-60,MinRate=1024 body=30-120,MinRate=1024
             RequestReadTimeout phase=0

and than restart WebUI through isi services -a isi_webui disable and enable

But nothing change, I have 408 timeout error once again.

I analyzed /var/log/apache log but nothing interresting.

I'm really grateful for all kind of help you can provide

Benoit

3 Apprentice

 • 

622 Posts

September 21st, 2020 21:00

No Events found!

Top