Unsolved
2 Posts
0
1110
July 23rd, 2020 08:00
408 Request Timeout when sending data to Isilon WebHDFS
Greetings,
I'm currently deploying an hortonworks lab with OneFS for HDFS stack. I have the OneFS 8.2.1 simulator release.
I installed with success HDP 3.0.1, but I have un problem for initiate module like Spark2 which send at the first startup a 100MB tar file to Isilon.
The command is :
curl -sS -L -w '%{http_code}' -X PUT --data-binary @/tmp/spark2/spark2-hdp-yarn-archive.tar.gz -H 'Contentpe: application/octet-str' 'http://isilonsczone-hdp.positiveyes.com:8082/webhdfs/v1/apps/spark2/spark2-hdp-yarn-archive.tar.gz?op=CREATE&user.name=hdfs&overwrite=True&permission=444
I tried to play with curl option as max-time but it's not working yet. I have the following answer :
Request Timeout
Server timeout waiting for the HTTP request from the client.
So I transfer the file with hadoop fs -copyFromLocal :
hadoop fs -copyFromLocal /tmp/spark2/spark2-hdp-yarn-archive.tar.gz hdfs://isilonsczone-hdp:8020/hdp/apps/3.0.1.0-187/spark2/
The operation is successfull but it took at least 2 minutes (non production environment...)
To be sure it was not another problem than transfer time I tried curl transfer with a simple text file and it worked like a charm :
curl -sS -L -w '%{http_code}' -X PUT --data-binary @/tmp/spark2/test/txt -H 'Content-Type: application/octet-str' 'http://isilonsczone-hdp.positiveyes.com:8082/webhdfs/v1/apps/spark2/test.txt?op=CREATE&user.name=hdfs&overwrite=True&permission=444'
I tried to change apache conf on my Isilon Cluster (/etc/mcp/template/apache24.conf + webui_httpd.conf) with :
TimeOut 600
+
# RequestReadTimeout header=10-60,MinRate=1024 body=30-120,MinRate=1024
RequestReadTimeout phase=0
and than restart WebUI through isi services -a isi_webui disable and enable
But nothing change, I have 408 timeout error once again.
I analyzed /var/log/apache log but nothing interresting.
I'm really grateful for all kind of help you can provide
Benoit



Phil.Lam
3 Apprentice
•
622 Posts
0
September 21st, 2020 21:00
@benoitvillage
Have you checked this out?
http://www.unstructureddatatips.com/hadoop-rest-api-webhdfs-on-onefs/