This post is more than 5 years old
1 Rookie
•
16 Posts
0
1170
March 24th, 2015 10:00
ISILON OneFS 7.1.0.2 - hdfs block size ?
Currently on my compute cluster
block size is set to 128 MB.
I am not passing any custom split size etc.. in my MapReduce jobs, but I see the split size is 64MB.
Does the ISILON has any block size implemented in OneFS hdfs process? Please explain.
Thanks in advance.
Ashok Beesam
0 events found
No Events found!
jason_f1
8 Posts
0
March 24th, 2015 10:00
You can set the HDFS block size on Isilon, I believe the default is 64MB. You can view your current settings with: 'isi hdfs settings view' and change it with 'isi hdfs settings modify --default-block-size=128MB'.
I did some work last summer on Hadoop with Isilon and saw some definite advantages/efficiencies to using larger HDFS block sizes if your files are as large or larger than the selected HDFS block size.