Unsolved
This post is more than 5 years old
18 Posts
0
479762
December 11th, 2012 19:00
H310 performance problems
I posted a message in the the storage community forum but not sure if that was the best place for it. http://en.community.dell.com/techcenter/storage/f/4466/p/19480687/20249968.aspx#20249968
Anyhow, I have done some additional testing and I think I need to have my drives hooked up in a different way or have a different/additional drive controller.
I have an R720xd with 12 x 2TB SAS drives in the front and a pair of SSD in the back. They are all hooked up to an H310 and are setup as non-RAID in the bios.
In my testing of sequential throughput with dd if I am writing to one of the SSDs the performance is as expected (almost 400MB/s) however as soon as I also write to one of the spinning drives the throughput drops to 140MB/s on the SSD to match that of the spinning disk. There is still headroom on the CPU so I think this is different than my other posting referenced above. If I write to both SSDs they average over 370MB/s each. So basically I am loosing some of the benefit of having an SSD when writing concurrently to one of the spinning disks.
I am guessing I would benefit from having a separate controller for the SSDs. Thoughts on this? I am thinking I would keep the SSDs on the current H310 and then move the others (front backplane) to a different controller. Any suggestions on which controller which might help me get around the issue from my other post (or is that a not enough CPU problem)? I hear there is an IT firmware that can be applied to LSI based cards would this be of benefit to me for the 12 drives (I am not going to use hardware raid). Should I go for a PERC H200 since that supposedly is already in IT mode?
thanks.
tutao
2 Posts
0
July 5th, 2013 02:00
Which controller should be used for a 12 disk JBOD configuration? As I understand, the h310 and the h710 does not support JBOD...
Kind Regards
Matthias
theflash1932
11 Legend
•
16.3K Posts
0
July 5th, 2013 08:00
The H310 supports JBOD, but the H710 does not.
create123
18 Posts
0
July 5th, 2013 10:00
Not sure if any of the newer firmware updates fixed anything.
If you really want JBOD get a Dell 1U system with a separate drive enclosure and use an LSI card (the Dell SAS HBA card may work too) . I don't think it is worth the headache to try making the R720xd work in JBOD.
I ended up sucumbing to the Dell way and setup HW RAID using a H710P (they were nice enough to swap the cards at no cost). I am using the SSDs for CacheCade.
tutao
2 Posts
0
July 5th, 2013 10:00
Yes, but what is DELLs proposed way for JBOD without performance degradation?
theflash1932
11 Legend
•
16.3K Posts
0
July 5th, 2013 11:00
Yes, they have a 6Gbps SAS HBA as well ... those ARE Dell's proposed solutions for JBOD. I'm with create ... if you want JBOD, I'd go third-party too.
larkw
2 Posts
0
July 9th, 2013 12:00
In a newly purchased R720xd with H310 + 12 2TB disk, I run 12 dd's simultaneously and get the result as the following:
(
OS: debian wheezy amd64, base system
)
create123
18 Posts
0
July 9th, 2013 13:00
That looks a heck of alot better than what I had experienced. As I added additional dd after 4 drives the w/s would steadily drop on each such that the aggregate was around 500MB/s. I saw this with ubuntu 12.04 and CentOS 6.
Looks like JBOD is now possible.
Did you also try with an SSD in the back slots? I'm interested if writing to those is still impacted by writes to the main drives.
larkw
2 Posts
0
July 10th, 2013 11:00
I use fio to test again and the same result. But OTOH, I contact familiar sysadm in other company and they are on the schedule to purchase a batch of storage servers, they do extra test and get bad result (R:W is 650MB/s:550MB/s) on the test server dell provided.
I will compare our servers with their test server, to sort out what is the key difference.
trm201
9 Posts
0
July 18th, 2013 18:00
Did you get any results yet?
We were about to order R720XD servers with the h310 until we read this post.
We are now looking at another option, which would be to use LSI controllers as an expansion card. We've read a few forums posts stating that people have tried this, and had a few problems with the correct connections to the backplane, and also the length of the wiring ( The cables need to be 1M long - See last post here: www.nexentastor.org/.../9447 ).
I have found documentation on LSI's website stating that the LSI SAS 9207-8i PCIe card is supported in the R720xd: www.lsi.com/.../Dell.aspx
I also found this: www.lsi.com/.../Dell_QIG.pdf which seems to be Dell documentation on how to install the controller.
We've got our sales account guys at dell looking into getting a CFI setup for us to get the servers directly from dell with this setup as standard.
create123
18 Posts
0
July 18th, 2013 19:00
Connecting to the backplane is not as easy as you would think. The dell cable uses a custom plug that combines 2 standard plugs which wont work with any card out there. If you use standard cables, good luck finding one that is angled the right way so that you can close the case.
Here is what another person that got it to work responded to me on hardforum:
hardforum.com/showthread.php
trm201
9 Posts
0
July 18th, 2013 19:00
That setup is rather crazy, and not something we'd even consider! I'm not sure I understand why they've had to include extra power. We know of some R710s that are using custom LSI HBAs and there was no problem with power.
Hopefully some of those issues are fixed with the 9207-8i as this is official dell documentation on how to install it : www.lsi.com/.../Dell_QIG.pdf
It seems there are other people having issues with the cabling though:
community.spiceworks.com/.../334913-r720xd-w-lsi-card-cables
AxisTele
1 Message
0
August 1st, 2013 14:00
Create123,
As far as i remember you had 12 SAS drives in the backplane and two SATA SSD in the back slots of the server, while LarkW has 12 SATA drives and getting much better performance. Also please confirm that you did your original tests (with poor performance maxed out at 500-550) on Ubuntu or any other Linux, not on Nexenta or OI, cause I recall on you mentioning ZIL and L2ARC...
I'm a big fan of ZFS myself, but I'm not gonna use it on this server. Just wanted to make sure that we have no issues on Linux (preferably XenServer, but it's linux based).
Thanks!
create123
18 Posts
0
August 1st, 2013 17:00
AxisTele,
I first ran into this after installing ZoL on Ubuntu 12.04. Since that was not a supported OS and filesystem combo I experimented with CentOS 6 and mdadm which still had a problem and then finally with plain old JBOD and etx4.
For JBOD testing I prepped with the following (after booting from a live centos image through idrac):
for i in {a..l} ; do
wipefs -a /dev/sd${i}1; mkfs.ext4 /dev/sd${i}1; mkdir /mnt/sd${i}1; mount /dev/sd${i}1 /mnt/sd${i}1) &
done
Then I ran my test:
for i in {a..l} ; do
dd if=/dev/zero of=/mnt/sd${i}1/test.dd oflag=nonblock bs=50k count=400000 &
sleep 20
done
And watched in another window with "iostat -m 2" and saw the total MB_wrtn/s get up to a cumulative ~550 and then stay there.
As for the HW, you are correct, I had 12 SAS drives in the front and 2 SATA SSD in the back. Although I don't remember if I had them in during the JBOD testing as I was trying to have a supported setup and the SSD were aftermarket. Interesting observation ... it is possible that the mixing of SAS and SATA could have been the problem especially with the other issue of the SSDs getting limited in their write throughput if one of the SAS drives was accessed.
Anyhow I used the above results and Dell support acknowledged the problem and sent me a 710P so I am doing HW RAID10 now.
Bernie from Mel
1 Message
0
September 24th, 2013 18:00
This is an interesting thread.
We have been testing a 720xd in various configurations here.
We have 12 4Tb SATA drives in the front, and 2 300Gb SAS drives in the flexbay.
We were provided with a H310, and also an add on LSI 9207-8i with appropriate cables to connect it to the back plane.
Dell will sell you a 720xd with an LSI 9207-8i correctly cabled to the "front" backplane. It is a supported configuration now.
This afforded us the opportunity to test with 3 configurations:
1) The H310 with hardware raid1 on the 2 flexbay drives, and JBOD mode for the 12 front drives.
2) The LSI 9207-8i controlling all drives.
3) The LSI 9207-8i running the 12 front drives, and the H310 running the 2 rear flexbay drives in raid1 mode. This used the existing H310 SAS cable to connect to the flexbay SAS connector. This is apparently unsupported by DELL at this time- but hey - it works fine!
The OS in use was Oracle Solaris 11.1, and all filesystems are ZFS - the 12 fronts disks forming a raidz2 pool (as this is how we want to configure the machine).
iozone results show that there really isn't much in it between all the configurations, and filebench shows pretty much the sameness as well.
The only other test worthy of note was that we did the big "dd" test to each disk with a sleep of 20 seconds.
We made 12 zfs pools each with one disk, broke the mirror on the 2 rear flexbay disks and also made a single disk zpool on one of it's disks. Then we dd-ed from /dev/zero to a file in each pool whilst watching iostat every 2 seconds.
Yes, when the io gets up high, the rear flexbay disks throughput did significantly slow - but the the 12 sata front disks keep pretty much around the same level of io.
In our case, this probably isn't a problem, as the flexbay drives are dedicated OS disks - ie not doing any really critical IO apart from what the OS needs for itself - and the 12 4Tb disks are for data. As long as the data disks don't suffer speed wise we'll be happy. Of course, running zfs raidz2 will also slow the io on the 12 4Tb disks as well.
We propose doing "real world" test over the next few days - this will tell us more.
The results can be found at http://people.eng.unimelb.edu.au/bernie/R720xd
How much are the results influenced by the OS and software raidz2 with zfs? Probably a lot. I suspect that there may also be issues with the "front" backplane and the expander on it rather than the actual controller. For our purposes though, seems like the machine will do OK in either configuration.