5 Posts
0
3003
August 12th, 2020 01:00
T640 split backplane for 2 controllers (cbl0009)
Hi,
On my new T640 server I have 32 disk capable backplane (which consists of two 16 disk capable backplanes with a "patch" between them). I understand it was (probably) possible to buy it with multiple controllers but that was overlooked so there is a single H730 controller in it.
I would now like to add another HBA card (dell compatible) in order to virtualise Freenas while retaining the H730 as a boot device and datastore (RAID:ed) for vmware. I don't want to just give Freenas access to a few of the drives (by running it in HBA mode) but to the entire HBA card connected to the lower backplane with 16 drives while using the H730 in RAID mode connected to the upper backplane.
The issue is I get a persistent warning on each boot (cbl0009) where the server cannot find the lower backplane as I've removed the "patch" between them to physically split them. I cant seem to figure out a way to shut that off, there is a "Enclosure configuration" named "BP14G+EXP 0:1" in idrac but that does not seem to be changeable.
I want the server to boot up without warning on the single backplane - as its possible to order it with a single backplane I assume it can be done, but can it be changed after it was ordered?
emiwal
5 Posts
0
August 29th, 2020 01:00
Short report-back on this in the event it helps someone else. I added a HBA330 internal card. The few observations I have is;
1. As was stated above, it's needed to use a supported PCI slot (or else it wont be visible to VMware - this might be because I use single CPU but still).
2. Splitting the backplane works as expected - the HBA card gets the disks in the lower half and the H730p gets the disks in the upper (no surprise there).
3. I did try to just keep the backplane connected together and use one of the channels for the H730p and one for the HBA330 to see if I could utilise all disk-slots (single channel SAS would be fine for my use-case) - just keeping in mind what disks are used where. This caused the server to not boot (H730p initialised but I guess that caused issues for the HBA330).
4. The error which says the backplane is no connected did not go away from connecting to another card - its possible to mute it in the LC and configure the server to bypass it in boot but it will always be there which isn't very elegant.
To summarise - the best solution in my mind to run a virtual ZFS solution on a Dell tower server is to use 2 cards, 1 for boot (RAID) and one for ZFS appliance/server (HBA) connected to one backplane each. That gives 16 drives to the ZFS appliance (max) and will throw a (mutable) error. I know there are supported solutions (740XD with one card connected to flex-bays comes to mind) for this but for a home/office solution - the above is the best way forward I could figure out.
DELL-Stefan R
Moderator
•
790 Posts
0
August 12th, 2020 06:00
Hi emiwal,
it is like you already thought, the system is also sold with two 16 bay backplanes, but also with two controller cards. If the system was bought with the "single" 32 bay backplane it's not possible to re-configure it as you want.
I'm also unsure if it is possible to install the 2 16 bay backplanes in this system - the chassis may not offer the needed bolds to fix the backplanes.
Would be a trail'n'error session, but I can not recommend this, it would be very costly for you, you need the two backplanes + cables + additional controller.
I'm not firm with Freenas, but wouldn't it be easier if you virtualize it completely, so you can decide which datastore get's connected to it?
Best regards,
Stefan
emiwal
5 Posts
0
August 12th, 2020 07:00
Hi Stefan,
Thank you for your reply! The second backplane is already in the server so there are two and they are separated by a SAS-"patch" cable (as in a 10cm long standard internal SAS cable) that I removed (and will replace with a connection to a second HBA card). So from a technical standpoint, its doable. My question is - do I have to live with the server pausing on each boot saying "backplane missing" or can that somehow be turned off or reconfigured?
Its not an option (sadly) to use the single controller for the setup I have in mind but if its not possible to avoid the error, I will have to re-think my setup and not use Freenas.
emiwal
5 Posts
0
August 12th, 2020 07:00
Addition:
It seems to me there are "profiles" in the LC (or idrac or somewhere) that tells it what kind of backplane solution is implemented. I have profile "BP14G+EXP 0:1" which I take it, is standard 16 disk backplane (PB14G) + an expansion (EXP 0:1).
I now realise it seems the tower version cannot be ordered with multiple HBA/PERC cards other then for external JBODs (usually work with rack version which can, but this is for on-prem so its tower) but it CAN be ordered with a single backplane for 16 drives and thats really what I want to achieve (while retaining the extra "stand alone" backplane in the chassie).
So a reconfiguration "somewhere" to a 16 disk backplane, no expansion.
Dell-DylanJ
6 Operator
•
2.9K Posts
0
August 12th, 2020 08:00
Hello,
I'm not seeing that what you're looking to do was validated. That having been said, that doesn't necessarily mean it won't work.
As for the pre-boot error message, I don't think you can truly shut it off. I think the BIOS has an option to ignore pre-boot errors (BIOS, Miscellaneous Settings, F1/F2 Prompt on Error), but I'd still expect the error to be logged in the LC log, but likely the SEL, as well.
The internal HBA that was validated for this system is the HBA330. There are a few options on where you can install that, those being PCI slots 0, 1, and 3. Adding in the controller and disabling the prompt may give you the behavior that you're looking for, but I can't say that authoritatively, since the config wasn't validated. I'd be inclined to think that either replacing your controller with the HBA330 or passing FreeNAS individual disk RAID 0 volumes would be the most workable option.
Hopefully, this information is of some use. I'd be curious to hear about how it works out in your testing. It may well benefit another forum user in the future, too. Let us know if you need any additional info or clarification.
EDIT: I thought this wasn't an option, but I was just in an H730's config utility. You might consider a much simpler solution. If you go to the PD Mgmt section, you can convert a disk to non-RAID mode and pass it straight through. Just making this change might give you what you're looking for.
emiwal
5 Posts
0
August 13th, 2020 00:00
Hi,
Thank you for your reply! I will order a HBA330 and see if somehow the error goes away if the BP is just connected to a card (whichever - guess it wont) - if not I might just enable ignore pre-boot errors (but the server will constantly glow orange which is annoying).
As for your edit, it was my original plan to just do a RAID volume on the first 3-5 disks for booting / VMware datastore and then exposing the remaining disks directly (non raid) and pass them through in VMware. Apparently that can cause dataloss with FreeNAS / ZFS (as well as vsan) - I cannot find the link where I read it again but apparently the combination of raid and non-raid will mess with ZFS in some ways.
As for RAID0 - it can be done but will require disabling the both scrubbing (which is one of the reasons I want ZFS/FreeNAS) and write caching on the controller. Not sure if the risk of dataloss (as per above) still remains.
The only workable solution seems to be a PCI passthrough of a dedicated HBA to FreeNAS in VMware.