1 Copper

caching problem with MD3000i and Linux multipath

We have 2 Dell servers and a MD3000i SAN.

The configuration is the following:

host1 =----switch1=---=|SPA
               \ /       \ /              |
               X          x             |  MD3000i
               / \        / \             |
host2 =----switch2=---=|SPB

Both hosts are Fedora 13. We use dm-multipath (multipathd). Qemu-kvm is the hypervisor.
The problem is that when host1 writes something to the SAN host 2 won't see it until the host reestablishes a connection to the LUN (i.e. log of the iscsi targets and log in again). So writing a file on the LUN by host1 is not visible by host2 which has also mounted that LUN.

The consequense of the cachingproblem is that when a VM migrates to the other host it will crash because the virtual disk is different for host 1 and 2. Migrating with Qemu-kvm succeeds when both hosts are just rebooted since both hosts see the same virtual disk (no caching yet).

Another problem that arises is the when the virtual machine is "forced off" the guest OS won't start again and gives a lot of disk errors.

Does anyone have a clue which component is caching? Is it the MD3000i itself? iSCSI-client (iscsiadm)? Multipathd? LVM?

Thanks advance!

0 Kudos
3 Replies
Dev Mgr
5 Iridium

Re: caching problem with MD3000i and Linux multipath

I wouldn't depend on your multipathing solution too much as it's not certified for this storage solution and I've been told it's known to not work 100% (of the times) when a failover were to be needed.


Another thing is that your cabling is very wrong.

The cabling should be like the picture in this link. Each switch does need to use a different subnet obviously as they are isolated (from the LAN and eachother) networks.

Member since 2003

0 Kudos
3 Argentium

Re: caching problem with MD3000i and Linux multipath

Are your 2 hosts setup for clustered file systems?

This guide might be of help in setting it up as a cluster:


I, however, agree that dm-multipath is not supported and neither is Fedora core:).

I am not sure that it is a caching issue as much as a clustering issue.



0 Kudos
1 Copper

Re: caching problem with MD3000i and Linux multipath

I agree with Mohan - this sounds like you need Redhat (....or Centos) and GFS if you want to share those LUNs properly. Good luck!

0 Kudos