PC 5324 Slow Link to another Switch

We have our 5324 in our main office connected to a Fiber Media converter which connects to our remote office.

I am noticing slow speeds somewhere between the devices.

Here is a little overview:

PC 5324 Switch:
**port of switch**\ PCs and Servers
**port of switch**\ Fiber Media Converter plugged into the above 5324 via cat5e
***\
10Mbit Fiber to remote office...
***\
PC 5324 Switch at remote office
**port of switch**\ Fiber Media Converter plugged into the above 5324 via cat5e
**port of switch**\ Some Servers

So, as you can see we have some servers on each end of the fiber, connected by our switches.

Ideally, we should see transfer speeds of 1.23MB/s each way, right? Well, we don't.

I do see 1.23MB/s one direction, but not the other. HOWEVER - if I remove the 5324 Switch at our main office, plug a laptop directly into the media converter, and test transfer speeds between the networks: 1.23MB/s both ways.

Does anyone know what this means? How to pinpoint it?

I have turned off AutoNeg on ports that connect to the media converter, and hard coded them to 100/Full.
I have had the ISP confirm that the Fiber is running 10Mbps. Yes, it is.