Issues with LAG across N4032 Switch Stack

I have had issues with stacking 2 N4032's.  First issue was the slave switch would disconnect and stop responding after 8 hours or so.  I applied firmware version 6.3.1.13 which appears to have solved that issue.

My current issue is that is that I have a LAG (LACP)  that has port on each switch in the stack and I have intermittent communication issues. I can ping a client through the LAG and some times it responds some times it doesn't.  

Both ports show as active when running "show interface Port -Channel 11" .  The FortiGate it is connecting too is also shows both ports active and connected.  The only thing I noticed is that the FortiGate shows its LACP speed as fast and the remote connection as LACP speed as slow,

Here is the config:

interface Te1/1/1
channel-group 11 mode active
exit

interface Te2/1/1
channel-group 11 mode active
exit

interface port-channel 11
switchport mode trunk
switchport trunk allowed vlan add 10,15,20,30,39,50
exit

Here is the weird part, if I move the port so it is on the same switch everything works fine.  

I have tried disabling STP on the port-channel and the individual ports with no success.

Any ideas?  Bad Switch?  Bad Firmware? Bad User?