UNSOLVED

imota

updated

16 years ago

I

imota

8 Posts

0

19907

January 28th, 2011 12:00

2 x 6248 stack, 2 provider static IP routes, cannot use both & load balance?

Hi there,

 

I have been using 2 dell 6248s at my data center and prior to this issue I was having no problems using a single fiber link from my provider (Cogent) to access the internet:

Cogent provided me with a static gateway IP through which my internet traffic was to be routed. It was a /29 network where the first IP was the Cogent Gateway (for internet access) and the second IP was the port IP on my side (for Cogent traffic monitoring and for my fiber port to use).

The Cogent router IP is 38.113.166.49 and my port IP (on its own vlan) was 38.113.166.50

I had a static route set up to use this: 0.0.0.0 /0 38.113.166.49

The second segment was my actual IPs: a /27 network where the first usable IP was my local gateway (38.113.166.65) and all my servers were set up to use it as their gateway.

For redudancy, failover and load sharing I have a second link coming from Cogent through their Cisco's but it is a separate /29 network, with a separate set of gateways.

I tried initially to add the gateway information to the fiber port on my second switch and expected the dell switch to treat the two static routes as two possible paths to the internet for my public vlans. However the switch will not use both links, relying only on one of the two.

 

I tried setting up both static routes:

 

ip route 0.0.0.0 /0 38.113.166.49 metric 1

ip route 0.0.0.0 /0 38.113.185.33 metric 1

 

And physically unplugging the second link the switch would still use the first route. However unplugging the second link and leaving the first link up the switch refused to use the second static route, as would be expected.

 

Also there doesn't seem to be any possibility to use load balancing between the two routes. I am still trying to get them both to work and then worry about load balancing.

 

My servers all have dual nics, running linux with the two links bonded together in an round-robin load balancing mode (sequentially using both links for packet transmission). They are connected to both switches identically (For example, if one server is going to port 1/g3 with one nic, the second nic is connected to 2/g3, and so on) - this hopefully will help explain why my configuration looks the way it does.

I chatted with the Cogent engineer about this and they don't support ethernet bonding on their end, leaving that responsibility to the end-client. They are running Cisco's CEF protocol on the two /29 networks they are providing me so there is load balancing and failover setup on their end.

So in essence my question is: Can the 6248's support 2 static routes with load balancing and failover? If so, how do I go about getting it to work?

I've already tried assigning different metrics to the two static routes, won't work. Setting either route will work as long as there is only ONE ROUTE defined. Otherwise putting both routes will default to the first one defined in the config file.

This is my current configuration:

!Current Configuration:
!System Description "PowerConnect 6248, 2.2.0.3, VxWorks5.5.1"
!System Software Version 2.2.0.3
!
configure
vlan database
vlan  10,20,30,40,50,60
vlan association subnet 38.113.166.48 255.255.255.248 10
vlan association subnet 38.113.166.64 255.255.255.224 20
vlan association subnet 38.113.185.32 255.255.255.248 60
vlan association subnet 192.168.0.0 255.255.255.0 30
vlan association subnet 192.168.1.0 255.255.255.0 40
vlan association subnet 192.168.2.0 255.255.255.0 50
exit
sntp unicast client enable
sntp server pool.ntp.org
clock summer-time recurring Usa offset 60 zone "DST"
clock timezone -5 minutes 0 zone "EST"
stack
member 1 2
member 2 2
exit
ip address 192.168.3.2 255.255.255.0
ip default-gateway 192.168.3.1
ip name-server 66.28.0.14
ip name-server 66.28.0.45
ip name-server 66.28.0.61
logging file debug
ip routing
ip route 0.0.0.0 0.0.0.0 38.113.166.49
ip route 0.0.0.0 0.0.0.0 38.113.185.33
interface vlan 10
name "Cogent_T"
routing
ip address  38.113.166.50  255.255.255.248
exit
interface vlan 20
name "Public_IPs"
routing
ip address  38.113.166.65  255.255.255.224
exit
interface vlan 30
name "Local"
exit
interface vlan 40
name "Balanced"
exit
interface vlan 50
name "ndb"
exit
interface vlan 60
name "Cogent_T_2"
routing
ip address  38.113.185.34  255.255.255.248
exit
username "admin" password xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx level 15 encrypted
!
interface ethernet 1/g1
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 1/g3
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 1/g5
switchport mode general
switchport general pvid 30
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 30
exit
!
interface ethernet 1/g7
switchport mode general
switchport general pvid 50
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 50
exit
!
interface ethernet 1/g9
switchport mode general
switchport general pvid 30
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 30
exit
!
interface ethernet 1/g11
switchport mode general
switchport general pvid 50
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 50
exit
!
interface ethernet 1/g13
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 1/g15
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 1/g17
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 1/g19
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 1/g21
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30,40
exit
!
interface ethernet 1/g23
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30,40
exit
!
interface ethernet 1/g25
switchport mode general
switchport general pvid 30
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30,40
exit
!
interface ethernet 1/g27
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30,40
exit
!
interface ethernet 1/g29
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20
exit
!
interface ethernet 1/g31
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20
exit
!
interface ethernet 1/g41
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20
exit
!
interface ethernet 1/g43
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20
exit
!
interface ethernet 1/g45
switchport mode general
switchport general pvid 10
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 10
exit
!
interface ethernet 2/g1
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 2/g3
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 2/g5
switchport mode general
switchport general pvid 30
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 30
exit
!
interface ethernet 2/g7
switchport mode general
switchport general pvid 50
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 50
exit
!
interface ethernet 2/g9
switchport mode general
switchport general pvid 30
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 30
exit
!
interface ethernet 2/g11
switchport mode general
switchport general pvid 50
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 50
exit
!
interface ethernet 2/g13
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 2/g15
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 2/g17
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 2/g19
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30
exit
!
interface ethernet 2/g21
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30,40
exit
!
interface ethernet 2/g23
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30,40
exit
!
interface ethernet 2/g25
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30,40
exit
!
interface ethernet 2/g27
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20,30,40
exit
!
interface ethernet 2/g29
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20
exit
!
interface ethernet 2/g31
switchport mode general
switchport general pvid 20
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 20
exit
!
interface ethernet 2/g45
switchport mode general
switchport general pvid 60
no switchport general acceptable-frame-type tagged-only
switchport general allowed vlan add 60
exit
!
interface port-channel 1
shutdown
description 'INTERNET'
exit
!
interface port-channel 2
shutdown
description 'TEST'
exit
exit

 

 

 

  • bh1633

    909 Posts

    377

    0

    Posted January 31st, 2011 14:00

    Some suggestions:

    - update to the latest firmware (3.2.1.3)  FOLLOW UPGRADE INSTRUCTIONS CAREFULLY.  Specifically, update the bootcode from the bootcode.

    - check that spanning tree is not getting in your way.   You should just disable it on your congent ports.

    - Can your servers ping 38.113.166.50 and 38.113.185.34? (switch uplink IPs)

    - Can they ping 38.113.166.49 and 38.113.185.33? (congent switches IPs)

    - If it still does not work, post the output of:

    "show ip route"

    "show spanning-tree"

  • imota

    8 Posts

    377

    0

    Posted February 1st, 2011 12:00

    Hi and thank you for your suggestions.

     

    I started the process by upgrading my switches to the latest firmware and bootloader (3.2.1.3). Update went without a hitch, both units in the stack... EXCEPT that after the update both my fiber links appeared as DOWN (no LED on the port); I was sure it could not be the switches and started troubleshooting with Cogent and my Co-Lo - We used a light meter, loop, etc at my termination to make sure the signal was coming in and sure enough it was.

     

    I tried to look into any settings that might have been screwed up after the upgrade but everything looked fine, including a diagnostics on the 2 fiber ports (show fiber-ports optical-transceiver 1/g45 and 2/g45)

     

    console#show fiber-ports optical-transceiver 1/g45
                                       Output    Input
    Port     Temp  Voltage  Current     Power    Power   TX     LOS
              :emotion-44:   [Volt]     [mA]     [dBm]    [dBm]   Fault
    -------  ----  -------  -------   -------  -------   -----  ---
    1/g45    40.9    3.341     18.2    -6.126   -3.917   No     No
     Temp - Internally measured transceiver temperatures.
     Voltage - Internally measured supply voltage.
     Current - Measured TX bias current.
     Output Power - Measured optical output power relative to 1mW.
     Input Power - Measured optical power received relative to 1mW.
     TX Fault - Transmitter fault.
     LOS - Loss of signal.
    console#show fiber-ports optical-transceiver 2/g45
                                       Output    Input
    Port     Temp  Voltage  Current     Power    Power   TX     LOS
              :emotion-44:   [Volt]     [mA]     [dBm]    [dBm]   Fault
    -------  ----  -------  -------   -------  -------   -----  ---
    2/g45    41.6    3.321     19.6    -6.876   -8.407   No     No
     Temp - Internally measured transceiver temperatures.
     Voltage - Internally measured supply voltage.
     Current - Measured TX bias current.
     Output Power - Measured optical output power relative to 1mW.
     Input Power - Measured optical power received relative to 1mW.
     TX Fault - Transmitter fault.
     LOS - Loss of signal.

    I tried to power cycle and reboot the switches, did a full factory reset wipe of the settings and nothing with the fiber links!

    Then just before leaving the Data Center I decided to revert back to the 2.2.0.3 firmware and bootloader - Lo and behold everything started working again!

    Now it seems I have an issue with the upgrade and my configuration...

    My servers CAN ping:

    38.113.166.49

    38.113.166.50

    38.113.185.34

    but CANNOT ping:

    38.113.185.33

     

    The switch can ping all 4 IPs.

    For completeness here's the output for the requested commands you asked for, running on 2.2.0.3:

    console#show ip route

    Route Codes: R - RIP Derived, O - OSPF Derived, C - Connected, S - Static
           B - BGP Derived, IA - OSPF Inter Area
           E1 - OSPF External Type 1, E2 - OSPF External Type 2
           N1 - OSPF NSSA External Type 1, N2 - OSPF NSSA External Type 2

    S      0.0.0.0/0 [1/0] via 38.113.166.49,   vlan 10
                           via 38.113.185.33,   vlan 60
    C      38.113.166.48/29 [0/0] directly connected,   vlan 10
    C      38.113.166.64/27 [0/0] directly connected,   vlan 20
    C      38.113.185.32/29 [0/0] directly connected,   vlan 60

     

    console#show spanning-tree
    Spanning tree Enabled  mode rstp
    CST Regional Root:        80:00:A4:BA:DB:78:9F:04
    Regional Root Path Cost:  0
    ROOT ID
                  Address         80:00:A4:BA:DB:78:9F:04
                  This Switch is the Root.
                  Hello Time 2 Sec Max Age 20 sec Forward Delay 15 sec
    Interfaces

     Name    State   Prio.Nbr    Cost       Sts  Role PortFast  RootPort
    ------  -------- --------- ----------  ---- ----- --------  -------
     1/g1    Enabled  128.1             0   DIS  Disb      No      No
     1/g2    Enabled  128.2             0   DIS  Disb      No      No
     1/g3    Enabled  128.3         20000   FWD  Desg      No      No
     1/g4    Enabled  128.4             0   DIS  Disb      No      No

     1/g5    Enabled  128.5         20000   FWD  Desg      No      No
     1/g6    Enabled  128.6             0   DIS  Disb      No      No
     1/g7    Enabled  128.7         20000   FWD  Desg      No      No
     1/g8    Enabled  128.8             0   DIS  Disb      No      No
     1/g9    Enabled  128.9         20000   FWD  Desg      No      No
     1/g10   Enabled  128.10            0   DIS  Disb      No      No
     1/g11   Enabled  128.11        20000   FWD  Desg      No      No
     1/g12   Enabled  128.12            0   DIS  Disb      No      No
     1/g13   Enabled  128.13        20000   FWD  Desg      No      No
     1/g14   Enabled  128.14            0   DIS  Disb      No      No
     1/g15   Enabled  128.15        20000   FWD  Desg      No      No
     1/g16   Enabled  128.16            0   DIS  Disb      No      No
     1/g17   Enabled  128.17        20000   FWD  Desg      No      No
     1/g18   Enabled  128.18            0   DIS  Disb      No      No
     1/g19   Enabled  128.19        20000   FWD  Desg      No      No
     1/g20   Enabled  128.20            0   DIS  Disb      No      No
     1/g21   Enabled  128.21        20000   FWD  Desg      No      No
     1/g22   Enabled  128.22            0   DIS  Disb      No      No
     1/g23   Enabled  128.23        20000   FWD  Desg      No      No
     1/g24   Enabled  128.24            0   DIS  Disb      No      No
     1/g25   Enabled  128.25        20000   FWD  Desg      No      No
     1/g26   Enabled  128.26            0   DIS  Disb      No      No
     1/g27   Enabled  128.27        20000   FWD  Desg      No      No
     1/g28   Enabled  128.28            0   DIS  Disb      No      No
     1/g29   Enabled  128.29       200000   FWD  Desg      No      No
     1/g30   Enabled  128.30            0   DIS  Disb      No      No
     1/g31   Enabled  128.31        20000   FWD  Desg      No      No
     1/g32   Enabled  128.32            0   DIS  Disb      No      No
     1/g33   Enabled  128.33            0   DIS  Disb      No      No
     1/g34   Enabled  128.34            0   DIS  Disb      No      No
     1/g35   Enabled  128.35            0   DIS  Disb      No      No
     1/g36   Enabled  128.36            0   DIS  Disb      No      No
     1/g37   Enabled  128.37            0   DIS  Disb      No      No
     1/g38   Enabled  128.38            0   DIS  Disb      No      No
     1/g39   Enabled  128.39            0   DIS  Disb      No      No
     1/g40   Enabled  128.40            0   DIS  Disb      No      No
     1/g41   Enabled  128.41       200000   FWD  Desg      No      No
     1/g42   Enabled  128.42            0   DIS  Disb      No      No
     1/g43   Enabled  128.43       200000   FWD  Desg      No      No
     1/g44   Enabled  128.44            0   DIS  Disb      No      No
     1/g45   Disabled 128.45            0   FWD  Disb      No      No
     1/g46   Enabled  128.46            0   DIS  Disb      No      No
     1/g47   Enabled  128.47            0   DIS  Disb      No      No
     1/g48   Enabled  128.48            0   DIS  Disb      No      No
     1/xg1   Enabled  128.49            0   DIS  Disb      No      No
     1/xg2   Enabled  128.50            0   DIS  Disb      No      No
     1/xg3   Enabled  128.51            0   DIS  Disb      No      No
     1/xg4   Enabled  128.52            0   DIS  Disb      No      No
     2/g1    Enabled  128.53            0   DIS  Disb      No      No
     2/g2    Enabled  128.54            0   DIS  Disb      No      No
     2/g3    Enabled  128.55        20000   FWD  Desg      No      No
     2/g4    Enabled  128.56            0   DIS  Disb      No      No
     2/g5    Enabled  128.57        20000   FWD  Desg      No      No
     2/g6    Enabled  128.58            0   DIS  Disb      No      No
     2/g7    Enabled  128.59        20000   FWD  Desg      No      No
     2/g8    Enabled  128.60            0   DIS  Disb      No      No
     2/g9    Enabled  128.61        20000   FWD  Desg      No      No
     2/g10   Enabled  128.62            0   DIS  Disb      No      No
     2/g11   Enabled  128.63        20000   FWD  Desg      No      No
     2/g12   Enabled  128.64            0   DIS  Disb      No      No
     2/g13   Enabled  128.65            0   DIS  Disb      No      No
     2/g14   Enabled  128.66            0   DIS  Disb      No      No
     2/g15   Enabled  128.67        20000   FWD  Desg      No      No
     2/g16   Enabled  128.68            0   DIS  Disb      No      No
     2/g17   Enabled  128.69        20000   FWD  Desg      No      No
     2/g18   Enabled  128.70            0   DIS  Disb      No      No
     2/g19   Enabled  128.71        20000   FWD  Desg      No      No
     2/g20   Enabled  128.72            0   DIS  Disb      No      No
     2/g21   Enabled  128.73        20000   FWD  Desg      No      No
     2/g22   Enabled  128.74            0   DIS  Disb      No      No
     2/g23   Enabled  128.75        20000   FWD  Desg      No      No
     2/g24   Enabled  128.76            0   DIS  Disb      No      No
     2/g25   Enabled  128.77        20000   FWD  Desg      No      No
     2/g26   Enabled  128.78            0   DIS  Disb      No      No
     2/g27   Enabled  128.79        20000   FWD  Desg      No      No
     2/g28   Enabled  128.80            0   DIS  Disb      No      No
     2/g29   Enabled  128.81            0   DIS  Disb      No      No
     2/g30   Enabled  128.82            0   DIS  Disb      No      No
     2/g31   Enabled  128.83        20000   FWD  Desg      No      No
     2/g32   Enabled  128.84            0   DIS  Disb      No      No
     2/g33   Enabled  128.85            0   DIS  Disb      No      No
     2/g34   Enabled  128.86            0   DIS  Disb      No      No
     2/g35   Enabled  128.87            0   DIS  Disb      No      No
     2/g36   Enabled  128.88            0   DIS  Disb      No      No
     2/g37   Enabled  128.89            0   DIS  Disb      No      No
     2/g38   Enabled  128.90            0   DIS  Disb      No      No
     2/g39   Enabled  128.91            0   DIS  Disb      No      No
     2/g40   Enabled  128.92            0   DIS  Disb      No      No
     2/g41   Enabled  128.93            0   DIS  Disb      No      No
     2/g42   Enabled  128.94            0   DIS  Disb      No      No
     2/g43   Enabled  128.95            0   DIS  Disb      No      No
     2/g44   Enabled  128.96            0   DIS  Disb      No      No
     2/g45   Disabled 128.97            0   FWD  Disb      No      No
     2/g46   Enabled  128.98            0   DIS  Disb      No      No
     2/g47   Enabled  128.99            0   DIS  Disb      No      No
     2/g48   Enabled  128.100           0   DIS  Disb      No      No
     2/xg1   Enabled  128.101           0   DIS  Disb      No      No
     2/xg2   Enabled  128.102           0   DIS  Disb      No      No
     2/xg3   Enabled  128.103           0   DIS  Disb      No      No
     2/xg4   Enabled  128.104           0   DIS  Disb      No      No
     ch1     Enabled  128.626           0   DIS  Disb      No      No
     ch2     Enabled  128.627           0   DIS  Disb      No      No
     ch3     Enabled  128.628           0   DIS  Disb      No      No
     ch4     Enabled  128.629           0   DIS  Disb      No      No
     ch5     Enabled  128.630           0   DIS  Disb      No      No
     ch6     Enabled  128.631           0   DIS  Disb      No      No
     ch7     Enabled  128.632           0   DIS  Disb      No      No
     ch8     Enabled  128.633           0   DIS  Disb      No      No
     ch9     Enabled  128.634           0   DIS  Disb      No      No
     ch10    Enabled  128.635           0   DIS  Disb      No      No
     ch11    Enabled  128.636           0   DIS  Disb      No      No
     ch12    Enabled  128.637           0   DIS  Disb      No      No
     ch13    Enabled  128.638           0   DIS  Disb      No      No
     ch14    Enabled  128.639           0   DIS  Disb      No      No
     ch15    Enabled  128.640           0   DIS  Disb      No      No
     ch16    Enabled  128.641           0   DIS  Disb      No      No
     ch17    Enabled  128.642           0   DIS  Disb      No      No
     ch18    Enabled  128.643           0   DIS  Disb      No      No
     ch19    Enabled  128.644           0   DIS  Disb      No      No
     ch20    Enabled  128.645           0   DIS  Disb      No      No
     ch21    Enabled  128.646           0   DIS  Disb      No      No
     ch22    Enabled  128.647           0   DIS  Disb      No      No
     ch23    Enabled  128.648           0   DIS  Disb      No      No
     ch24    Enabled  128.649           0   DIS  Disb      No      No

    I'm really stumped here as to why upgrading the firmware would prevent my GigE links from working. They come in on LC fiber to an SFP transceiver...

  • bh1633

    909 Posts

    379

    0

    Posted February 1st, 2011 13:00

    Ask Cogent to enable autonegotiation on their fiber links.  They are probably forced to 1000 without negotiation.  The latest Dell firmware requires negotiation on the 1G fiber ports.

    I am still thinking about your routing problem.  My guess would be that the Cogent router is not responding to the ARP or ping from the 6248 switch since your servers can ping  38.113.185.34.  Can you ping 38.113.185.33 from the 6248?  If not, use a sniffer or mirror the port to a PC running wireshark to see anything obvious in the traffic between the switch and the router while pinging.

     

  • imota

    8 Posts

    379

    0

    Posted February 1st, 2011 13:00

    The switches can ping 38.113.185.33 no problem. It is the servers that can't.

    While on the phone with Cogent they asked me if I had auto-negotiation turned on on my ports, so I am assuming they already have that on? I can ask them again but even if they are forced to 1000, can't I set the fiber ports on the powerconnect to 1000?

    I would prefer to solve the routing problem before contacting them as upgrading to the latest firmware is basically introducing a new problem and bringing down my services. If I can have a starting point where both routes are properly working then I can attempt the firmware upgrade again. I wasn't too happy with the 2+hrs of downtime I had today trying to figure out why the ports weren't working after the upgrade...

    As a side note I should include here also my server configuration (since it is the device that can't ping the .33 IP) for completeness. All my servers are similarly configured so this one is enough as a representative sample. Each has 2 network cards (eth2 and eth3) bonded together. Each NIC is physically connected to the two switches separately (so this server, for example, is connected to 1/g3 and 2/g3) and balance-rr is a round-robin scheme that uses both NICs sequentially to transmit packets. (http://www.linuxhorizon.ro/bonding.html)

     

    # This file describes the network interfaces available on your system
    # and how to activate them. For more information, see interfaces(5).

    # The loopback network interface
    auto lo
    iface lo inet loopback

    auto bond0 bond0:1
    iface bond0 inet static
            address 38.113.166.72
            netmask 255.255.255.224
            gateway 38.113.166.65
            bond_mode balance-rr
            bond_miimon 100
            bond_downdelay 200
            bond_updelay 200
            bond_primary eth2
            bond_slaves none

    iface bond0:1 inet static
            address 192.168.0.224
            netmask 255.255.255.0
            bond_mode balance-rr
            bond_miimon 100
            bond_downdelay 200
            bond_updelay 200
            bond_primary eth2
            bond_slaves none

    auto eth2 eth3
    iface eth2 inet manual
            bond-master bond0
            bond-primary eth2 eth3

    iface eth3 inet manual
            bond-master bond0
            bond-primary eth2 eth3

     

  • bh1633

    909 Posts

    379

    0

    Posted February 1st, 2011 14:00

    Can you monitor the traffic on port 2/g45 while trying to ping 38.113.185.33 from the server?

    My guess is that the cogent router at address 38.113.185.33  does not have a route to your 38.113.165.64 /27 network.  So your packet capture will show the ICMP echo request going 38.113.185.33 with a source IP of 38.113.166.72 and no ICMP echo response.

     

  • imota

    8 Posts

    379

    0

    Posted February 1st, 2011 14:00

    If I try to run a ping to that IP I get 100% packet drop from the server. I can see the server attempting to send ICMP requests on iftop (linux network monitoring) but it doesn't go through.

    Would Cogent have to define a route on their end for their gateway to see my 38.113.166.64/27 block or do I have to define it as yet another static route on my stack?

     

    Thanks for your help!

  • imota

    8 Posts

    379

    0

    Posted February 1st, 2011 15:00

    Hey

     

    I will have to do this tomorrow morning but I will definitely give it a try and post back the results here.

     

    Thanks again.

  • bh1633

    909 Posts

    379

    0

    Posted February 1st, 2011 15:00

    It is my understanding that Cogent would have to define a route to your 38.113.166.64/27 network.

    Can you get a packet capture on the 2/g45 interface while pinging from the server?  This would be the most helpful.  Here are the switch commands to enable port monitoring of port 2/g45

    monitor session 1 destination interface 2/gXX
    monitor session 1 source interface 2/g45
    monitor session 1 mode

  • imota

    8 Posts

    379

    0

    Posted February 2nd, 2011 07:00

    It looks like Cogent never assigned a route from their .33 gateway to my IP block because the gateway was incorrectly terminated on a different router (different from my request). They are currently scheduling a re-termination service request to place the link on the same router and allow me to use both links.

     

    I will post back here once that's done and I've tried the configuration again. But you definitely put me on the right track as I asked the right questions and they found out the mistake! Thank you very much.

     

    Here's the current configuration on their Cisco gateways:

    Normal 0 false false false EN-US X-NONE X-NONE

    > Building configuration...

    >

    > Current configuration : 287 bytes

    > !

    > interface GigabitEthernet2/13

    > description

    > ip address 38.113.185.33 255.255.255.248

    > no ip redirects

    > no ip proxy-arp

    > carrier-delay 2

    > speed nonegotiate

    > flowcontrol send off

    > storm-control broadcast level 1.00

    > no cdp enable

    > end

    >

    > this has nothing routed to it

    >

    >

    > this one has the block in questions routed to it

    >

    > !

    > interface GigabitEthernet1/26

    > description

    > ip address 38.113.166.49 255.255.255.248

    > no ip redirects

    > no ip proxy-arp

    > carrier-delay 2

    > speed nonegotiate

    > flowcontrol send off

    > storm-control broadcast level 1.00

    > no cdp enable

    > end

    >

    > mpd02.yyz02#sho ruN | i 38.113.166.50

    > ip route 38.113.166.64 255.255.255.224 38.113.166.50 permanent name

     

  • bh1633

    909 Posts

    117

    0

    Posted February 2nd, 2011 08:00

    Yes.  This turns off autonegotiation for SFP ports.  Ask cogent to turn on autonegotiation on both ports if you upgrade the 6248 firmware to 3.x.

    Let me know when you get this working.