Routing outside the 6248

On the 6200 series switches, you can think of the "Management VLAN" as an OOB interface. It is not required that you manage the switch through this VLAN. You can manage the switch by accessing any of the IP addresses you have assigned to a VLAN.

since your cable modem is not aware of the other VLANs, it wont know how to direct returning traffic destined for these other VLANs. The port on the 6248 that faces upstream should be in access mode for VLAN 1. Then your upstream device needs to have the ability to have static routes in place that directs returning traffic.

Just like the switch has a static route directing traffic out.

ip route 0.0.0.0 0.0.0.0 192.168.1.1

The upstream device would need routes back.

ip route 192.168.5.0 255.255.255.0 192.168.1.25

ip route 192.168.6.0 255.255.255.0 192.168.1.25

typically this would be facilitated by a firewall. As an example, here is a KB article on static routes on a SonicWall.

http://dell.to/1SdpdUh

Thanks for your reply, Daniel. So you're saying that unless my cable modem gateway is able to configure static routes, that this basically won't work, correct?

Correct, you need a device that will have instructions on how to direct the returning traffic. There are open source software based options that you might look into. 

this list has several free ones.

http://bit.ly/1suhgzT

Let's assume for a moment that such a device was placed at 192.168.1.30 (so, on VLAN 1) as a virtual machine and that the 6248's interface at 192.168.1.25 still existed. Here are questions and assumptions I would ask to validate:

1. The router at 192.168.1.30 must have a default route for 0.0.0.0  0.0.0.0  192.168.1.1, correct?
2. The router at 192.168.1.30 must also have routes as you suggest directing traffic for VLANs 5 and 6 back to .30, correct?
3. All hosts on VLAN 1 must have their gateways reconfigured to point to .30, which would either direct that traffic out to the cable modem gateway at 192.168.1.1, or to either VLANs 5 or 6, correct?

1. This really depends on which product you are using. For instance, while configuring pfSense, the WAN interface is statically established, then the LAN interfaces. The default config will know to forward traffic out the WAN interface. Most routers/firewalls will have this dedicated WAN interface. 

2. The routes on 192.168.1.30 would direct vlan 5 and 6 traffic to 192.168.1.25.

3. The switch is still performing L3 functionality on your LAN, so end devices on VLAN 1 would still have a default gateway of 192.168.1.25. 

The static route on the switch would need to be changed to point to the Router/firewall.

ip route 0.0.0.0 0.0.0.0 192.168.1.30

From the little bit of information we have, it doesn't seem like the network you are working with is very large? How many devices are on the network? Is it imperative that they be in different VLANs? If possible, it may simplify things greatly if you can transition to a flat network. 

Thanks again for the clarification, Daniel. As a last question, if a separate router *were* added into the picture with an existing cable modem/gateway and the 6248 switch functioning as L3, what would the "outside" interface be like versus the "inside" if the router is straddled entirely on VLAN 1?

The network as it sits now is very simple and, up to this point, has been completely flat. I am putting the 6248 into the picture in preparation for network virtualization, and to mimic a customer's datacenter network infrastructure. For this, I need a dedicated transport zone on its own VLAN, and additional VLANs are needed to simulate different sites for purposes of DR/BC testing.

By default the Wan interface is typically going to have certain protective measures in place to restrict traffic from entering your network. While the LAN side allows connection to WAN. As an example here is an excerpt from pfSense install document.

• All incoming connections to WAN are blocked
• All outgoing connections from LAN are allowed
• NAT is performed on IPv4 traffic leaving WAN from the LAN subnet

When it comes to the flow of traffic though, the router will have the ability to take the returning traffic and direct it to the switch. The router may not know where 192.168.5.2 is, but because of the static route, it knows to forward the packet to 192.168.1.25. Where as with the Modem, it will receive a packet with a destination that it is not able to resolve.  

Here is a post on the Cisco forums that may help to better explain this.

http://bit.ly/1suwipv

If your ESXi host has enough interfaces, you can setup pfSense as a guest OS.

http://bit.ly/1X7CWOX

I hope this helps.

Thanks again, Daniel. I'm familiar with how such a device would work and the principles of it, just wasn't sure if its operation would work consistently with both legs in the same VLAN.

Let me run this scenario by you to see if return routing would work properly.

What if your cable modem/gateway you put on, say, VLAN 250 and set its IP as 192.168.250.2. You create VLAN 250 on the 6248, assign the switch's interface for that VLAN as 192.168.250.1, and plug an uplink from port g48 into the said cable modem/gateway setting it as an access port for VLAN 250. All other ports on the 6248 are trunk ports for various other VLANs except for 250. The default route on the switch is set for 192.168.250.2, thereby sending all traffic to that address if it doesn't have an internal path for it. If a host on VLAN 6 (ex., 192.168.6.26) attempts to communicate northbound to the Internet, it sends its packet to the gateway--the switch's interface of 192.168.6.1--and the switch in turn forwards that on to 192.168.250.2, the cable modem/gateway. Since VLAN 250 is only used for that one switch interface, it is forcing it to talk to the cable modem/gateway. Would return traffic from the Internet get routed properly back to 192.168.6.26? Or would this still not work?

Apologies for any frustration. I'm trying to determine what's the simplest way to make this work while keeping the network and infrastructure to a minimum.

It is not uncommon, and in several instances common practice, to dedicate subnets for connections from switch to router/firewall. Changing the setup to the one you described, would not change the outcome. The switch does not have the ability to NAT.