I have a fairly high speed internet service (300Mbps). I have another desktop plug to the same dumb switch and using fast.com to confirm my speed is about what I expect. But, when I test the speed with a XPS 9550 + TB16 ethernet connected to the same switch, I'm only getting around 100Mbps even after I've upgraded to the latest Thunderbolt controller driver and ethernet driver.
Have you installed the recently released update for the TB16's ASMedia USB controller driver? There's also an Ethernet FIRMWARE update for it, for what it's worth. The fixes don't list anything about this, but the release notes don't always list all fixes. If neither of those fixes the issue, is this consistently reproducible, i.e. as soon as you connect audio, your bandwidth dips, and as soon as you disconnect, it goes right back up?