You will probably find that having the long (analog) headphone wire along-side the computer tower it attracting EMI/EMF interference like an antenna.

You could try wrapping the headphone-cord around a ferrite core-ring or something similar.

Another solution ...

Get yourself a low-cost digital USB headset. It will plug-into the front/top again.