U4919DW, keeps switching USB inputs automatically

I purchased the U4919DW so I can work on 2 desktops at the same time with better productivity.  Thus far the unit works great as I have an Optiplex using the HDMI output powering 1 side with a workstation on the other side using dport.  I got my own hot key to switch USB inputs back and forth but on its own during the day the system will just switch itself to the 2nd input workstation which I will not be working on.  It does it 5-10 times a day and quite annoying as I can be working and bam it just switches on its own and I got to hit the keys to switch it back and at times it again switches back to the 2nd input immediately.  Any ideas on what can be causing this?