Does My Dimension 4600 Support Booting From a USB Flash Drive?

Yes, it does.  I use a flash drive all the time on multiple 4600s.  But not all flash drives are equal.  It depends on how they're made to appear to the BIOS, and that can vary by manufacturer.  For example, I've had good luck with Kingston (assorted Data Travelers), but not with Corsairs.

I haven't tried a 4GB DataTraveler, but I have a couple of 512MB and 1GB DataTravelers that successfully boot DOS7.  I don't boot linux, so am not sure what, if any, issues that throws into the mix.

I do have a 4GB Lexar that I use for data storage, so I thought I'd experiment and see if I could make that bootable.  My usual technique is to plug in the flash drive, boot from a DOS CD or floppy, delete/create a partition on the flash drive, format and sys it.  I found that it would not boot if I did this from the 4600--looks like the 4600 BIOS sees it with odd geometry (16 heads, et al).  So I plugged it into a more recent machine (a new HP laptop) and repeated the procedure.  Everything worked, and the Lexar now boots in the 4600.  So, I can boot the 4600 from the Lexar if it's already prepped, but I couldn't prep it in the 4600.  This was with DOS, so I can't say what you'd run into with linux.