EPS files are a special format type.  See this info http://www.prepressure.com/formats/eps/fileformat.htm

Thanks fireberd, that's more than I would ever want to know about EPSs, but what I'm trying to find, is a way for the EPS files to show up as thumbnails in Windows Explorer, much like JPGs and other image file formats do.

I don't think that's possible because of the proprietary format.  EPS needs special software to view them, they are not like the common jpg, bmp, etc.  The Windows explorer does not support that proprietary format.

I thought that GIMP supported EPS but I can't confirm it.