MUST BE  LOW DENSITY 4GB module 512MB x 64 , INTEL Compatible, 1.35v  16 gigs is max and 4 modules is max

MUST NOT BE  Buffered, Registered, ECC, HIGH DENSITY 8GB module 1024MB x 64 . 

https://www.crucial.com/usa/en/optiplex-780-desktop/CT7339271

DDR3 PC3-12800 • CL=11 • UNBUFFERED • NON-ECC • DDR3-1600 • 1.35V • 512MEG X 64 •

Crucial RAM is guaranteed to work in Dell's.  1066MHz is the max speed a 780 will take.  In the fine print on pg. 9 - "This system supports both 1066MHz and 1333MHz DDR3 memory, however the memory will operate at 1066MHz speed due to chipset specifications."

Optiplex 780 Tech Guide