1) I definitely would encourage you to open a support ticket on this

Is this a brand new clean image from Dell support site? 

https://appservices.wyse.com/pages/serviceandsupport/support/dlOraFW.asp?which=193&model=5060(Windows%20Embedded%20(WES))

No other software installed?

There is a hotfix on the page link above for the 5060, Win10IOT that address write filter issues you will want to make sure you have on the device.

• https://appservices.wyse.com/supportdownload/WES/addons/UWF_OVERLAY_HOTFIX_WIE10.msi

You may also want to take a look at the recently released Overly Optimizer

• http://downloads.dell.com/wyse/OverlayOptimizer/1.0/Overlay_Optimizer_1.0_Release%20Notes.pdf
• http://downloads.dell.com/wyse/OverlayOptimizer/1.0/OverlayOptimizer_1.0.0.4018.exe

Thanks

The recommendation from buffalobound for the UWF fix is the right recommendation.  Also, make sure you do have the UWF/FBWF OFF.  Otherwise, any updates you apply with FBWF still enabled will cause the device to flush said updates on next reboot. 

Please note that the thin clients are not designed to be updated via WUS, You should be getting the Security updates from www.dell.com/support and use WMS to deploy.