5050SFF stock psu is 180w.

you can easily upgrade to Dell oem 240w psu for 5050MT.  the psu is same dimension and a direct drop in.

https://i.dell.com/sites/csdocuments/Shared-Content_data-Sheets_Documents/en/OptiPlex-5050-Towers-Technical-Specifications.pdf

for newer psu you can also install Dell 260w psu for 5070 MT which has same connectors as 5050 psu and of identical dimension.

https://www.compuram.de/system/008232/dell_optiplex_5070-sff.pdf

​Optiplex 5050 MT PSU​
 
​6F0T1 240W Acbel​
​22M8Y 240W Acbel​
​D5PCJ 240W Bestec​
​W2DN4 240W Bestec​
​4GJV9 240W Chicony​
​J61WF 240W Chicony​
​5WNDJ 240W Delta​
​CJG5K 240W Delta​
​JVW1K 240W Liteon​

​Optiplex 7050 MT PSU​
 
​22M8Y 240W Acbel​
​6F0T1 240W Acbel​
​D5PCJ 240W Bestec​
​W2DN4 240W Bestec​
​4GJV9 240W Chicony​
​J61WF 240W Chicony​
​5WNDJ 240W Delta​
​CJG5K 240W Delta​
​DK87P 240W Liteon​
​JVW1K 240W Liteon​

​Optiplex 7060 MT =​
​52CKC 260W Acbel​
​GDCXM 260W Chicony​
​4THK0 260W Delta​
​3YNRJ 260W LiteOn​
​TP18W 260W LiteOn​

​Optiplex 7070MT​

​3YNRJ 260W LiteOn Bronze​
TP18W 260W Liteon Platinum​

For budget reasons, if I was to stay with the stock 180W what would be the best GPU upgrade that would fit my SFF Optiplex 5050?

Low-Tier Cheap to High Tier list

choice are quite limited with 180w psu which is weak for most good gpu.  even GT1030 needs more psu power unless you do not use any mechanical hdd and dvd

5050 SFF can accommodate low profile double slot width gpu in the PCIex4 slot.

validated Dell OEM video cards when using our OEM 200W/260W PSU =

NJJMW AMD Radeon RX550 OP14L, 4GB, Low Profile, mDP/mDP/DP, 55W
R9J9P AMD Radeon RX550 OP14L, 4GB, Low Profile, mDP/mDP/DP, 55W
 

9G0TG AMD Radeon R5-430 OU14L, 2GB, Low Profile, DP/DP, 50W
 K6T46 AMD Radeon R5-430 OUGA14, 2GB, Low Profile, DP/DP, 50W
9VHW0 AMD Radeon R5-430 OUGA14L, 2GB, Low Profile, DP/DP, 50W

CNRTY Nvidia GeForce GT 730 OU14LN, 2GB, Low Profile, DP/DP, 30W

@redxps630​ I've heard that the RX6400/6500 is good and has low-profile variants that might be compatible. I also considered that a 500W power supply could fit the case. For instance, the Dell G5 PSU could be a suitable upgrade as long as the power connectors match, which should presumably be safe. Fun fact: Dell offered to recycle this PC for me based on their PC scanning tool and a survey they walked me through.

"I've run VRChat before on a non-VR-rated card, similar to the one currently in my system, by using SteamVR with the Rift-S via DisplayPort. Like any budget gamer, I just turned down certain settings, and the graphics were good enough for avatar/video conferencing or hanging out by a mirror. As long as the GPU has a display connector compatible with a VR headset, this PC should be able to run VR with tweaked settings.

This PC will essentially be a backup, so if my main PC crashes, I can still get by. It's similar to how a backup generator provides less power than the power company but still keeps the main appliances running. I don't expect superior performance or graphics, but I do care that it can reliably run most AAA games or other applications at its own peak performance reliably regardless of the FPS being it'd be above 15-25 a lil bit on most cases, even with a budget-tier card.

I'm also curious to know if others have used a similar or lower-tier card for VR. Thanks for all your help

if you upgrade to Dell 500w psu Y7R0X it can support a serious low profile gpu

this psu was validated in 7050 etc

User validated Y7R0X 

Optiplex 7050

https://www.dell.com/community/en/conversations/optiplex-desktops/upgrading-power-supply-in-optiplex-with-a-dell-psu/647fa2d9f4ccf8a8de88264f?keyword=Y7R0X