Thank you for your reply. Yes it is the same symptom as 9370 described here for my 9560.
Basically, my laptop stays quiet when doing normal light works like web surfing and using Office. However, when plugged in the power adaptor, the fan will generate some noise. I'm pretty sure the noise comes from the fan. When under load, the fan noise will become louder. For the light work, if I unplugged the AC adaptor noise will disappear and the whole system will become quiet again.
To confirm this, I installed Intel Extreme Tuning Utility and I saw that with AC adaptor plugged in, when temperature goes higher than ~ 47 C, fan noise will come, whereas if unplugged, the fan won't generate any noise even when the temperature goes to ~ 52 C. Clearly there is a difference. I use the same power mode (selected in task bar) for both case.
I tried to edit the power plan as described here by setting the passive cooling for the case when AC adaptor plugged in: https://www.reddit.com/r/Dell/comments/7hwyjf/xps_15_9560_fan_noise/ and I also tried using the "Quiet" profile in Dell Power Manager but they don't work.
There are some other users also complained this issue: https://www.dell.com/community/XPS/Dell-XPS-15-9560-Fan-Always-On-and-Always-Loud/td-p/6056537 https://www.dell.com/community/Laptops-General/Dell-XPS-15-9560-fan-noise-when-charging/td-p/5096456
All of the drivers including BIOS and Windows system are up-to-date.