So I ran into the same issue. Really stupid but I think the sub-domain of "VirtualMachines" is special to Avamar so you can't use/reuse it. I even tried deleting it then editing the client and pointing to it. To my disappointment you get the same error, as you've noted.

Try entering any other name for the domain. I used /vm_clients and tagged my rules (VM Tags) to that. And from there you'll see a sub-domain in my case //vm_clients. After which I was able to sync with vCenter and a few minutes later the VM's appeared within the sub domain /vm_clients.

Thanks for the reply. I need to allow system admins to do restores and I think everything has to be in the /VCENTER/VirtualMachines domain. Right now I’m just running a script to move the clients. Tried creating a sub-domain inside VirtualMachines, but the rule still wouldn’t allow me to move. Fails with the same error.

If anyone needs to have the VM clients in the VirtualMachines domain, this script works for me:

mccli client show --domain=/VCENTER > /home/admin/work/vm-clients.txt

i=0
while read p; do
   (( i++ ))
   if [ ${i} -gt 3 ]; then
      vm=$(echo $p | cut -d ' ' -f 1)
      if [ ! -z "$vm" ]; then
         if [ ${vm} != "VCENTER" ]; then
            echo "Move ${i}  ${vm}"
            mccli client move --domain=/VCENTER --new-domain=/VCENTER/VirtualMachines --name=${vm}
         fi
      fi
   fi
done