Is this because I do not use Xara to host my website?
No, this is not the cause. Many ISP's offer more speed, services (such as Control Panel), and support.

Even though Xara only makes the latest changes when you publish, it does delete previous files which can take some time.