I spent many years coding everything by hand, refining things down to the byte, and running my sites through every compliance checker imaginable. As the webmaster at a college that received a fair amount of traffic, this was the right choice. It's even more important if your site is receiving massive traffic, as every piece of bloat is magnified a million times.
All that being said, nowadays on my personal sites I build just about everything in Xara. For the traffic levels the types of sites most folks here are building, with just the smallest degree of caution the page size difference is completely insignificant considering the speed of most connections, and the pages are often actually leaner than many template driven solutions. While the perfectionist is toiling on refining a single site to pristine authenticated beauty (and having to stay on top of every annoying hack to maintain browser compatibility) you can build 10+ sites in Xara. And for many situations, that is exactly the right choice.