Ah! That is it! It's in the auto generated sitemap. So I am actually pointing to it, not form my site but from my sitemap!!
Keep me right here, can I add a noindex for all bots? i.e. is <meta...
Type: Posts; User: Turan Mirza; Keyword(s):
Re: sitemap and or robots.txt question
Ah! That is it! It's in the auto generated sitemap. So I am actually pointing to it, not form my site but from my sitemap!!
Keep me right here, can I add a noindex for all bots? i.e. is <meta...
sitemap and or robots.txt question
I have a custom 404 page as I felt that was the right thing to do. Then a few days ago I realised it should maybe be disallow'ed in the robots.txt file so they would not crawl the page.
Now, I...