Meta Tags and Templates from Database

[BobInce]

<blockquote>

I am using the robots.htm file in the way I have believed was normal practice</p></blockquote>

I can't vouch for 'normal practice', but I for one have never seen this approach used before!</p>

Currently, in IE, I see an explanation page with a link to the 'non-JavaScript' version of the page at the bottom. But the non-JavaScript page turns out to be a redirect implemented in JavaScript, so I don't get any further. (Tip: JavaScript and meta-refresh are not good substitutes for real HTTP redirects.)</p>

How about linking directly to sitemap.htm from the explanation page instead? Or even better, including the sitemap in index1.htm as the noscript content?</p>

[stlewis]

Thanks for these tips, Andrew and Jen. I have both a robots.txt file for exclusions and a robots.htm file for inclusions. The latter is part of the Joust environment that I use, but I can replace it with a site map better integrated into the user-visible part of the site.

How about this for a set of guidelines for people to shoot at? I realise it is a constantly evolving situation; this is what makes handling it such hard work.

For all sites:

Use a robots.txt file to list directories/files you don't want crawled.
Do not use a file of links to pages which does not appear in user-visible navigation; this will irritate spiders.
Include meta-keywords if you wish but they are now given less weight. The meta-description has most value but more for the users of a search engine.

For a frame-based site:

Use a sitemap.htm file to help spiders reach the pages you want. This file must be included in the user-visible site navigation.
Also, either put these same links to all your pages as a sitemap in the <noframes> section or add a link to the sitemap file.

For a non-frame site where the links are handled through JavaScript:

Use a sitemap.htm file to help spiders reach the pages you want. This file must be included in the user-visible site navigation.

www.bricksandbrass.co.uk

[This message was edited by Jen Worden on March 15, 2002 at 17:29.]

[Jen Worden]

I'd say you about hit it on the head, Simon! [img]/infopop/emoticons/icon_smile.gif[/img]

cfn ... Jen

Jen Worden
Web Developer
www.meadoworks.com