Welcome to TalkGraphics.com
Results 1 to 8 of 8
  1. #1

    Default robots.txt for subdomain

    I have a .com domain with one subdomain. I want the .com domain to be fully indexed by the search engines but none of the content on the subdomain to be indexed.

    Does anyone know what the robots.txt file should look like to achieve this?

    - the .com domain uses www[.]domain[.]com whereas the subdomain is subdomain.domain.com ...if that makes any difference

    Thank you for any advice

  2. #2
    Join Date
    Aug 2000
    Location
    Placitas, New Mexico, USA
    Posts
    41,646

    Default Re: robots.txt for subdomain

    Megg - I am not certain, but I think when you fill in the URL for the site you are publishing, the robots.txt is created for that domain.

    There is one sure way to find out. Publish to this sub-domain, then check the published files for the robots.txt file.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	publish.png 
Views:	220 
Size:	31.2 KB 
ID:	96405  

  3. #3
    Join Date
    Apr 2012
    Location
    SW England
    Posts
    18,378

    Default Re: robots.txt for subdomain

    Megg, I think you need to add <meta name="robots" content="nofollow" /> in the sub-domain's Website HTML Code (head) textbox if you are using a Xara product.

    Search Google for "nofollow"; you include it in the HEAD tag of each page of your sub-domain.

    Acorn

  4. #4
    Join Date
    Aug 2000
    Location
    Placitas, New Mexico, USA
    Posts
    41,646

    Default Re: robots.txt for subdomain

    @Acorn - Does this mean that Megg's site will or will not be indexed?

  5. #5

    Default Re: robots.txt for subdomain

    sorry, I should have said - the .com domain will have a xara site published to it, the subdomain is a blog only - so no code to edit on the page, its a redirect from another address to the subdomain.

    I've always created my robots.txt files manually after site publish - but not too sure how to tackle not indexing the subdomain only without blocking the whole site from the search engines

  6. #6
    Join Date
    Aug 2000
    Location
    Placitas, New Mexico, USA
    Posts
    41,646

    Default Re: robots.txt for subdomain

    OK that clears things up. Somewhat.

    Then you could add the code that Acorn provided to the pages that you do not want indexed.

  7. #7

    Default Re: robots.txt for subdomain

    Thank you for your advice. I have found that the blog has a section for custom robots.txt so I dont have to upload it to the server separately. I have inserted the following:
    User-agent: *
    Disallow: /
    As I want the entire blog not to be indexed - so hopefully this will work.

  8. #8
    Join Date
    Jun 2012
    Location
    Brooklyn, NYC
    Posts
    46

    Default Re: robots.txt for subdomain

    Check the index status in google webmaster tool,also verify there any alert related to this robots.txt

 

 

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •