London Dev Meetup Rescheduled! Due to unavoidable reasons, the event has been moved to 21st May. Speakers remain the same—any changes will be communicated. Seats are limited—register here to secure your spot!

AI OnAI Off

Robots.txt for different sites

Vote:
 

Hi All,

I have a EPi server web application. The same web application is hosted under 3 domains.  Now my requirement is to have diffrenet Robots.txt for different domains. But since all the domains are pointing to sane folder I can have only file placed in that folder. Is there any workaround for placing seperate Robots.txt file under each domain.

 

Thanks in Advance.

#33163
Oct 03, 2009 11:03
Vote:
 

You can create 3 files:

robotsDomain1.txt 

robotsDomain2.txt 

robotsDomain3.txt  

and then define your own handler, assign it to robots.txt in web.config

<httpHandlers>
<add verb="*" path="robots.txt" type="Project.RobotsHandler"/>
</httpHandlers>

and inside the handler check Request.Url.DnsSafeHost and return the content of one of the files.  If you use IIS7 you define handlers in section

<system.webServer>
<handlers>
<add name="robots" path="robots.txt" verb="*" type="Project.RobotsHandler, Project" />
</handlers>
</system.webServer>
 

#33165
Oct 03, 2009 11:14
Vote:
 

Thanks for your reply.

Can you point me to any URL that explains the code to be written in the Handler.

#33166
Oct 03, 2009 11:39
Vote:
 
http://www.kleenecode.net/2007/11/17/dynamic-robotstxt-with-aspnet-20/
#33167
Oct 03, 2009 12:11
Vote:
 

Thanks Maras. Below link is also usefull

http://www.atlasbay.com/2008/11/06/robotstxt-httphandler-in-iis7/ 

Thanks for your suggestion.

 

#33168
Oct 03, 2009 12:25
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.