Hi Sandeep,

You can add rewrite rules

  <!-- Rewrite robots file requests for sites -->
        <rule name="Robots" enabled="true" stopProcessing="true" patternSyntax="Wildcard" xdt:Transform="Insert">
          <match url="robots.txt" />
          <conditions>
            <add input="{HTTP_HOST}" pattern="www.*.*" ignoreCase="true" />
          </conditions>
          <action type="Rewrite" url="/Resources/Robots/robots.{C:1}.txt" />
        </rule>

You will have

/resources/Robots/robots.abc.txt

/resources/Robots/robots.def.txt

so if user comes to abc.com/robots.txt it will be redirected to abc.com/resources/robots/robots.abc.com

this way it will resolve your physical file issue.

Cheers

Ali, Murtaza

For a robots.txt file there is an option to use the plug-in POSSIBLE.RobotsTxtHandler to allow this to be configured dynamically for each site in your solution, rather than using static files.  This will present the site administrators with access to a text input field to manage the content of the file.  

If the administrators of your sites have requirements beyond a simple text entry field then you would need to develop a bespoke handler, but in most circumstances the POSSIBLE.RobotsTxtHandler plug-in will suffice.

<p>Hi Murtaza</p> <p>Thanks for the reply.&nbsp;</p> <p>Will the crawlers be able to read the file if there is a redirect in place. I am not sure. Also we need to place some other config and feed files in the root which are read by third party apps. We have seen issues with automated programs reading the file if there are redirects in place.</p> <p>Is there any other option ?</p> <p>We can actually write code to dynamically show output via a controler but would want to avoid that as it would create a lot of work for something very simple.</p> <p>Kind Regards</p> <p>Sandeep</p> <p></p>

You don't need a physical file at all. You can ccreate a controller and map it to whatever URL you want. Then you can put logic inside the controller/action to render different contentent depending on the current domain/website.

<p>Thanks Johan</p> <p>I had just mentioned the same in my comment above. I was hoping for some asy cheeat which would save some dev time.</p> <p>Kind Regards</p> <p>Sandeep</p>

Hey Sandeep

By the sounds of it POSSIBLE.RobotsTxtHandler will do exactly what you need (as Darren has described above). Just install it and go, no dev time needed apart from the install itself.

https://nuget.episerver.com/package/?id=POSSIBLE.RobotsTxtHandler

David