November Happy Hour will be moved to Thursday December 5th.

Eric
Apr 27, 2015
  10247
(4 votes)

Simple robots.txt built with asp.net mvc

The other day there was a question about serving static files from episerver, like a robots.txt. Since most editor like to be able to update the robots.txt them self many of us has build solutions for that. There are also some packages in the nuget feed from EPiServer that will help you do just that.

But if you are like me, I do not like to have pre-compiled solutions from other companies in my project, some are better then others but still I like to be able to change most of my code in a project. Of course you can download the source code in many of those packages but some times they tend to do a bit more then you might expected as well.

 

In this case I just would like to serve a robots.txt if it is requested by a search engine for instance.

Solution

First I create the robots.txt controller. Since we are working with EPiServer CMS I have added a textarea on the startpage type and therefore editors are able to change the content of the file.

 

   1:      public class RobotsTxtController : Controller
   2:      {
   3:          private Injected<IContentLoader> ContentLoader { get; set; }
   4:   
   5:          [ContentOutputCache]
   6:          public ActionResult Index()
   7:          {
   8:              var startPage = ContentLoader.Service.Get<StartPage>(ContentReference.StartPage);
   9:   
  10:              string content = startPage.RobotsTxtContent;
  11:              return Content(content, "text/plain");
  12:          }
  13:      }

 

Nesx step is to create an initialization module were we map a route to our robots.txt file.

 

   1:      [InitializableModule]
   2:      [ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
   3:      public class RenderingInitialization : IInitializableModule
   4:      {
   5:          public void Initialize(InitializationEngine context)
   6:          {
   7:       
   8:              //Maps route for robots.txt
   9:              RouteTable.Routes.MapRoute("RobotsTxtRoute", "robots.txt", new { controller = "RobotsTxt", action = "Index" });
  10:          }
  11:   
  12:   
  13:          public void Preload(string[] parameters) { }
  14:   
  15:          public void Uninitialize(InitializationEngine context)
  16:          {
  17:              //Add uninitialization logic
  18:   
  19:          }
  20:      }

 

And finally we add the property to our startpage model

        [UIHint(UIHint.Textarea)]
        [Display(Name = "Robots.txt", Order = 90, GroupName = Global.GroupNames.SiteSettings)]
        public virtual string RobotsTxtContent { get; set; }
Apr 27, 2015

Comments

Marija Jemuovic
Marija Jemuovic Apr 27, 2015 09:57 AM

Nice idea!

Eric
Eric Apr 27, 2015 04:26 PM

Thanks.

valdis
valdis Apr 28, 2015 12:30 PM

No offense, just wanted to know reason using Injected..? :)

Vincent
Vincent Apr 29, 2015 06:57 AM

To Valdis

Is it used for querying the pagedata from startpage reference?

Feb 14, 2017 06:01 PM

Thanks Eric, nice & simple => a perfect solution.

Eric
Eric Feb 15, 2017 09:02 AM

Happy you like it :) 

Please login to comment.
Latest blogs
Optimizely SaaS CMS + Coveo Search Page

Short on time but need a listing feature with filters, pagination, and sorting? Create a fully functional Coveo-powered search page driven by data...

Damian Smutek | Nov 21, 2024 | Syndicated blog

Optimizely SaaS CMS DAM Picker (Interim)

Simplify your Optimizely SaaS CMS workflow with the Interim DAM Picker Chrome extension. Seamlessly integrate your DAM system, streamlining asset...

Andy Blyth | Nov 21, 2024 | Syndicated blog

Optimizely CMS Roadmap

Explore Optimizely CMS's latest roadmap, packed with developer-focused updates. From SaaS speed to Visual Builder enhancements, developer tooling...

Andy Blyth | Nov 21, 2024 | Syndicated blog

Set Default Culture in Optimizely CMS 12

Take control over culture-specific operations like date and time formatting.

Tomas Hensrud Gulla | Nov 15, 2024 | Syndicated blog

I'm running Optimizely CMS on .NET 9!

It works 🎉

Tomas Hensrud Gulla | Nov 12, 2024 | Syndicated blog

Recraft's image generation with AI-Assistant for Optimizely

Recraft V3 model is outperforming all other models in the image generation space and we are happy to share: Recraft's new model is now available fo...

Luc Gosso (MVP) | Nov 8, 2024 | Syndicated blog