Another perplexing question - i have just confirmed the out of the box setup for sitemap appears to be that when a page is set to 'do not display in menu', it does not appear in the sitemap either. Is this correct? Why would the two correlate? Obviously we would like the page to be spidered, just not navigable by users.
Cheers
You're right Mitchell, from a SEO perspective it is most often bad practice to let content expire (thus breaking links to it).
Page expiry should be reserved for content that is MEANT to be taken offline at a certain date and never accessed again , nothing else.
Also, because EPiServer does not leave a redirect code when content is moved (out of the box at least - there are several plugins for that), moving content to a new location/URL (e.g. archiving) is equally bad for SEO.
For SEO-friendly archiving, you should look into adding an event handler when a new page is created, and automatically move it to its correct location in a page structure based on e.g. years and months. That way you won't need expiry and archiving, and won't lose your link juice. Or use a plugin that maps the old and new page locations and gives the proper redirect code.
For the sitemap question (I assume you mean the built-in EPiServer:Pagetree control), there is an attribute named "EnableVisibleInMenu" that you can use to determine whether the pagetree honors the VisibleInMenu property.
If you need more granular control over which pages are shown, consider adding a custom filter: http://world.episerver.com/Modules/Forum/Pages/Thread.aspx?id=45843
To answer your first question, there are many use cases to expire content. Say that you have created a campaign that should be active for a week. After it has expired, you don't want to have users or search engines finding this content any more since it's no longer valid.
The site map and templates are just samples on how to display content on your web site. I'm no search engine expert, but a web crawler should find content both on the site map but also pages that have links from any page on the site so I see no need to worry about content not showing up on the site map.
Adding some thoughts:
Page expiration is mostly used (in my experience) when you have a teaser/block functionality built with pages (pre EPi 7). So you're typically scheduling expiry of a teaser for an event or something similiar that's part of a page and not something that has it's own URL.
A sitemap page that uses EPiServer:Pagetree control is meant to help a human user. To help a spider you are better off serving a XML sitemap.
That makes things much clearer thanks very much. It is a bit perplexing as an enterprise CMS not to think in terms of SEO and require a plugin to redirect URLS, ill defeitley look for an event handler that can do these redirects for me. Many thanks!
I have a question regarding the page expiry function, and sometimes used archive function.
Why would I use this? For SEO purposes, this seems very unfriendly. Am I correct in saying that if i select a page to 'expire' on a certain date, and leave it there (ie: do not archive to a diffeent location), this page is no longer spidered by google from that selected date, given it no longer appears in the sitemap? This is essentially now lost content, with any links to it now broken.
Subsequentally, it seems organisationally sensible to move to an archive, however this now breaks the URL (bad seo again), and all the links in it. Im at a loss? Why wouldnt you just hide from menu, but keep google spidering all that content youve worked hard on creating.
Cheers,
Mitch