November Happy Hour will be moved to Thursday December 5th.

Geta SEO Sitemaps for hidden pages (behind login)

Vote:
 

Hello

We are using an external tool called Siteimprove, who are using sitemaps to crawl the webpage for spelling errors and etc.
Recently we have released a self-service environment, where there are pages behind account authorization, that we wish to crawl. In order to do that, we have to add them to a sitemap.
 
Using Geta Seo Sitemaps (version 4.0.0), has anyone any experience with adding pages behind a login to a sitemap, which aren't public for search engines?

#292905
Dec 09, 2022 9:52
Vote:
 

If you can override this (https://github.com/Geta/SEO.Sitemaps/blob/eebf3007c5004a7351ca5ce59eb6de95c9ff0252/src/Geta.SEO.Sitemaps/Utils/ContentFilter.cs#L18) implementation, then you can control filter yourself and decide whether you want to include or exclude protected pages.

#294988
Jan 19, 2023 17:42
Vote:
 

Even if you add the URLs to sitemap.xml, how are Google supposed to crawl the pages if they are behind account authorization?

#295004
Jan 19, 2023 20:57
valdis - Jan 19, 2023 21:29
Maybe Siteimprove is capable to "login" and check page state behind the "firewall"?! Don't know the tool..
Tomas Hensrud Gulla - Jan 19, 2023 21:33
Valis, you are – as always – correct. The question was about Siteimprove, but I was thinking Google-only.
Looks like Siteimprove can log in, if configured correctly.
Vote:
 

Yes, I've had experience with this. Using Geta SEO Sitemaps of one of our personal project, you can include pages behind a login in your sitemap by creating a separate sitemap for authenticated pages. You would need to ensure this sitemap is only accessible to Siteimprove by providing them with the necessary authentication credentials. Be cautious, though, to avoid exposing sensitive information to public search engines.

Hope this helps!

#326049
Edited, Jul 26, 2024 11:45
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.