Developer forum

Forum » Development » Url to sitemap in robots.txt

Url to sitemap in robots.txt

Anders Ebdrup
Anders Ebdrup
Reply

Hi Dynamicweb,

 

We have a minor issue with the automatic link to the sitemap in the robots.txt, as the link does not take ssl into account. Even if the website is set to force ssl, the sitemap link is still http://www.website.com/sitemap.xml which will will result in a link like this: "https://www.website.com/admin/public/404.aspx?404;http://www.website.com:80/Sitemap.xml"

 

Can this be fixed or am I missing something in the setup?

 

Best regards, Anders


Replies

 
Nicolai Pedersen
Reply
This post has been marked as an answer

Yes, that is a bug.

TFS#28488 - just fixed for the next 9.1.4 and 8.9.0.5

BR Nicolai

Votes for this answer: 1
 
Kristian Kirkholt Dynamicweb Employee
Kristian Kirkholt
Reply

Hi Anders

The problem TFS#28488 "Wrong schema for sitemap.xml in robots.txt using SSL" has now been fixed in version 8.9.1.1 and version 9.1.4

You are able to find this build in the download section:

http://doc.dynamicweb.com/releases-and-downloads/releases

Please contact Dynamicweb Support if you need any additional help regarding this.

Kind Regards
Dynamicweb Support
Kristian Kirkholt

 
Mikkel Toustrup Olsen
Reply

Hi Nicolai,

I recently got a case from one of our customer's SEO bureau, and they pointed out that you are able to access the sitemap.xml on either: www.domain.com/sitemap.xml and www.domain.com/Sitemap.xml . They state, that google's crawler becomes confused if there a "multiple" sitemaps on the solution..

The robots.txt specifys that the sitemap is located on www.domain.com/Sitemap.xml .

I tried to do a rewrite rule on the lowercased url, to rewrite it to the proper Sitemap.xml url, however it doesn't seem to apply as I'm still able to access the lowercased url ..

I tried to trace it on the IIS, and the rule does indeed get hit, though it seems DW or anything in there lowercases the URL afterwards.

Any suggestions on this matter. The solution is running 9.3.10 and I can't seem to find the checkmark from previous DW versions, like: "Use casesensitive urls" . I found a global settings entry regarding this, however it does not seem to apply either: /GLOBALSETTINGS/SYSTEM/URL/CASEINSENSITIVE (true/false)

BR Mikkel

 
Nicolai Pedersen
Reply

Hi Mikkel

The lower case setting is by default set in DW9.

SEO experts say a lot of things - if I get a link to an official Google documentation stating it is a problem I will have it fixed very fast. Otherwise I will consider it for 9.5

Thanks, Nicolai

 

You must be logged in to post in the forum