Developer forum

Forum » CMS - Standard features » Adjust Robots meta for secondary pages of product list

Adjust Robots meta for secondary pages of product list

Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi guys,

I have received a weir request from one of my customers. Apparently, their SEO specialists have recommended that we should adjust the robots meta value to "noindex, follow" for all pages containing PageNum=*.

I see that I can create a custom tag in meta with these values but the old one is still visible. Is there any way I can control the value of the current one? Or replace it with my custom meta?

Thank you,
Adrian


Replies

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi guys,

One more observation. FOr product lists with multiple pages, I see that the Canonical meta is not rendered for the first page but it is rendered when PageNum=*. I understand why this is happening but in my case, I need to render the Canonical URL as an absolute URL instead of a relative URL. For the main page, I am adding a custom Canonical tag containing the absolute URL.

Is there any way I can force DW to render the Canonical URL absolute? Or avoid rendering it in the page and keep my custom Canonical tag?

Thank you,
Adrian

 
Nicolai Pedersen
Reply

You can just place a robots.txt on the server, and DW will stop serving the autogenerated one.

Your templates are customizable so go ahead and make your own rules for canonicals. Just load our version in a variable and change it as needed using URI object from .NET.

BR Nicolai

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi Nicolai,

Thank you for the clarification.

Adrian

 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

Hi Nicolai,

 

Using this thread, since I had a similar one here https://doc.dynamicweb.com/forum?ThreadID=71315&PID=1605

 

Would it not be expectable to have the canonical for product lists? We have it everywhere else (Product page and content pages) that I am surprised that for products lists we'd have to develop it, specially when there are navigation providers that generate the URL.

 

Best Regards,

Nuno Aguiar

 
Nicolai Pedersen
Reply

Well, I made the mistake of reading the manual made by Google. And understanding what canonicals is and when to use them. That was a stupid mistake, as I should instead have made 1000 misunderstandings and just added random URLs to the canonical tags as most people care about that 'something' is there, and not that 'the correct information' is there....

You would not need canonicals on product list pages unless you list the same content over and over again on different URLs. I know I can never win that discussion... Page 1 one of a list of t-shirts has X products - page 2 has X other products. Different content, different URL, no need for canonicals - on the contrary.

Some argue that when you go to page 2, like this: /products/components?PageNum=2, you need a canonical to page one. Why, I will never understand since it is a different URL showing different content... But ANYWAYS, you can set the checkbox to use the first page URL as canonical when on page 2 - see dumps 1 & 2.

If the URL in the browser is /products/components and the calculated canonical is /products/components - it is the same value and the canonical tag has absolutely no meaning... But if you go to page /other-products-list-page/components the calculated canonical for the group /components is /products/components if, and only if, you have added a primary page on the components group. OTHERWISE, Dynamicweb will not be able to understand which URL is the correct one - it could be any page with the product catalog. So you need to have primary or unique group path and each group needs to get specified what page is its primary page to be shown on in order to calculate a precise and correct canonical that is only pointing to exactly one location. See dump 3+4.

Canonicals could be added on all pages no matter what - but for what reason...? And what should the value be? What is the problem to solve...?

Remember the entire purpose of canonicals is not to have something in the meta tags that has the name canonical - it is to tell Google your preferred URL for content (exactly the same content) that has 2 or more URLs. So if Dynamicweb does not have that info - the ONE and UNIQUE path to a piece of information, it cannot be calculated and cannot be inserted without being faulty which you do not want.... Or maybe you want, but it would be a mistake.

Feel free to tell how you think they should work. Maybe I don't understand what canonicals are...

 

Capture.JPG Capture2.JPG Capture3.JPG Capture4.JPG
 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

Hi Nicolai,

 

I appreciate the lengthy explanation :)

 

I don't think you're wrong, but I don't have the same knowledge you do on this (I am sure it's because you just spent way more time on this than I have - you're no smarter than me :P), so I did not have a lot the details top of mind to get that configuration.

 

These "discussions" are annoying for me too. Your dumps are very helpful at least to provide closure on some topics/frustrations, regardless of their effectiveness.

 

In a way, this could be a series of "How to" articles. i.e. "How to configure Canonicals" with a step by step guide, since this comes up every now and again, and there are a lot of detaisl that are documented in a different way. Just a thought.

 

Thank you,

Nuno Aguiar

 
Nicolai Pedersen
Reply

So anything missing from this page?: https://doc.dynamicweb.com/documentation-9/how-tos/general/canonical-urls-for-products

 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

For product lists, just like you explained :)

 

For example, it does not address the "Use first page as canonical" checkbox, and when I was going through the "SEO expert" report and they highlighted the products lists, that completely missed me.

   
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

Thank you

 

You must be logged in to post in the forum