Developer forum

Forum » CMS - Standard features » An item with the same key has already been added at ProductService.GetByAutoIDs

An item with the same key has already been added at ProductService.GetByAutoIDs

Thomas Larsen
Reply

On version 9.6.7 we periodic get this error on a productlist. 

System.ArgumentException: An item with the same key has already been added. at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource) at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add) at Dynamicweb.Ecommerce.Products.ProductService.GetByAutoIDs(IList`1 autoIDs) in E:\agent\_work\125\s\05 - Modules\Dynamicweb.Ecommerce\Dynamicweb.Ecommerce-1.6\Dynamicweb.Ecommerce\Products\ProductService.vb:line 758 at Dynamicweb.Ecommerce.Frontend.Frontend.GetProductsFromIndexQuery(ProductCatalogShowType settingsType) in E:\agent\_work\125\s\05 - Modules\Dynamicweb.Ecommerce\Dynamicweb.Ecommerce-1.6\Dynamicweb.Ecommerce\Frontend\Frontend.vb:line 599

If I check the index it contais the same product 2 times.

It only happens when the index has been partially updated, but not always.

Rebuilding the full index helps.

Anyone?

BR
Thomas

 


Replies

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi guys,

I just got the same error this morning on a solution that did not report it previously.

The solution runs on DW 9.6.12.

I tried running the index again with the same outcome every time.

This is a pretty critical issue.

Thank you in advance for a prompt reply.

Adrian

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi guys, 

A quick update on this issue. The problem might be caused by an incorrect configuration of the index. I have noticed that the 2 instances that I have configured were pointing to the same folder (Products) instead of separate folders (Products/A and Products/B).

Probably the issue surfaced only when the update of the 2 instances overlapped.

I have set separate folders for each instance, update the index and now it seems ok.

I will update this thread is the issue come back.

Thank you,

Adrian

 
Thomas Larsen
Reply

In my case the two instances points on different folders. 

And also on my solution it's a periodic error.

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hmm,

Strange. The error message seems to be identical.

I will keep an eye on my solution. Maybe my suggestion just fixed it temporarily.

Thank you,

Adrian

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi guys,

Apparently, I have the same problem.

Now, the index seems to run but I get only 2 products indexed out of 24871 products.

And the previous fix is not applicable since the index was generating separate folders anyway.

We are back to square one.

@Dynamicweb: can you please investigate this? What do you need from us in order to replicate it?

Thank you,

Adrian

 
Steffen Kruse Hansen Dynamicweb Employee
Steffen Kruse Hansen
Reply

Hi Thomas and Adrian,

I just have a couple of questions, and maybe a possible fix, but it could be a long shot.

1) Do you experience the problem on a live solution with actual traffic on the frontend?

2) Which balancer are you using on the index, which are causing the errors? ActivePassive?

If you can say yes to both questions, you can maybe try to switch the balancer to LastUpdated. I should not break anything for you, and it should be easy to switch back if it gives any undesired behavior. I can not guarantee that it fixes your problem, but it would be interesting to see if it changed anything.

Best regards,

Steffen

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi Steffen,

Yes. Live solution. Pretty critical :)

Yes. Active PAssive. The default that comes with Rapido.

I have switched it to LastUpdated.

Fingers crossed.
Adrian

 
Thomas Larsen
Reply

Hi Steffen,

Yes and yes.

I can try and change the balancer. But remember the following two things 

It only happens when the index has been partially updated (when some products has been edited in PIM), but not always. Se attached screenshot

- If I check the index it contais the same product 2 times. Meaning that the index is corrupted.

.

Repo_2019-09-30_13-51-14.jpg
 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

I got the exact Image. See attachment.

The fact that we are both experiencing the same issue, make it worth investigating further. Would you agree?

Adrian

Index-chrome_2019-10-03_20-22-26.png
 
Steffen Kruse Hansen Dynamicweb Employee
Steffen Kruse Hansen
Reply

So you still keep experiencing the 'An item with the same key....' exception after switching the analyzer? I thought that maybe somehow the balancer locked some files, and therefore prevented them from being updated as part of a partial update. That could explain why Lucene will then create a new file, but it was a long shot.

I have only seen this problem one time before, and at that point, the problem only occurred on live solutions and was not reproducible on staging or development, so I think the live traffic has something to say in this matter. 

I think it would be a good idea to create a support case for this one because we probably need a copy of the solution.

Best regards,

Steffen

 
Jeppe Eriksson Agger Dynamicweb Employee
Jeppe Eriksson Agger
Reply

Hi all,

Sorry for jumping in, but it seems that different issues are being conflated.

You're seeing "8/8" and "2/2" because that's the number of products that were changed. Partial updated with show you "(indexed products)/(changed products)", so it's expected to see exactly what you're seeing.

I don't have additional information about the exception, but I just wanted to make sure that we focus on the actual issue :)

- Jeppe

 
Thomas Larsen
Reply

That is correct Jeppe, it was just to point out that the problem, in my case, happens when the index was partial updated.

The acual problem is that the Index contais the same product 2 times.
 

/Thomas

 
Terri Donahue
Terri Donahue
Reply

Is there a resolution for this one yet? I have the same issue on a solution running 9.6.11.

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi Guys,

This issue resurfaced for me again on the same solution as before.

It's not happening very often but it is happening and since it's on a live solution, it's pretty critical.

My solution is running DW 9.6.12.

I tried an upgrade to 9.7.x and I got the same result. I can say that the problem was even more persistent than it is now with 9.6.

Thank you,
Adrian

 

 
Nicolai Pedersen
Reply

Hi Adrian

Please try the attached assembly for 9.6 - it has an added check to avoid the exception. It will not fix the issue that indicates that the product is twice in your index. But if it is, it should not go into exception. Could result in wrong paging information on productlists.

9.7+9.8 packages is coming in nuget.

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Thank you very much, Nicolai.
Adrian

 

You must be logged in to post in the forum