Developer forum

Forum » CMS - Standard features » Duplicate users in Index with Load Balancing

Duplicate users in Index with Load Balancing

Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

Hi,

 

This is weird one. A particular index is generating duplicate documents every now and again. We can't consistently reproduce it, we just know it happens and that rebuilding the indexes manually fixes it.

 

The particularities of this project are (and may not mean anything):


The customer ends up having to manually rebuild the index, and because there's no explicit error, I don't have any logs to look into. 

 

Can anyone think of a way I can get more info? This has been going on for months as we've been trying different approaches to reproduce the problem consistently.

 

Best Regards,

Nuno Aguiar

DuplicateUsersInIndex.jpg

Replies

 
Nicolai Pedersen Dynamicweb Employee
Nicolai Pedersen
Reply

Hi Nuno

The errors on already running tasks is a very clear indication that the taskrunner/scheduler run from more than one place.

Another thing it can be that files /System/Diagnostics/* is somehow copied between nlb instances - that could also cause that error to happen and cause the same data to be collected.

When looking at the UserIndexBuilder the users are quired only once, so it cannot do duplicates unless it runs more than once at the same time...

Do you have any update or updatewithids builds defined?

Please post your IndexBuilderExtender and NoticationSubscriber - you never have to rule out anything, so let's see that code as well...

I will try to add to the LogInformation method of the tracker (the thing that writes to all the files) a thread id of the executing thread. Then if you have various IDs something is wrong in your setup.

Go Sherlock!

BR Nicolai

 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

Hi Nicolai,

 

Sherlock here... took me a while to get all the answers because I don't have access to the webserver:

  • The errors on already running tasks is a very clear indication that the taskrunner/scheduler run from more than one place.
    Ok, could this be the cause https://www.screencast.com/t/hqxnFQ5opIpe ? So if an index would be rebuilding (taking more than 5min to finish) and the scheduled task was triggered again?

     
  • Another thing it can be that files /System/Diagnostics/* is somehow copied between nlb instances - that could also cause that error to happen and cause the same data to be collected.
    I was provided the robocopy script and both the Diagnostics and the Indexes folders are being excluded (among others)

     
  • When looking at the UserIndexBuilder the users are quired only once, so it cannot do duplicates unless it runs more than once at the same time...
    Do you have any update or updatewithids builds defined?

    Yes. Could a partial rebuild while a full rebuild is running cause this? https://www.screencast.com/t/A5goklv07

     
  • Please post your IndexBuilderExtender and NoticationSubscriber - you never have to rule out anything, so let's see that code as well...
    Sure. I am attaching the full projects. There is a ton of stuff there since they do a lot, but this happens ONLY with Users.

    (This was started to be built 2 years ago back in 9.7.x. The Users repository may provide some of the document properties already by default but we've been experiencing this for a while, even while we were on 9.8. We're on 9.9.7 currently and they don't want to easily update)

     
  • I will try to add to the LogInformation method of the tracker (the thing that writes to all the files) a thread id of the executing thread. Then if you have various IDs something is wrong in your setup.
    That would be awesome. They are very reluctant to update due to their corporate policies. What package would this be? I will try to see if we can simply update 1 dll (and/or a couple of dependencies)

 

Let me know what you think,

Nuno Aguiar

 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

Hi Nicolai,

 

Did you ever manage to see this? Could you get us a package or a pirate version of the dll with additional log? 

 

The website is in 9.9.7 and the customer is reluctant to update (too many internal processes), so if we could simply update a dll would be awesome.

 

Best Regards,

Nuno Aguiar

 

You must be logged in to post in the forum