Custom Product Index Builder with too many internal references

Nuno Aguiar

Posted on 19/04/2022 17:09:06

Hi,

TL; DR;

I need some help creating a custom Product Index Builder using Dynamicweb's as the foundation, so we can exclude Order related data, but after having looked at DW's source code, there are a lot of internal or private methods and properties from Ecommerce that prevent us from doing so.

Is there a way around that?

What we did?

We simply got ProductIndexBuilder.cs, set the Nuget Packages and tried to build it.

Internal or Private methods and properties

These are only some of the results:

LoadPrimaryDetailImages
LoadImagePatternManagers
ProductIndexSchemaExtender
Status.FailReason
Status.FailException
Status.FailExceptionMessage
Status.FailExceptionStackTrace
ProductCategoryFielValues
ProductIndexSchemaExtender.FieldSeparator
ProductIndexSchemaExtender.GetProductPriceFieldBaseSystemName
ProductIndexSchemaExtender.GetComparablePrices
PriceHelper
ImagePatternManager
ProductField.ProductCustomFieldPrefix
Services.ProductRelated.Extract
ProductCategoryFieldValues.PrepareProductValues
ProductCategoryFieldValues.GetProductFieldValues

Why do we need a custom one?

We have projects with thousands (sometimes millions) of orderlines and orders.
These projects don't need the "Order Count" and "Order Growth" (nor any order related data) that is currently indexed
These indexes are taking a lot of minutes to index
There are no settings to exclude the order related data
We know the indexes take a considerable amount of CPU usage and/or can leave the frontend hostage, so they need to run them in the middle of the night
The scheduled task frequency (i.e. every 1 day), consideres the completed datetime, not the start datetime
so this means that in time, the index will start running in the middle of the day, and taking multiple minutes, and impacting the frontend becomes an issue

Feature Requests / Related threads

Just FYI, we've been posting some feature requests about this:

Best Regards,

Nuno Aguiar

Replies

Nuno Aguiar

Posted on 20/04/2022 21:10:19

Hope this helps show how critical this is for us.

The website shop.dasinc.com has more than 500.000 records in EcomOrders in the last 12 months
This results in records in EcomOrderlines
They have 166K products, but it takes about 20min to index (x2 instances)
Because the rebuild scheduled task considered the completed time, this means we set it to start at 2am, but the next day it starts at 2:45, then 3:30 so on until it's rebuilding during "office hours"

This is starting to hurt some of our customers and/or forcing us to do some workarounds (updating Status.xml to trick the rebuild task), but I can't get around the orders count.

Any help would be appreciated, while waiting for the related forum threads to be addressed.

Best Regards,

Nuno Aguiar

Posted on 27/04/2022 21:42:56

bump

Nicolai Pedersen

Posted on 28/04/2022 09:12:01

This post has been marked as an answer

Made this feature:

https://dev.azure.com/dynamicwebsoftware/Dynamicweb/_workitems/edit/7544

Looks like this:

Votes for this answer: 1

Nuno Aguiar

Posted on 28/04/2022 09:17:38

Hi Nicolai,

That looks perfect, thank you. Can't wait to try it.

Nuno

Nuno Aguiar

Posted on 28/04/2022 15:07:28

Hi Nicolai,

I looked into DevOps and I see this assigned to 9.14. If this is being released in August, could it then be added to a 9.13 version? We could really use this as soon as possible.

Best Regards,

Nuno Aguiar

Posted on 03/05/2022 21:56:46

bump

Oleg Rodionov

Posted on 05/05/2022 04:18:49

Hi Nuno,

The feature has already been merged to 9.13(admin+ecom 9.13.47), it will be provided in upcoming DW9.13.7

BR, Oleg QA

You must be logged in to post in the forum

Developer forum

Custom Product Index Builder with too many internal references

Replies