Developer forum

Hi Nicolai,

Thank you, this is useful information.

In my case though, I have several indexes (3 to be precise) connected to a set of PIM Queries.

Each PIM query is connected (more or less) to a WorkflowStateID (in some cases the rules are a bit more complex). Using a regular index with a small set of products, the migration of products from one list to another is instant. With a large set of products (even with ProductCategoryfields excluded) the migration from one list to another is not instant anymore and it creates confusion. And I want to optimize the size of the index and only include what's relevant for each set of lists. Maybe I am trying the wrong path to solve it ?

Does it make sense?

Thank you,
Adrian

Can't you add a custom field to the products table and then filter based on that?

Hi Andrew,

I want to filter the products from the Index, not from the query.

Basically, I need to create 2 separate indexes. My product catalog has around 400K products, with a lot of properties. I need to separate this catalog into 2, based on the product source, and optimize the size (and update time) of each index.

Separation by shop is good, but in my case, they are all added to the same shop and I cannot separate them.

I hope this makes sense.

Thank you,
Adrian

You can also filter indexes by shop/channel - so if you divide by channel or groups in dfiferent channels, you can exclude them in that way.

Hi Nicolai,

I know I can use shops to segment the index, and I have used that in the past. In my case, all products are in the same shop and it will be a huge complication to separate them.
By separating the products based on another property. I would end up with 2 separate indexes, each one being considerably more performant than the existing one.

To put it in context, I have products coming from import and following a documentation flow with dedicated PIM queries and products coming from a Marketplace solution (integration) that only require approval. The marketplace products have separate PIM queries for the approval process.

Marketplace products are 60-70% of the total product catalog and it makes no sense to complicate the life of the product editors by keeping the Marketplace products in the same index.

We will probably end up creating another Index Builder with an additional Parameter.

Thank you,

Adrian

Could you *add* them to an additional channel / shop based on SQL logic? Then they could stay in the main shop together, but also exist in separate channels that you could use for the index logic.

Imar

Hi Imar,

I could, definitely. But I am not sure it will work.

If I add all products to SHOP1 and only the Marketplace products to SHOP2, I can surely create a separate index for SHOP2 but the index for SHOP1 will still include both.

I would have to add regular products to SHOP1 and SHOP3, Marketplace products to SHOP1 and SHOP2, create a general index for SHOP1 (all products), an index for SHOP2 (Marketplace) and an index for regular products (SHOP3).

This way it could work but I may generate other issues with these allocations.

All products have the same value in DefaultShopId. I am not sure what will happen if I remove that value on all products and use virtual Channels to separate products.

Adrian

Yeah, I can see how that could lead to problems.

Then maybe a custom index builder is the way to go. You could fire a notification from GetProductsQuery or so to fetch an extra where clause from a subscriber that is then append to the main query. That way, your builder is almost the same as the one in the box and your custom logic is spearated out in a subscriber, making maintenance easier.

Hi Imar,

That's actually very clever, thank you.

We'll give it a try.

Adrian