Developer forum

Forum » PIM » Create a partial index

Create a partial index

Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi guys,

Can I create a Product index that would include only the products added/edited in the last 7 days?

And by "create" I mean to configure the build in a way that would only include the last 7 days. Is it enough if I set the HoursToUpdate to 168?

My purpose is to keep a working index as small as possible with the purpose of fast updating and searching in the index. The index will be used in Shared Queries in PIM

Thank you,
Adrian


Replies

 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

Hi Adrian,

 

After a full re-index, won't every partial index only capture the last few minutes? And won't your full re-index happen every day or every week, in which case, this wouldn't make sense?

 

Best Regards,

Nuno

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi Nuno,

In my case, I have a very large product catalog (+300k). The users will work in some queries just with the recent products. It does not make sense to load the entire product catalog because that would slow down the performance.

Of course, I can filter in the query the list of products but still, if I can avoid carrying over a large index in the queries, I would surely do it.

Does it make sense? I am open to any other suggestions that can improve the performance for the back-end users.

Thank you,
Adrian

 

 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

Hi Adrian,

 

I'm still missing something because in my mind, a partial index looking for products edited more than 7 days ago suggests you don't rebuild your index for at least 1 week, which surely leads to some issues.

 

In any case, can't you create a Query with an expression where the edited date is less than 7 days, if you're wanting content editors to focus on the most recent edited products?

 

BR

Nuno

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi Nuno,

I understand why this is confusing since it's not a usual use case.

My purpose is to keep a very small index, containing ONLY recent products. 

I already have a separate index where I get all products.

There are some updating/importing activities that would require an instant (or almost instant) update of the index. Which seems to be troublesome even with a partial build.

Again, I am not sure if this is the right direction to explore but I am trying to cover all bases.

Thank you,
Adrian

 

 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

Hi Adrian,

 

I understand now, you want a new Index, and I kept thinking of a partial index builder.

 

I don't know if you can do that with configuration only, but shouldn't be too hard to develop either a new builder that takes a new build setting for that.

 

Best Regards,

Nuno Aguiar

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi Nuno,

Thank you for the suggestion.
I will keep it in mind. I am still hoping there's a ninja trick waiting to surface :)

Adrian

 
Nicolai Pedersen
Reply

If you create an index with only "Update" as builder action and set the hours to update to 168, you should get products updated (not created) in the past 7 days only

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi Nicolai,

Thank you very much for the clarification. It makes total sense.

When the product is created, does it not set the ProductUpdated to the same value as ProductCreated?

Thank you,

Adrian

 
Nicolai Pedersen
Reply

Yes, if done from the backend. If it is integration, it might differ.

 
Adrian Ursu Dynamicweb Employee
Adrian Ursu
Reply

Hi Nicolai,

It's a data integration import. if needed, I can provide a timestamp when importing.

Important thing is that this Update approach would solve my need for now. Not sure if it's the right thing to do, but I will give it a try.

Thank you,

Adrian

 

You must be logged in to post in the forum