Developer forum

Forum » Ecommerce - Standard features » Facet Performance Issue

Facet Performance Issue

António Ramos
Reply

Hi guys,

We are having a performance issue when activating the facet group at a product list. See screncast please:

http://screencast.com/t/q2hC6Lqy

It takes around more 3 seconds that it should.

12:58:41 PM.838 <-- eCom: Get products from Query --> 0 -4000
12:58:45 PM.135 Index query executed - took 3274.9094 ms:
+Active:True +LanguageID:en-us +ShopIDs:SHOP1
3297 -703

Best regards,

António Ramos

 


Replies

 
Nicolai Høeg Pedersen
Reply

Hi Antonio

You have created facets for Brand, Price and Sub category. Then you run the facet calculation against your entire product catalog, meaning you have no other search parameters.

You have 5900 different brands, 30500 products that gives you the same number of prices, and some hundreds sub groups.

So when you execute the calculation on that, you get 5900 facet options of brands, 30000+ on prices and a couple of hundreds on your sub groups. That will always be slow and will never render in a timely manner, and no user will be able to find the right option anyway...

So - you should rethink this. If the first choice is a product category - you would limit the entire result drasticly and not get into trying to calculate that many facet options.

Facets are not meant to group by price or by any field with thousands of options - no user can use that in anyway. It makes no sense and it is slow...

If you need 5900 brands in one screen, you should load them asyncronously from a seperate feed and best if the user starts to search for a specific brand.

The underlying Lucene supports 2048 options for one facet - which is too many already. Dynamicweb handles it if you get above that number, but normally you would get an exception.

BR Nicolai

 
Nuno Aguiar
Reply

Hi Nicolai,

 

Thanks for tracking this down. This project in particular is an upgrade from an existing website (not DW), new design and "the whole shebang".

 

The facets are the same as the existing website and we cannot suggest other ones. For scenarios like this I see 2 options:

  1. Somehow be able to get the Top X facets options (if that would improve performance)
  2. Lazy load the facets

 

Do you know if option 1. is possible or could be accepted as a feature request? If not we should probably go with lazy loading the facets to get the entire options

 

Best Regards,

Nuno Aguiar

 
Nicolai Høeg Pedersen
Reply
This post has been marked as an answer

Hi Nuno

I am aware of this issue. We have been discussing a solution for this - a new type of facet that can be used for fields where you have zillion of destinct records.

I would like to see their current solution - what it is. Are they showing a list of 5900 manufacturers on their page? I'm sure they do not treat it as facets - that would give you 5900x30.000 = 177.000.000 combinations that Lucene needs to calculate. That is a lot - and if you add a 3rd facet group of say 100 options, it will be 17.700.000.000 calculations at least. That is also a lot :-).

Prices can NEVER be facets - not like this. You can group them when you index them and use a facet. Otherwise it is a range query which is a different thing showing the highest and lowest of this search. Making it a facet is the same as saying SELECT DESTINCT Price FROM PRODUCT WHERE .... That will not give you anything useful - just a list of all the prices and how many that cost i.e. 299.

Brands I better understand - but not with that many.

We have to make sure in this discussion that we do not confuse filters with facets. Take a look at Zalando - big shop, big website, search is what they live from. They have a brand filter (you cannot see how many of each brand in the current search). There is a BIG difference calculation wise - to see what is present and what is present combined with how many in the given search context.

You can easily create that lists of filtervalues by creating the search on a seperate page where you return the values from a specific field - and you can also add an additional criteria to search i.e. the brand field so you can do AJAX filtering in the search page.

I know it is not super simple - but it is important to understand what facets are and what they do.

We will implement a new type of facet - "Term facet" which will give you a list of values for a given field in the current search (but without the count - that is the expensive one). Then you can render the entire list (5900 brands in your case) or page them, filter them and other great stuff.

BR Nicolai

Capture.PNG
Votes for this answer: 1
 
Nuno Aguiar
Reply

Hi Nicolai,

 

I took my time to get back to you, because I wanted it to soak in on me. This is enlightening and I see your point.

 

We changed a few  things and there was a huge impact:

  • Created a non-analyzed field for the Manufacturer (Brands)
  • Created a non-analyzed field for the Groups Names (we need them as facets - listing by parent groups)
  • Created a group field for the price

 

So I am looking forward for the new "Term facet" and I am sure we'll use it a lot. Hope there's a way to switch between the two. I can see clients wanting a demo on how slower it gets if we activate results per facet option.

 

About facets vs filters part of the discussion, I honestly did not see them as two seperate things, and If I understood you correctly there is a thin line. Filters are options that have no calculations, just the raw/distinct information from the DB/index, whereas Facets have calculations associated with them.
If this is true, the new "Term facet" will basically be a Filter - a somewhat inexpensive facet we can set in the UI, is that accurate?

 

Best Regards,

Nuno Aguiar

 
Nicolai Høeg Pedersen
Reply

Yes, there is a thin line. And it is kind of geeky, agreed.

Term Facet will be a filter - and can make you a list of 6000 brands or 100.000 composers based on a search and do it fast. But it will of course make the browser crash if you render all of them :-).

BR Nicolai

 
Nuno Aguiar
Reply

Hi Nicolai,

 

Of course, so as long as we can manage a limit that would relate to the Top X in a query statement, we can live with that. I doubt I'll ever get rid of 6000 brands and 100.000 composers, but as long as I can limit it, without a lot of template "ninja tricks" (like you call them smiley) we can do it easilly.

 

Looking forward for that new facet.

 

Best Regards,

Nuno Aguiar

 
Nuno Aguiar
Reply

Hi Nicolai,

 

I saw you demo the "Facet Term" on DWTC16. I was wondering when will it be available. I have a project coming up where that would be in handy. I checked and it is not available in 8.8.1.5 yet.

 

Best Regards,

Nuno Aguiar

 
Nicolai Høeg Pedersen
Reply

It is currently on the 8.9 branch only.

It can be merged to 8.8.1...

 
Nuno Aguiar
Reply

Hi Nicolai,

 

I would appreciate it very much :)

 

Nuno Aguiar

 

You must be logged in to post in the forum