Sorting of search results

Justin Sjouw

Posted on 28/06/2022 11:27:06

Hi,

A customer has asked me to look into the sorting of search results.

On this screenshot you can see a product with the exact name that has been searched for is way down the page, ofcourse they expect an exact text match to be the first result. Any idea why the exact match is pushed down the list and instead partial matches are shown on top?

Thanks for any assistance!

Justin

Replies

Nuno Aguiar

Posted on 28/06/2022 11:41:05

This post has been marked as an answer

HI Justin,

That depends on the way you have sorting set up. You can set up multiple criteria for sorting.

I'd say this is the normal expectation

That way it's sorting by relevance/best match first (and if you're not searching for anything, all results have the same value, so that's fine), but then use "NameFortSort" as the 2nd criteria.

Best Regards,

Nuno Aguiar

Votes for this answer: 1

Justin Sjouw

Posted on 28/06/2022 11:47:59

Hey Nuno,

Thanks, I was expecting from previuous experience that this was the default setup. Turns out no sorting seems to be setup in the latest Swift database by default So now I know I need to set that up if I am doing a new project :-)

Thanks,

Justin

Nicolai Pedersen

Posted on 29/06/2022 15:45:06

The default Swift query has this setup on Products.query in Repositories under settings:

The screen you are showing is the local overrides sitting on the app. You do not have to (or want to) override sorting on that screen in this case.

But it can also be the rest of your query. And how you have setup your scoring and expressions.

Justin Sjouw

Posted on 01/07/2022 09:52:30

Hey Nicolai,

Good point! I have a few followup questions on this then :-)

1: When I remove all overides from the app, and just run the search with the "Score" setup from the repository, the result with the exact match at the start "Gulpener Pils 24*30" is still third in list. Is there any way I can influence that Score?

2: You say "And how you have setup your scoring and expressions." I did find this thread about this topic, but is has some missing screencasts. Is there any documenation about setting up scoring? https://doc.dynamicweb.com/forum/ecommerce-standard-features/ecommerce-standard-features/control-scoring-of-keywords-in-search-results

Nicolai Pedersen

Posted on 01/07/2022 10:42:28

This post has been marked as an answer

With that search and what Lucene (the underlying search engine) can do, I think this is as close as you get.

Your search will technically have 2 terms (because it is analyzed) - the terms are "gulpener" and "pils". Since all your results have both terms, scoring is probably pretty similar on all of them.

Scoring can help you get results higher using boost values, but I do not think it will help in this case because Lucene 3, as we are using, does not take term closeness into account. So what you are seeing here is that "Gulpener Pils 24*30cl krat" is divided into 6 terms in the index - "Gulpener", "pils", "24", "30", "cl", "kraft" and since your search only hit 2 out of 6 terms in that field, scoring is lower than your other products that gets hit in 2 out of 4 terms giving a slightly higher score.

If you really want to go further here, a solution could be one of these:

Use stopwords to remove terms from being indexed and hence increase scoring in this case - i.e. add "24", "30", "cl" (and maybe "30cl") to the stopwords list. This will improve scoring for products with these kind of terms on them. Caveat is that those terms are not searchable. So be careful. Stopwords here: https://doc.dynamicweb.com/documentation-9/repositories/indexing/custom-fields#4786 - one thing about stopwords is that they are affecting all fields in the index.
On a product create a custom field for search optimisation - i.e. "product search terms". In that field for this product, add "Gulpener Pils" and leave the remaining terms out of it. On the index, add this field, analyzed with a boost of i.e. 5, and add the field to the query using "Equals" operator on the "q" parameter. This will increase scoring for the product quite a bit and probably fix these things. You might be able to automate this by adding a SQL scheduler that takes the name of the product and remove stuff like numbers and abbrevations like you would do in stopwords.

BR Nicolai

Votes for this answer: 1

Justin Sjouw

Posted on 04/07/2022 10:37:59

Hey Nicolai,

Thanks a million for your elaborate answer, that saves me quite a few hours of digging around documentation :-)

Fortunaterly I already suggested to the customer that it would be best to create separate text fields on products in order to control search results more, so that is most likely the path we will be going down.

Best regards,

Justin

You must be logged in to post in the forum

Developer forum

Sorting of search results

Replies