Developer forum

Forum » CMS - Standard features » How do I boost search score based on number of search hits

How do I boost search score based on number of search hits

E-mail notifications
Peter Leleulya
Peter Leleulya
Reply
Posted on 27/11/2020 12:53:39

Hi guys,

I have a 9.8.11 application which still uses the weighted search as once implemented when it was still a DW8 application.
The customer detected that when searching on a specific term some pages are shown above pages with the term in the title.
I've set the title weight of the titles to the max of 10 in the select  option, but the top pages have the term more than 200 times in the content so they are still on top.
The customer doesn't want that, so I thougth it to be a good time to switch to an indexed search on a content index. There I can set weight to whatever value I want.

I created a content repository, content index, content builder, content query.

For the index fields I added 
- the contentindexschemaextender
- A summary field with text fields of paragraph items I want to search through, boost 1
- A summery field with page PageManuText and PageItem titles, boost 100

A query with OR expression with equal and containsextended on the summary fields compared to the search parameter value.
Sorted by score descending


The query builder output rates the result with the term in the title higher than with the term in the content.
Thats fine.

But all results with the term in the content are scored the same.
Doesn't matter if it is there once or more than 200 times ...
I want the score to be higher depending on the number of times the term is found.

But can't seem to find a way how.
What am I forgetting here?

I feel I should know this, but I'm stuggling .... and I 'm not too big to ask ...
I need weekend ...

Second question, and related ....
The weighted search provides a short text to present with the result, the title and a bit of content.
What should be the best approach to have that available on query output? Since it can come from all kinds of item fields ...
And not kill the performance too much ...

 

BR Peter


Replies

 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Posted on 27/11/2020 13:37:29
Reply

Hi Peter,

 

Here's the answer https://doc.dynamicweb.com/forum/ecommerce-standard-features/ecommerce-standard-features/control-scoring-of-keywords-in-search-results

 

You need to use "Equal" if you want scoring. And since the fields should be analyzed, that will work since it can match on every term.

 

"Contains" and "ContainsExtended" will not give you any scoring, hence being the same in your test.

 

Best Regards,

Nuno Aguiar

 
Peter Leleulya
Peter Leleulya
Posted on 27/11/2020 14:59:27
Reply

Hi Nuno,

Thanks for your response ....

Equal is what I had initially ... but that doesn't seem to work ...
Then I added the ContainsExtended, which didn't change anything in the sort order ...

After your response I reduced the query back to just an equal on my 2 summary fields (with an exclusion of 2 layout types):

 

The old weighted search gave this result on a search term, in the top result the term does not exist in the title, but does exist 210 times in the text field of an item on the page:

 

The new indexed query result with just the above mentioned equals returns that same item on a position around 70st with a zero point zero zero something score.
While for example the homepage, with the same bs score, is positioned around 50th where the term only exists once.
(in the weighted search the homepage doesn't even get returned as a result)

The summery field does contain the text field where the term resides so many times ...

 

In the ContentBlock_Text is where the 210x of the search term resides.

 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Posted on 01/12/2020 12:45:36
Reply

Hi Peter,

 

The way Lucene calculates boosting is very complex and I don't remember all of the details, but I know having the same term multiple times in the same field (which is what you end up havving in the summary field) does not affect the scoring too much. Additionally looking at your screendump, you have a boost of 1, which is very little, providing only a small relevance.

 

I've been having some success using the Search for Editor's guidelines for boosting, being 5, 20, 50 and 100.

 

Using your 2 fields "Titels" and "Free Text" I'd say 100 for Titles and 5 for Free text. And as you determine importance based on fields, use those 4 values.

 

Let me know how that works out.

 

Best Regards,

Nuno Aguiar

 

You must be logged in to post in the forum