Developer forum

Forum » Swift » Fields and case sensitivity

Fields and case sensitivity

Imar Spaanjaars Dynamicweb Employee
Imar Spaanjaars
Reply

Hi there,

I had a customer report an issue where searching for a product number like 1234B5678 didn't return any results, but 1234b5678 (same number, but a lower case b) did return results. After some searching I found this post that explains why that is (fields needs to be analyzed in order to be stored in lower case): https://doc.dynamicweb.com/forum/cms-standard-features/cms-standard-features/is-the-index-query-case-sensitive

I then checked how Swift handles this and noticed a few things:

1. The search text is lower cased before submitting to the server
2. There are two fields for the number in the repository: ProductNumber_Search which is only indexed and Product number which is indexed, analyzed and stored.

- \What is the reason for the Search field? If lower casing values, wouldn't the standard number field (which is analyzed) be sufficient?
- Is there a good reason to convert the text to lower case before searching or is that just a Swift convention?

Thanks!

Imar


Replies

 
Nicolai Pedersen Dynamicweb Employee
Nicolai Pedersen
Reply

When a field is analyzed boosting does not work as we want for some fields - i.e. product number.

For a number like ABC123 it is not an issue because the analyzed and non-analyzed version would give the same 'term'. But if the number is ABC-123 an analyzed would give 2 terms and a non-analyzed would give one term. And then boosting would be different - i.e. if searching for 123 would boost high in the analyzed but find nothing in the non-analyzed version.

So we use the nonanalyzed version of the field to ensure high boosting when you enter a precise SKU to ensure that product is on top - i.e. in express buy. But if you search for random things and press enter, we might want to fin ABC123 when searching for AB - and still if you have a product with SKU AB put that up higher.

By default in Swift product number is part of free text summary field that is analyzed and used on a contains expression (which will not add scoring) and then also on the product numnber search (non-analyzed) using Equals which will score the document... And in this way both scoring for precise searches and partial searches for SKU would come out fine.

 
Imar Spaanjaars Dynamicweb Employee
Imar Spaanjaars
Reply

I see.

>>and then also on the product numnber search (non-analyzed) using Equals which will score the document

But that requires you to force to lower case on the search term or the non-analyzed product number wouldn't match and the boost would still not work, no?

Imar

 
Nicolai Pedersen Dynamicweb Employee
Nicolai Pedersen
Reply

Or just add an equals on the analyzed product number would do the same in 99% of cases. Most customers have sane product numbers.

Then crazy customer have SKUs like '"!#¤ % & / 213 ___ØÆÅÅLK ```!RRS(' and they will run into issues no matter what.... :-)

 
Imar Spaanjaars Dynamicweb Employee
Imar Spaanjaars
Reply

>>Or just add an equals on the analyzed product number would do the same in 99% of cases. 

Yeah, that's what I've been doing so far. Think I'll stick with that.

Thanks!

Imar

 

 

You must be logged in to post in the forum