Developer forum

Oh. I have forgotten something. If I use "Analyzed" it is converted properly into a String array but the values inside are split also by space. I want to avoid that part.

Maybe I can use a new Field Type with an analyzer?

Thank you,

Adrian

They should not be analyzed.

Hi Nicolai,

I have not analyzed it.

This is an example: http://farmaci.cloud.dynamicweb-cms.com/produktet/farmaci/rruget-e-frymemarrjes/dhimbje-fyti

You will see values with comma: Softgel,Tablet

I have attached a screenshot with my field configurationm.

I am not sure if it makes any difference if the field is a Reference field.

Adrian

Hi Adrian,

The main issue is that the field type of the Reference Field is Selection Box, which is a field type that you've created. That means that the index builder doesn't know how to deal with the values. In this case, the index builder simply writes the values as it's given by the database.

There are a couple ways to solve this, but they all require custom code.

1. Create an index builder extender for the ProductIndexBuilder by inheriting IndexBuilderExtenderBase<ProductIndexBuilder> and change the value for the "Formulation_facet" field. You need to split the value on "," and set the value in the document as an array. In addition, you need to change the field data type to System.String[].
2. Create a custom analyzer that is capable of dealing with values concatenated by ",". Be aware that depending on how you configure the field data type, it may cause array values to be treated a little different than normal. This is only for fields that are defined to use a custom field type that uses this analyzer.

Personally, I prefer 1).

- Jeppe

Hi Jesse,

Thank you very much for your detailed answer.

I was basically expecting one of these scenarios but I wanted to make sure what the right direction would be.

We will probably go with option 1 since it feels a bit more familiar. I would imagine that option 2 involves a bit more Lucene knowledge.

Thank you very much,

Adrian

Hi Jeppe,

I have re-analyzed my options and it looks like option 2 might be more flexible in our situation.

There will be new fields added in the future based on the same FieldType provider and this means that we will have to always adjust the Index extender to include new fields. Which is not that flexible.

I would like to explore your second option by creating a custom analyzer. Do you have any example of such an analyzer that I can use as a starting point?

Also, I was wondering if it would be possible to set the "presentation type" (or other property) in my FieldTypeProvider that would help our current setup identify these fields properly?

If I remember correctly, the current approach is that the index could convert into a string[] any text field that would contain a comma-separated list of values no matter if the field is set as a Text or ListBox (it might do it automatically for ListBox). It just seems that in our situation, the system cannot handle it properly and I am wondering if we missed something in the code for our FieldTypeProvider.

Thank you,
Adrian

Hi Adrian,

Looking into this a bit further, you might actually be able to use the "DynamicwebMetadataKeywordAnalyzer" to tokenize your text string. It does split on a few more characters in addition to ",", namely ";" and "|". Also, it transforms all tokens to lowercase. If you want this behavior, the I'd recommend that you use this one instead of creating your own.

If, however, this behavior is not what you seek, you can take a look at the implementation of that analyzer. The code is under Providers in the Lucene project. You can also take a look at the Keyword analyzer built into Lucene here: https://github.com/apache/lucenenet/blob/3.0.3/src/core/Analysis/KeywordAnalyzer.cs. My guess is you probably need an analyzer that does a very similar thing but also splits the tokens.

I hope that helps.

- Jeppe

Hi Jeppe,

Thank you very much for the info.

I have already tested DynamicwebMetadataKeywordAnalyzer and it splits correctly but the lowercase part is what is not working for my case. I will have a lot of values and it would be too complicated to add them into Translations.

However, the tip about the "|" is very useful for another scenario :)

It looks like for this scenario, I will have to create a custom one.

I will have a look at the code you have indicated and I will come back with more questions if I stumble on anything.

Thank you very much,
Adrian