Developer forum

Forum » CMS - Standard features » File Index - Query Publisher with metadata

File Index - Query Publisher with metadata

Florin Patrascanu
Reply

Hello,


I've got the following scenario : 

File Index Builder that indexes a folder in which the files have some metadata tags set on them. In the settings the SkipMetadata property is set to 'False'. However , in the query I cannot find my metadata field (called 'Model') and only the IPTC group is being displayed in the source dropdown. Is there a way to display the DynamicwebMetadata group as well? Or more precisely , what should I do to be able to search for that metadata field?

Best regards,

Florin

FileIndexSetup.png FileMetadata.png QueryResult.png QuerySource.png

Replies

 
Jeppe Eriksson Agger Dynamicweb Employee
Jeppe Eriksson Agger
Reply
This post has been marked as an answer

Hi Florin,

I've looked into your issue, and there is definitely something amiss here.

The major thing preventing you from using Dynamicweb Metatags in your queries and facets is that fields are serialized into one array field. This is not solvable without custom code. The issue is that you get one field where each metatag becomes one value in the format "<tag label> <tag value>". This means that you cannot make proper queries using this field and you cannot create facets for these tags either.

I'll add a task to our backlog to get this fixed, but until then you can create a custom project with a new SchemaExtender and IndexBuilderExtender to add the metadata in a format that can be used for querying and facetting. Maybe something like this:

using Dynamicweb.Content.Files;
using Dynamicweb.Content.Files.Metadata;
using Dynamicweb.Core;
using Dynamicweb.Indexing;
using Dynamicweb.Indexing.Schemas;
using System.Collections.Generic;
using System.IO;
using System.Linq;

namespace ExtendedFilesIndexer
{
    public class ExtendedFilesIndexBuilderExtender : IIndexBuilderExtender
    {
        public void ExtendDocument(IndexDocument indexDocument)
        {
            var fullPath = indexDocument?["DirectoryFullPath"]?.ToString();
            var fields = Helper.GetMetaFields(fullPath);

            if (!fields.Any())
                return;

            var fileFullPath = indexDocument?["FileFullName"]?.ToString();
            var metadata = EditorFactory.GetMetadataForFile(fileFullPath);
            if (metadata == null)
                return;

            foreach (var metafield in fields)
            {
                indexDocument[$"MyMetaTags|{metafield.Id}"] = metadata.GetValue(metafield.Id);
            }
        }
    }

    public class ExtendedFilesSchemaExtender : FileIndexSchemaExtender, IIndexSchemaExtender
    {
        public new IEnumerable GetFields()
        {
            var baseFields = new List(base.GetFields());

            var fields = Helper.GetMetaFields(SystemInformation.MapPath("/Files"));
            foreach (var field in fields)
            {
                baseFields.Add(new FieldDefinition
                {
                    Name = field.Label,
                    SystemName = $"MyMetaTags|{field.Id}",
                    Source = $"MyMetaTags|{field.Id}",
                    Indexed = true,
                    Stored = true,
                    Analyzed = false, // This can be changed according the needs
                    TypeName = "System.String",
                    Group = "Metadata Tags",
                    IsFieldFromSchemaExtender = true
                });
            }

            return baseFields;
        }
    }

    internal static class Helper
    {
        public static List GetMetaFields(string path)
        {
            var fileDirectory = new DirectoryInfo(path);
            var subdirectoriesAndSelf = GetSubDirectoriesAndSelf(fileDirectory);

            var fields = new List();
            foreach (var dir in subdirectoriesAndSelf)
            {
                var relativePath = dir.FullName.Replace(SystemInformation.MapPath("/Files"), "Files");
                var metaFields = EditorFactory.LoadConfig(relativePath);
                if (metaFields != null)
                {
                    fields.AddRange(metaFields);
                }
            }
            return fields;
        }

        private static List GetSubDirectoriesAndSelf(DirectoryInfo dir)
        {
            var list = new List();
            FillSubDirectoriesAndSelf(dir, list);
            return list;
        }

        private static void FillSubDirectoriesAndSelf(DirectoryInfo dir, List list)
        {
            list.Add(dir);
            foreach (var subdir in dir.GetDirectories())
            {
                FillSubDirectoriesAndSelf(subdir, list);
            }
        }
    }
}

I hope this helps you proceed.

- Jeppe

Votes for this answer: 2
 
Jeppe Eriksson Agger Dynamicweb Employee
Jeppe Eriksson Agger
Reply

Forgot to mention that once the custom project is added to the bin-folder, you'll need to switch the schema extender type for your index and rebuild the index.

 
Florin Patrascanu
Reply

Hello Jeppe,

 

Thanks for your quick reply on the issue. I will try to use what you've posted above and see how it goes. 

Also , it would be very handy if this will turn into a feature request for future use cases.

Best regards,

Florin.

 
Nuno Aguiar Dynamicweb Employee
Nuno Aguiar
Reply

Hi Jeppe,

 

If you're having someone invest some time on this, might be useful to add a secondary index builder option so that  "Skip metadata" and "Skip Dynamicweb metadata" are 2 separate things. We've run into issues in a project where the amount of files resulted in 30+ minutes of indexing because we needed DW metadata, but not file metadata.

 

Best Regards,

Nuno Aguiar

 

You must be logged in to post in the forum