Dynamic indexing of Commerce Attributes

Vote:
 

Hi Commerce forum

In our new webshop, we are maintaining product data and structure in an external PIM system. Even though product structure can be dynamic, we still want to achieve a great search and navigation functionality, so we need to dynamically decide which product attributes to index.

I have created this topic in the Search & Navigation forum: https://world.optimizely.com/forum/developers-add-ons-forum/Search/Thread-Container/2024/8/dynamic-indexing-of-commerce-properties/. I'm also posting my question here, since it is also related to Commerce. I hope that is OK, and I suggest that questions, suggestions and help is given on the thread that I have linked to in the Search & Navigation forum.

Let me know if I have broken any rules by posting my question in this way - it was not my intention.

Regards

Anders

#326432
Aug 02, 2024 9:42
Vote:
 

Hi Anders,

I am not sure how dynamic attributes are built in your PIM but I think that there would be some of fixed properties and some of dynamic properties. I suggest that you can consider dynamic properties in PIM as a list of key and value and implement dynamic search based on that list.

#326554
Aug 05, 2024 6:38
Vote:
 

Hi Binh

Thank you for your reply. I think probably Optimizely already have this dynamic list you are mentioning. But how do we index this list in Optimizely Search & Navigation?

Our attributes are normal attributes in PIM. We want our PIM system to be master for product information. So in PIM attributes are managed as they usually are.

Since Optimizely is not the master of product information, we would like Optimizely to reflect how data is modelled in PIM.

We import data into Optimizely using the ServiceAPI and import data using the Catalog.xml file format. This creates products and their attributes nicely in the Commerce subsystem. We also create the MetaFields and MetaClasses in the Commerce subsystem just fine. As far as I can see, this is also reflected in the Commerce Manager user interface. So Optimizely Commerce is workning fine with these attributes being managed in PIM.

I also think CMS is working well with our data. We can create the C# properties that are "fixed" (our webshop functionality depends on these properties being there), and we can access the rest of the attributes using the PropertyDataCollection through the Properties property on the ContentData base class. Using this PropertyDataCollection, we have a dynamic collection with all our attributes and corresponding metadata from the MetaFields in Commerce. We can use this dynamically on the webshop pages, where we need the dynamic functionality.

So I guess Optimizely provides all our attributes as a dynamic collection. How do we ensure that they are indexed and available for search? All the documentation I have found points towards inclusion or exclusion of C# properties. Do you know of any documentation or example that shows how we can define search fields without having them as C# attributes?

Thank you for your help.

Regards

Anders

#326556
Aug 05, 2024 6:51
Manoj Kumawat - Aug 12, 2024 10:18
How complex are those dynamic properties? can a table property in commerce help?
I am assuming a case as if you have a property in future for a product in PIM, for instance "dimension" then I would add it dynamically on table in commerce's product and have search implemented. Since it will be a part of content then it would index it right away.
Vote:
 

Hi Manoj

We want to store the properties as regular Optimizely Commerce properties. We import using Catalog.xml, where we can specify both MetaClass and MetaFields and also set properties as specified from PIM. Using Catalog.xml we can add new MetaFields as we need them (we can't remove them again using Catalog.xml - that is OK).

So in the "Commerce" part, it will be real properties, and we can also see each property in the Commerce Manager UI.

But they are not directly available in the Content part, because we haven't created C# Properties. We can still access the information through the PropertyDataCollection, where they are also available as properties based on the MetaFields specified in Commerce.

But they are not indexed automatically, since indexing only automatically indexes C# Properties. And the Convention functionality also only works with C# properties, as far as I can see. The issue is probably, that C# Properties are defined on compile-time. Conventions are loaded on Commerce startup time (initialization) based on compile-time properties. But we want to index based on runtime information that is specified from PIM.

Regards

Anders

#326895
Aug 12, 2024 11:02
Manoj Kumawat - Aug 12, 2024 12:44
Hello Anders, got you. The commerce properties you are mentioning are not available in content part/CMS. But did you check in index (find) if those properties are being indexed ? I guess yes, For example check any product/Category in find index and check if the properties of your CommerceContent (commerce part) are part of the index document. If yes, it doesn't need to be part of code in content part/CMS in order to get the search work. I am not sure how you have implemented search but generally
_findClient.Search
Vote:
 

Hi Manoj

No, unfortunately our properties are not indexed. Optimizely only indexes fields that are avaiilable as C# Properties (or otherwise handled through the Conventions mechanizm.

We can add a Convention to include the GetProperties property in the index. But then it will be indexed as a 1 key in Search&Navigation with a serialized dictionary value. And we can't use that for facetting.

We are starting to reach the conclusion, that what we are trying to achieve is not possible in Optimizely. But it would be nice, if somebody can point me in the direction that it can somehow be done anyway :)

#326897
Aug 12, 2024 12:52
Vote:
 

Hi Anders,

By default, properties without managed by code are not indexed because indexing job loads contents via IContentRepository as class object of Content Proxy Type Class. This object is serialized as json to send to indexing service. That is why these propeties are indexed by default.

The most simple way that I can think in your case is:

  • Define a dynamic property class
 public class DynamicProperty
 {
     public string Name { get; set; }

     public string StringValue { get; set; }

     public int? IntValue { get; set; }

     public double? DoubleValue { get; set; }

     public DateTime? DateTimeValue { get; set; }

     public bool? BoolValue { get; set;}
 }
  • Add extension method to get all dynamic properties that you want to index
public static IList<DynamicProperty> SearchableProperties(this EntryContentBase content)
{            
    var searchableProperties = new List<DynamicProperty>();

    foreach (var property in content.Property)
    {
        searchableProperties.Add(new DynamicProperty()
        {
            Name = property.Name,
            IntValue = property.Value is int intVal ? intVal : null,
            StringValue = property.Value is string stringVal ? stringVal : null,
            DoubleValue = property.Value is double doubleVal ? doubleVal : null,
            BoolValue = property.Value is bool boolVal ? boolVal : null,
            DateTimeValue = property.Value is DateTime date ? date : null
        });
    }

    return searchableProperties;            
}
  • Include this extension method to index
[ModuleDependency(typeof(InitializationModule))]
[ModuleDependency(typeof(ShellInitialization))]
[ModuleDependency(typeof(IndexingModule))]
public class FindInitialization : IConfigurableModule
{
    public void ConfigureContainer(ServiceConfigurationContext context)
    {       
    }

    public void Initialize(InitializationEngine context)
    {
        if (context.HostType == HostType.WebApplication || context.HostType == HostType.TestFramework)
        {
            var client = ServiceProviderExtensions.GetInstance<IClient>(context.Locate.Advanced);
            client.Conventions.ForInstancesOf<EntryContentBase>().IncludeField(x => x.SearchableProperties());
            client.Conventions.NestedConventions.Add<EntryContentBase>(x => x.SearchableProperties());
        }
    }

    public void Uninitialize(InitializationEngine context)
    {
    }
}
  • Do search via extension method
 public static ITypeSearch<TSource> FilterByDynamicProperties<TSource>(this ITypeSearch<TSource> search,
   string propertyName, object propertyValue) where TSource : EntryContentBase
{
    if (string.IsNullOrWhiteSpace(propertyName))
        return search;
    
    if (propertyValue is int intVal)
    {
        search = search.Filter(x => x.SearchableProperties().MatchItem(y => y.Name.Match(propertyName) & y.IntValue.Match(intVal)));
    }
    else if (propertyValue is double doubleVal)
    {
        search = search.Filter(x => x.SearchableProperties().MatchItem(y => y.Name.Match(propertyName) & y.DoubleValue.Match(doubleVal)));
    }
    else if (propertyValue is bool boolVal)
    {
        search = search.Filter(x => x.SearchableProperties().MatchItem(y => y.Name.Match(propertyName) & y.BoolValue.Match(boolVal)));
    }
    else if (propertyValue is DateTime dateVal)
    {
        search = search.Filter(x => x.SearchableProperties().MatchItem(y => y.Name.Match(propertyName) & y.DateTimeValue.Match(dateVal)));
    }
    else if (propertyValue != null)
    {
        search = search.Filter(x => x.SearchableProperties().MatchItem(y => y.Name.Match(propertyName) & y.StringValue.Match(propertyValue.ToString())));
    }   
    return search;
}
#326900
Aug 12, 2024 13:39
Vote:
 

Hi Binh

Thank you very much for your suggestion. We will take a look at your code, and see if we can make it work. It looks promising.

Regards

Anders

#327407
Aug 13, 2024 7:00
Vote:
 

Hi,

You can use terms facet for nested object like this:

search.TermsFacetFor(prod => prod.SearchableProperties(), item => item.StringValue, item => item.Name.Match(propertyName), action => action.Name = propertyName);
#327414
Aug 13, 2024 10:07
Vote:
 

Hi Binh

Perfect. I think that was the last ting we needed to fix.

Thank you very much for your help.

Regards

Anders

#327415
Aug 13, 2024 10:11
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.