Product Recommendations - Common Pitfalls

With the added freedom and flexibility that the release of the self-service widgets feature for Product Recommendations provides you as merchandisers, I want to highlight some common pitfalls that you should be aware of, ensuring you can avoid any mishaps that may result in subpar, or even missing, recommendations. 

The Feed 

Although the feed itself is not directly tied to the new self-service widget feature, it plays a hugely important role in recommendations and is often a key factor when an issue does arise. 

Changes to the feed can have direct impacts on performance and the recommendations returned, so you should always be cautious when making updates to the feed. Below are some common pitfalls to be aware of: 

1. Removing products from the feed by accident 

• Explanation – If you remove a product from the feed, we mark it as inactive. Even if that product is available on the site, the recommendation engine does not know this, as it has been removed from the feed. Ensure the feed includes all products you want included in recommendations. This is the same for not having a product in the feed to begin with. For a product to be recommended, it must exist in the latest feed. 

• Solution – Ensure the feed includes all products you want included in recommendations. If you are expecting a product to be recommended, but it is not, then check the feed to see if that product exists in the feed. If not, work with your internal teams to ensure the product is added to the feed for subsequent feed runs. 

2. Removing attribution from the feed 

• Explanation – You may have a series of attributes you pass against your products, such as material or collection. You may be using some of these attributes directly in your recommendation rules, like “match on material.” If you were to remove the collection attribute from the feed, then in the example above, you would be breaking the rules attached to your algorithms, and this could result in no recommendations being returned, or subpar recommendations being returned.  

• Solution – Before removing any attributes from your feed, ensure they are not being used on any algorithms or in any merchandising campaigns. If you plan on updating an existing attribute, like changing collection to productCollection, ensure you update the rules on your algorithms and merchandising campaigns to reflect this new attribute. 

3. Stock levels 

• Explanation – Our system has a hard-coded rule in place that will never recommend products where quantity=0. 

• Solution – If you have products that you want to recommend that have 0 stock, such as made-to-order products, then you can pass a placeholder stock value against that product. As long as the product has quantity>=1, it can be recommended. 

4. Images missing from recommendations 

• Explanation – You may have set up your rendering code to use the product image passed in the feed to render the product image in your widget. There may be times when a product does not have an image attached to it or is assigned a placeholder image. You could therefore be recommending products that have no image displayed in the rendering, which could cause confusion for your customers and will likely result in them skipping that product. 

• Solution – Always ensure your products have an image attached to them in the feed. If this is not possible, I strongly recommend working with our Support team to add logic that does not allow products with no image (or placeholder images) to be recommended. Alternatively, you can update your feed to pass an attribute such as "image" with a y/n value. You could then add a rule to all of your algorithms that reads as r.image="y". This will ensure only products you have designated as having an image can be recommended. 

5. Adding the same attribute to every product 

• Explanation – We have a series of rules that allow you to recommend unique products, such as unique-category or unique-brand. These rules allow you to provide a variety of different recommendations if you wish to. For instance, on a home page, you may wish to provide a wide variety of recommendations for new users to increase the chance of helping them find a product directly relevant to them. You can achieve this by using unique-category on your algorithms. However, there are some edge cases you want to consider here to ensure recommendations are not impacted. There may be times when you apply a new category to a large portion of your catalogue, potentially your entire catalogue, such as a Black Friday or sale category. By doing so, you would make the unique-category rule redundant as now everything belongs to the same category. There is no uniqueness left, and you would end up with 1 product being recommended, and no more. 

• Solution – If you are planning on adding any attribute value that will be consistent across a large portion (or the entirety) of your catalogue, ensure you have updated your algorithms to remove any reference to the respective "unique-…" filter. 

Quick Filters and Advanced Filters 

Now that we have covered the common pitfalls to avoid with the feed, we can explore the filters you can add directly to your algorithms in the self-service widgets feature. 

When exploring the pitfalls below, there are two levels of severity to consider: 

1. Breaking the entire widget 

• If you are applying filters wholesale across your widget, in that every algorithm has a filter attached to it, you could potentially break the entire widget, resulting in 0 recommendations being returned. 

2. Breaking an algorithm, or series of algorithms 

• You may have a filter attached to a single algorithm or a handful of algorithms in the wider stack. For example, you may have a filter on algorithms 1-3 but no filter on algorithms 4-11. If your filters on algorithms 1-3 do not work as expected, this would result in the top 3 algorithms being skipped, and the first set of products coming from algorithm 4 onwards. 

• You will likely still get a full set of recommendations being returned to your widget, but the results will be less effective than intended. The algorithms stack is set up so that the top algorithms will return the most efficient recommendations for the strategy you are trying to achieve. Skipping these algorithms can therefore result in a less than efficient set of recommendations that may result in less engagement from your customers. 

We want to make sure both outcomes are avoided, so below is a list of common pitfalls you want to avoid: 

1. Conflicting rules 

• When adding filters, ensure that there are no conflicts. For instance, you may add a quick filter of "same category" and then add an advanced filter of "category=123". This rule will work perfectly if you are viewing a product from category 123, but for any other category, this rule would not work. 

• Another example can be shown with the downsale or upsale quick filters. These filters will recommend products above or below the price of the product currently being viewed. Additionally, you may have a price minimum or maximum threshold advanced filter, such as only ever recommending products >$30. Let's assume we have an algorithm that aims to recommend >$30 and downsale. If I was viewing a product for $25, the algorithm would try to return products >$30 and <$25 at the same time. This conflict would cause no products to be returned from this algorithm. 

2. Niche rules 

• You may wish to add a niche rule to your recommendations, such as on category 123, only recommend products from category ABC that have a specific colour, material, and style. Depending on how much behaviour these products have, as well as how large those categories are, this rule could be too niche to return a full set of recommendations. 

• If this is the case, you should ensure this rule is not attached to each algorithm in the stack, but rather to have a staggered approach to the rules. Consider having the niche rule on algorithms 1-3 and then removing one rule at a time. For instance, on algorithms 4-6, remove the requirement of style. On algorithms 7-8, also remove colour. Continue this trend until only the most important rule is attached to the algorithm. This will massively increase your chance of having a full set of recommendations in your widget 

3. Not having any fallback algorithms 

• Each widget should have at least two fallback algorithms: 

• Bestsellers by units sold – You are fine to add some quick filters here, but they should be limited to the "unique-…" filters. 

• Fallback recommendations – I recommend having no filters attached here to ensure we have the best chance of receiving a full set of recommendations in the case that the fallback algorithms are needed. 

• If you do not have any fallback algorithms, there is a chance that you may encounter less recommendations than desired, or even 0 recommendations. 

• You may be using fallback recommendations with filters attached to them. This can also potentially result in less than desired, or 0, recommendations. This is why we always suggest at least having 1 final fallback algorithm with no filters attached. 

• Note: There may be times when you are happy to have less than the desired, or 0, recommendations, maybe due to some strict requirements around product matching. If that is the case, let the Support team know, and they can ensure this is reflected in our monitoring systems. 

Merchandising Campaigns  

Merchandising campaigns are a tool available within the Product Recommendations system that allows you to add an additional filter to a specific widget. 

For context, widgets without merchandising campaigns can have up to 3 filters before returning a product.  

1. Algorithm (required)
2. Quick filter (not required)
3. Advanced filter (not required) 

Let's assume you are using a simple widget with 1 algorithm. That widget aims to recommend bestsellers by units sold. Let's also assume we want to serve 10 recommendations in our widget. Our algorithm may return 100 products to begin with. As we only want 10, we will sort those 100 products by their algorithm score, which in this case, is done by taking the top 10 bestselling products across the site. We may wish to then add a quick filter of "same category". This may reduce our pool of 100 products down to 75. We will then take the top 10 of those 75 using the same ordering process. Finally, we may add an advanced filter of sale price >$50, This may reduce our pool down to 50, and again, we will take the top 10 of that 50. 

If there is no merchandising campaign attached to this widget, then we have our 10 recommendations that will be rendered on a given product page. 

A merchandising campaign lets you add a fourth filter, which allows for further refinements if needed. Merchandising campaigns are best used when trying to achieve a rule on a specific portion of a widget. This is because you have the option to define a 'Master Rule', whereby you can use the attribution in your feed to target a specific product, or group of products. For instance, you may want to recommend products >$50 at the widget level, but for category XYZ, which has higher priced products, you want to add a rule to recommend >$100. In your campaign you could add a master rule of category="XYZ" and then add a 'Recommendation Rule' of sale price>=100. You have now added a 4th filter to this area of your widget. 

Now you understand how a merchandising campaign interacts with a widget; it is important to understand that adding a 4th filter can potentially cause complications. The main two concerns are: 

1. Creating a conflicting rule in your campaign that goes against a filter on your algorithms. For instance, having a >$50 filter on your algorithms, and then a <$50 rule in your campaign.
2. Adding an additional filter that when added to the algorithm filters, results in over-filtering, causing a less than expected number of recommendations to be returned. 

Thankfully, within a merchandising campaign, you have the option to preview the rules you are creating on a given widget, so you can ensure your rules are working before ever putting them live. 

I strongly recommend previewing multiple products. This will ensure consistency in your rules and allow you to potentially find any edge-cases you need to clean up. 

Final Thoughts 

When making any changes to your feed or filters, always ensure you are considering the impact this will have on your recommendations. If you take into consideration all the topics mentioned above, you will have a successful journey with your recommendations. 

If you are not directly managing the feed, ensure you have a communication channel internally whereby you are directly informed of any upcoming changes so you can provide feedback or capture any changes you may need to make within your filters. 

For your filters, always ensure you are actively previewing any changes you are making. You have preview functionality both on the widget and campaign level. Also make sure to run some checks on the live site when you push any changes to ensure everything has transitioned smoothly to the live environment. 

If you ever have any concerns, or run into any issues, please contact support@optimizely.com and tag your CSM.