Episerver Commerce - Catalog Low-level APIs - Improve the performance when importing thousands of product variants in bulk

The proposed implementation aims to offer a solution for importing thousands of product variants in a performant way using catalog low-level APIs.

Introduction

In our projects, we’ve built a connector between Dynamics 365 and Episerver to synchronize the online store and the published catalogs. The D365-Episerver connector, among other functionalities, has few Episerver scheduled jobs to import categories, products and variants from D365 into Episerver.

We are using the Catalog content APIs to synchronize and import catalogs, categories, products, variants, related entries, prices, inventories, images and more.

For one of the projects, we were facing a performance issue to import the catalog due to the number of variants that a product can have.

Some products have 100, 500, 1K, 2K, 3K, 4K, 5K, 6K and even 12+K variants for a total of ~ 100 000 variants in the entire catalog.

Using the Catalog content APIs for variants, the sync products job always failed with timeout exceptions and took in some cases more than 9 hours.

As you can imagine, we had to find a solution to import all these variants in a fast and performant way to reduce the synchronization time.

Solution

Leverage Episerver Commerce - Catalog Low-level APIs to import thousands of product variants in bulk to minimize the synchronization and import time.

Sync a full online catalog from D365 into Episerver in less than 1.5h (25 categories, 83 products and 99 201 variants in 85 minutes).

This blog post describes in detail the complete solution to import and bulk save thousands of variants using low-level APIs.

How it works

An Episerver scheduled job “Sync All Products from D365 into Episerver” is syncing the published Catalogs of the Online Store from Dynamics 365 into Episerver. The sync includes the catalogs, the categories of each catalog, the products of each categories and the variants for each products.

The current solution describes the implementation we are using to bulk save product variants.

Implementation

Let’s go into detail and explain the solution from a code implementation perspective. It consists of:

1. The data models - BaseDTO.cs and VariantDTO.cs
2. The service interface – IVariantIntegrationService.cs
3. The service implementation – VariantIntegrationService.cs
4. The scheduled job caller method - D365SyncProductsScheduledJob.cs

1. Data Models

The base DTO model for all the sync models defines the base properties and a dictionary of key=value properties used to sync the content properties:

public class BaseDTO
{
	public int? Id { get; set; }

	public string Code { get; set; }

	public string Name { get; set; }

	public string DisplayName { get; set; }

	public string Description { get; set; }

	public string Language { get; set; }

	// The key is the property name
	// The value is the property value
	public Dictionary<string, object> Properties { get; set; }

	public object SourceObject { get; set; }

	public string GetJsonSourceObject()
	{
		if (SourceObject == null)
			return string.Empty;

		return JObject
					.FromObject(SourceObject)
					.ToString();
	}
}

The variant DTO model has the following properties:

public class VariantDTO : BaseDTO
{
	public WebCategory Category { get; set; }

	public WebProduct Product { get; set; }
}

The WebCategory and the WebProduct types are simply the catalog content types that we are using for the NodeContent and ProductContent.

The result DTO model has the following implementation:

public class ResultModelDTO
{
	private List<Exception> _exceptions = null;

	public IEnumerable<Exception> Exceptions
	{
		get
		{
			if (_exceptions != null)
				return _exceptions;

			return Enumerable.Empty<Exception>();
		}
	}

	public bool IsSuccess
	{
		get
		{
			return (_exceptions == null || !_exceptions.Any());
		}
	}

	public bool IsFail => !IsSuccess;

	public void AddException(Exception exc)
	{
		if (exc == null)
			return;

		_exceptions = _exceptions ?? new List<Exception>();
		_exceptions.Add(exc);
	}

	public void AddError(string errorMessage)
	{
		AddException(new ApplicationException(errorMessage));
	}

	public void Fail(string errorMessage)
	{
		if (IsFail)
			return;

		AddException(new ApplicationException(errorMessage));
	}
}

2. Service Interface - IVariantIntegrationService.cs

We have an integration library that is responsible for importing data into Episerver Commerce using the Episerver APIs.

It acts as a wrapper library on top of the Episerver APIs and it is used by other libraries to accomplish our integration scenarios.

The variant interface defines the following method pertaining to bulk import variants (the other methods exposed are not relevant to this solution):

public interface IVariantIntegrationService
{
   ...

   ResultModelDTO SaveBulk(IList<VariantDTO> models);

   ...
}

3. Service Implementation – VariantIntegrationService.cs

The integration library contains a service implementation for the above variant interface and the save bulk method (additional code not related to his solution has been removed):

public class VariantIntegrationService : IVariantIntegrationService
{
	#region Members
   
	private readonly CatalogMetaObjectRepository _catalogMetaObjectRepository;
	private readonly int _batchSize = 1000;
	private static readonly ILogger Logger = LogManager.GetLogger();

	#endregion

	#region Constructors

	public VariantIntegrationService(         
		CatalogMetaObjectRepository catalogMetaObjectRepository
		) 
	{   
		_catalogMetaObjectRepository = catalogMetaObjectRepository;
	}

	#endregion

	#region IVariantIntegrationService
	
	public ResultModelDTO SaveBulk(IList<VariantDTO> models)
	{
		var result = new ResultModelDTO();

		var category = models.FirstOrDefault()?.Category;
		var product = models.FirstOrDefault()?.Product;
		
		if (category == null || product == null)
		{
			result.Fail("SaveBulk - Variants - Category or product models are empty.");
			return result;
		}

		var catalogId = product.CatalogId;
		var startDate = DateTime.UtcNow;
		var endDate = DateTime.UtcNow.AddYears(10);

		MetaClass mcsku = MetaClass.Load(CatalogContext.MetaDataContext, "WebVariant");
		int skuMetaClassId = mcsku.Id;

		var catalogContextCurrent = CatalogContext.Current;

		var catalogDto = catalogContextCurrent.GetCatalogDto(catalogId, new CatalogResponseGroup(CatalogResponseGroup.ResponseGroup.CatalogInfo));

		if (catalogDto.Catalog.Count > 0)
		{
			var productEntry = catalogContextCurrent.GetCatalogEntryDto(product.Code, new CatalogEntryResponseGroup(CatalogEntryResponseGroup.ResponseGroup.Variations));

			if (productEntry == null || productEntry.CatalogEntry.Count == 0)
			{
				result.Fail("SaveBulk - Variants - Product entry is empty.");
				return result;
			}

			// Step 1: Create all variants
			foreach (var variantDto in models)
			{
				var variantEntryRow = productEntry.CatalogEntry.FirstOrDefault(i => i.Code == variantDto.Code);

				if (variantEntryRow == null)
				{
					var variantCatalogEntryDto = catalogContextCurrent.GetCatalogEntryDto(variantDto.Code);

					variantEntryRow = (variantCatalogEntryDto.CatalogEntry != null && variantCatalogEntryDto.CatalogEntry.Count > 0)
									  ? variantCatalogEntryDto.CatalogEntry[0]
									  : null;
				}

				if (variantEntryRow == null)
				{
					variantEntryRow = productEntry.CatalogEntry.NewCatalogEntryRow();
					variantEntryRow.CatalogId = catalogId;
					variantEntryRow.ClassTypeId = "Variation";
					variantEntryRow.Code = variantDto.Code;
					variantEntryRow.MetaClassId = skuMetaClassId;
					variantEntryRow.IsActive = true;
					variantEntryRow.Name = variantDto.Name;
					variantEntryRow.StartDate = startDate;
					variantEntryRow.EndDate = endDate;

					if (variantEntryRow.RowState == DataRowState.Detached)
						productEntry.CatalogEntry.AddCatalogEntryRow(variantEntryRow);

					var newVariationRow = productEntry.Variation.NewVariationRow();

					newVariationRow.SetMerchantIdNull();
					newVariationRow.SetListPriceNull();
					newVariationRow.TaxCategoryId = 0;
					newVariationRow.TrackInventory = false;
					newVariationRow.PackageId = 0;
					newVariationRow.WarehouseId = 0;
					newVariationRow.Weight = 0;
					newVariationRow.Height = 0;
					newVariationRow.Length = 0;
					newVariationRow.Width = 0;
					newVariationRow.MinQuantity = 0;
					newVariationRow.MaxQuantity = 0;
					newVariationRow.CatalogEntryId = variantEntryRow.CatalogEntryId;

					if (newVariationRow.RowState == DataRowState.Detached)
						productEntry.Variation.AddVariationRow(newVariationRow);
				}
				else
				{
					variantEntryRow.IsActive = true;
					variantEntryRow.Name = variantDto.Name;
					variantEntryRow.StartDate = startDate;
					variantEntryRow.EndDate = endDate;
				}

				variantDto.Id = variantEntryRow.CatalogEntryId;
			}

			// Step 2 : Save product variants changes
			try
			{
				if (productEntry.HasChanges())
					SaveCatalogEntryDtoByBatch(productEntry);

				// Once saved by batches, set the variantDto.Id
				if (productEntry.CatalogEntry.Count > _batchSize 
					|| models.Any(i => !i.Id.HasValue || i.Id.GetValueOrDefault() <= 0))
				{
					productEntry = catalogContextCurrent.GetCatalogEntryDto(product.Code, new CatalogEntryResponseGroup(CatalogEntryResponseGroup.ResponseGroup.Variations));

					Parallel.ForEach(models.Where(i => !i.Id.HasValue || i.Id.GetValueOrDefault() <= 0), variantDto =>
					{
						variantDto.Id = productEntry.CatalogEntry.FirstOrDefault(i => i.Code == variantDto.Code)?.CatalogEntryId;

						if (!variantDto.Id.HasValue)
						{
							variantDto.Id = catalogContextCurrent.GetCatalogEntryDto(variantDto.Code)?.CatalogEntry[0]?.CatalogEntryId;
						}
					});
				}
			}
			catch (Exception ex)
			{
				var error = $"VariantIntegrationService - Save Bulk - Error when saving product entry with code = {product.Code}";
				Logger.Error(error, ex);

				result.AddError($"{error} : {ex.Message}");
				return result;
			}
			
			// Step 3: Save variant properties (meta objects)
			var language = product.Language.Name;
			var identityName = Thread.CurrentPrincipal.Identity != null ? Thread.CurrentPrincipal.Identity.Name : string.Empty;
			var metaDataContext = CreateMetaDataContext(language);
			var variantMetaObjects = new List<MetaObject>();

			foreach (var variantDto in models)
			{
				if (!variantDto.Id.HasValue)
					continue;

				var metaObject = _catalogMetaObjectRepository.Load(variantDto.Id.Value, skuMetaClassId, language, ReadMode.UnCached);

				if (metaObject == null)
				{
					metaObject = new MetaObject(mcsku, variantDto.Id.Value)
					{
						CreatorId = identityName,
						ModifierId = identityName
					};
				}

				metaObject["DisplayName"] = variantDto.DisplayName;
				metaObject["Description"] = variantDto.Description;
				metaObject["JsonSourceData"] = variantDto.GetJsonSourceObject();

				metaObject["Epi_IsPublished"] = true;
				metaObject["Epi_StartPublish"] = startDate;
				metaObject["Epi_StopPublish"] = endDate;

				metaObject.SetProperties(variantDto);

				variantMetaObjects.Add(metaObject);
			}

			try
			{
				SaveCatalogMetaObjectsByBatch(language, variantMetaObjects);
			}
			catch (Exception ex)
			{
				Logger.Error($"VariantIntegrationService - Save Bulk - Error when saving meta objects for product code = {product.Code}: ", ex);

				// Update meta object 1 by 1
				foreach (var metaObject in variantMetaObjects)
				{
					try
					{
						_catalogMetaObjectRepository.Update(metaObject, language);
					}
					catch (Exception exc)
					{
						var error = $"VariantIntegrationService - Save Bulk - Error when saving meta object for product code = {product.Code}";
						Logger.Error(error, exc);

						result.AddError($"{error} : {exc.Message}");
						return result;
					}
				}
			}
			
			// Step 4: Create all relations (variant - category AND product - variant)
			var categoryEntry = catalogContextCurrent.GetCatalogNodeDto(category.Code);

			if (categoryEntry == null || categoryEntry.CatalogNode.Count == 0)
			{
				result.Fail("SaveBulk - Variants - Category entry is empty.");
				return result;
			}

			var categoryRelation = catalogContextCurrent.GetCatalogRelationDto(catalogId, 
				categoryEntry.CatalogNode[0].CatalogNodeId, 0, "", 
				new CatalogRelationResponseGroup(CatalogRelationResponseGroup.ResponseGroup.NodeEntry));

			var productRelation = catalogContextCurrent.GetCatalogRelationDto(catalogId, 
				categoryEntry.CatalogNode[0].CatalogNodeId, productEntry.CatalogEntry[0].CatalogEntryId, "", 
				new CatalogRelationResponseGroup(CatalogRelationResponseGroup.ResponseGroup.CatalogEntry));

			foreach (var variantDto in models)
			{
				if (!variantDto.Id.HasValue || variantDto.Id.GetValueOrDefault() <= 0)
					continue;

				// Variant - Category
				var variantNodeEntryRelationRow = categoryRelation.NodeEntryRelation.FirstOrDefault(i => i.CatalogEntryId == variantDto.Id);

				if (variantNodeEntryRelationRow == null)
				{
					var variantRelation = catalogContextCurrent.GetCatalogRelationDto(variantDto.Id.Value);

					variantNodeEntryRelationRow = variantRelation.NodeEntryRelation.FirstOrDefault(i => i.IsPrimary);
				}

				if (variantNodeEntryRelationRow == null)
				{
					variantNodeEntryRelationRow = categoryRelation.NodeEntryRelation.NewNodeEntryRelationRow();

					variantNodeEntryRelationRow.CatalogId = catalogId;
					variantNodeEntryRelationRow.CatalogEntryId = variantDto.Id.Value;
					variantNodeEntryRelationRow.CatalogNodeId = categoryEntry.CatalogNode[0].CatalogNodeId;
					variantNodeEntryRelationRow.IsPrimary = true;
					variantNodeEntryRelationRow.SortOrder = 0;

					if (variantNodeEntryRelationRow.RowState == DataRowState.Detached)
						categoryRelation.NodeEntryRelation.AddNodeEntryRelationRow(variantNodeEntryRelationRow);
				}
				else
				{
					variantNodeEntryRelationRow.CatalogId = catalogId;
					variantNodeEntryRelationRow.CatalogEntryId = variantDto.Id.Value;
					variantNodeEntryRelationRow.CatalogNodeId = categoryEntry.CatalogNode[0].CatalogNodeId;
					variantNodeEntryRelationRow.IsPrimary = true;
					variantNodeEntryRelationRow.SortOrder = 0;
				}

				// Product - Variant
				var variantCatalogEntryRelationRow = productRelation.CatalogEntryRelation.FirstOrDefault(i => i.ChildEntryId == variantDto.Id);

				if (variantCatalogEntryRelationRow == null)
				{
					variantCatalogEntryRelationRow = productRelation.CatalogEntryRelation.NewCatalogEntryRelationRow();

					variantCatalogEntryRelationRow.ChildEntryId = variantDto.Id.Value;
					variantCatalogEntryRelationRow.ParentEntryId = productEntry.CatalogEntry[0].CatalogEntryId;
					variantCatalogEntryRelationRow.RelationTypeId = "ProductVariation";
					variantCatalogEntryRelationRow.Quantity = 1;
					variantCatalogEntryRelationRow.GroupName = "Default";
					variantCatalogEntryRelationRow.SortOrder = 0;

					if (variantCatalogEntryRelationRow.RowState == DataRowState.Detached)
						productRelation.CatalogEntryRelation.AddCatalogEntryRelationRow(variantCatalogEntryRelationRow);
				}
			}

			// Step 5: Save all relation changes
			try
			{
				if (productRelation.HasChanges())
					catalogContextCurrent.SaveCatalogRelationDto(productRelation);
			}
			catch (Exception ex)
			{
				var error = $"VariantIntegrationService - Save Bulk - Error when saving product relations for product code = {product.Code}";
				Logger.Error(error, ex);

				result.AddError($"{error} : {ex.Message}");
				return result;
			}

			try
			{
				if (categoryRelation.HasChanges())
					catalogContextCurrent.SaveCatalogRelationDto(categoryRelation);
			}
			catch (Exception ex)
			{
				var error = $"VariantIntegrationService - Save Bulk - Error when saving category relations for product code = {product.Code}";
				Logger.Error(error, ex);

				result.AddError($"{error} : {ex.Message}");
				return result;
			}
		}

		return result;
	}
		  
	#endregion

	#region Private Methods

	
	private void SaveCatalogEntryDtoByBatch(CatalogEntryDto dto)
	{
		if (dto.CatalogEntry.Count <= _batchSize)
		{
			CatalogContext.Current.SaveCatalogEntry(dto);
		}
		else
		{
			int index = 0;
			var newDto = new CatalogEntryDto();

			foreach (var catalogEntryRow in dto.CatalogEntry)
			{
				newDto.CatalogEntry.ImportRow(catalogEntryRow);

				foreach (var catalogItemSeoRow in catalogEntryRow.GetCatalogItemSeoRows())
					newDto.CatalogItemSeo.ImportRow(catalogItemSeoRow);

				++index;

				if (index % _batchSize == 0)
				{
					CatalogContext.Current.SaveCatalogEntry(newDto);
					newDto = new CatalogEntryDto();
				}
			}

			CatalogContext.Current.SaveCatalogEntry(newDto);
		}
	}

	private void SaveCatalogMetaObjectsByBatch(string language, IList<MetaObject> metaObjects)
	{
		var skip = 0;

		while (skip < metaObjects.Count)
		{
			var variantMetaObjects = metaObjects.Skip(skip).Take(_batchSize).ToList();

			// Save batch meta objects
			_catalogMetaObjectRepository.UpdateBatch(new Dictionary<string, IEnumerable<MetaObject>>()
			{
				{ language, variantMetaObjects }
			}, true);

			skip += variantMetaObjects.Count;
		}
	}

	private MetaDataContext CreateMetaDataContext(string language)
	{
		MetaDataContext metaDataContext = CatalogContext.MetaDataContext.Clone();
		metaDataContext.UseCurrentThreadCulture = false;
		metaDataContext.Language = language;
		return metaDataContext;
	}

	#endregion
}

The extension class to set the properties on the variant MetaObject:

public static class ModelsExtensions
{
	public static void SetProperties(this MetaObject metaObject, BaseDTO model)
	{
		if (model == null || model.Properties == null)
			return;

		var values = metaObject.GetValues();

		foreach (var property in model.Properties)
		{
			if (!values.ContainsKey(property.Key))
				continue;

			metaObject.SetMetaField(property.Key, property.Value);
		}
	}
}

Brief description of the above implementation:

• Step 1: Create all the variants
  • Using the CatalogEntryDto of the product we find the existing variant or we create a new catalog entry using productEntry.CatalogEntry.NewCatalogEntryRow() method and a new variant entry using productEntry.Variation.NewVariationRow()
  • When creating a new variation row it is important to set the default values
    
    
• Step 2 : Save product variants changes
  • If the product entry has changes, then save the changes by batches using SaveCatalogEntryDtoByBatch(productEntry)
  • Once saved by batches, initialize the variantDto.Id as it will be needed later on to save the variant properties
    
    
• Step 3: Save variant properties (meta objects)
  • Get the MetaClass of the WebVariant type using the MetaClass.Load()
  • Initialize a meta data context: CreateMetaDataContext(language)
  • Using the CatalogMetaObjectRepository repository, for each variant load or create the MetaObject and save the properties.
  • Use an extension methods to save all properties from the dictionary to the meta object: metaObject.SetProperties(variantDto);
  • Save catalog meta objects by batch
    
    
• Step 4: Create all the relations (variant - category AND product - variant)
  • A variant belongs to a category and has a NodeEntryRelation with a category
  • A variant is related to a product and has a CatalogEntryRelation with the product
  • For each variant, build the variant-category relation and product-variant relation using the GetCatalogRelationDto(), categoryRelation.NodeEntryRelation.NewNodeEntryRelationRow() and the productRelation.CatalogEntryRelation.NewCatalogEntryRelationRow() methods
    
    
• Step 5: Save all relation changes
  • If there are product and category relations, use the SaveCatalogRelationDto() to save the changes
    
    
• Return the result model
  • During the entire process of importing the variants, if there are any errors or exceptions, these are logged and captured in the ResultModelDTO.cs to be returned to the upper calling layers.

4. The scheduled job caller method - D365SyncProductsScheduledJob.cs

The variant integration service is called during the InternalExecute() method of the scheduled job to sync all products from D365 into Episerver.

A snippet of the pseudo code that is triggering the save bulk variants implementation:

private async Task<D365SyncResultModel> SyncProductsInternal(IList<ProductModel> products)
{
	…
	foreach (var product in products)
    {
		// Get product variants		
		var productWithVariants = await syncService.GetProductById(channelId, catalogId, catalogName, product.RecordId);

        product.Variants = productWithVariants.Variants.ToList();

		// Create product dto and variant dtos
	    var productDto = ToProductDto(product, language, webCategory);

	    var resultModelSaveProductDto = SaveProductDto(productDto);
		…
        
		// Bulk save variants
		if (productDto.Variants.Count > 0)
		{
			var resultModelSaveVariantDtos = SaveVariantDtos(productDto.Variants);

			resultModel.AddExceptions(resultModelSaveVariantDtos.Exceptions);

			// Fallback to individual save variant (1 by 1)
			if (!resultModelSaveVariantDtos.IsSuccess)
			{
				ChangeStatus($"Bulk Sync {productDto.Variants.Count} Variants for product code {resultModelSaveProductDto.Model.Code} - Failed. Sync variants 1 by 1 will take a while...", true);

				Parallel.ForEach(productDto.Variants, variantDto =>
				{
					SaveVariantDto(variantDto);
				});
			}
		}
	}
	…
}

private ResultModel SaveVariantDtos(IList<VariantDTO> dtos)
{
	var resultModel = new ResultModel<ResultModelDTO>();

	try
	{
		resultModel.Model = _variantIntegrationService.SaveBulk(dtos);

		if (resultModel.Model.IsFail)
			LogError(new ApplicationException($"Product {dtos.First().Product.Code} variants not saved successfully. See logs for error details."), resultModel);
	  
		resultModel.AddExceptions(resultModel.Model.Exceptions);
	}
	catch (Exception exc)
	{
		LogError(new ApplicationException($"Product {dtos.First().Product.Code} variants not saved successfully. An exception error has been logged!", exc), resultModel);
	}

	return resultModel;
}

We build a list of products and variants DTOs that we save in bulk and if it fails we fallback to the Catalog content APIs to save the variants one by one.

The ToProductDto() converts the ProductModel we have from our SyncService.cs into a DTO model used to import the Product content into Episerver:

private ProductDTO ToProductDto(ProductModel product, string language, WebCategory webCategory, bool skipProperties = false)
{
	var productDto = new ProductDTO()
	{
		CatalogName = product.CatalogName,
		Category = webCategory,
		Code = product.GetCode(),
		Language = language,
		SourceObject = !skipProperties
						? product.SourceObject
						: null
	};

	var propertyMapper = new D365PropertyMapper(D365Settings.RetailServerSelectDefaultValue, language);

	productDto.Name = GetProductName(product, language);

	productDto.DisplayName = propertyMapper.GetPropertyValue(product?.ProductProperties, "Description") ?? productDto.Name;
	productDto.Description = propertyMapper.GetPropertyValue(product?.ProductProperties, "ProductDescription");

	productDto.Properties = propertyMapper.MapProperties(product, language);
	
	productDto.Variants = product.Variants?.Select(variant =>
	{
		var variantDto = !skipProperties
						? ToVariantDto(product, variant, language)
						: ToVariantDtoImages(product, variant, language);

		variantDto.Category = productDto.Category;
		variantDto.ProductName = productDto.Name;

		return variantDto;

	}).ToList() ?? new List<VariantDTO>();

	return productDto;
}

The ToVariantDto() converts the ProductVariantModel into a DTO model used to import the Variation content into Episerver:

private VariantDTO ToVariantDto(ProductModel product, ProductVariantModel variant, string language, bool skipProperties = false)
{
	var variantName = GetVariantName(variant, language);

	var variantDto = new VariantDTO()
	{
		CatalogName = product.CatalogName,
		Code = variant.GetCode(),
		Name = variantName,
		DisplayName = variantName,
		Language = language,
		SourceObject = variant.SourceObject
	};

	if (skipProperties)
		return variantDto;

	var propertyMapper = new D365PropertyMapper(D365Settings.RetailServerSelectDefaultValue, language);
	
	variantDto.Properties = propertyMapper.MapProperties(variant, language);

	return variantDto;
}

Due to the number of variants under a product, we also setup the following property in the configuration file:

<add key="SimplifiedCatalogListingThreshold" value="100000" />

Conclusion

As mentioned in Episerver Commerce documentation, the recommended approach of working with the catalog is using the Catalog content APIs. We are using these APIs for importing the catalog, categories, products, relations and more.

However, in specific scenarios, like ours, the Episerver Commerce low-level APIs are useful for working with the Catalog.

It allowed us to reduce significantly the synchronization time and to sync a full catalog with 85 products and ~ 100 000 variants in less than 1.5h.

I hope it will help you too when importing large catalogs like in our use case!

The Episerver Commerce low-level APIs are described here: https://world.episerver.com/documentation/developer-guides/commerce/catalogs/low-level-apis/

I hope you enjoyed the reading and the proposed solution. Looking forward to your feedback! Thank you!