Catalog Import/Export API

Vote:
 

Commercer 14.12.0

Using this as a point of reference : https://docs.developers.optimizely.com/customized-commerce/docs/importing-catalog-data we're trying to do a catalog export, this seems to run for a fair amount of time before being thrown an internal server error:

TypeMismatchException: Content with id '5' is of type 'Foo.Features.Pages.Foo.SitePage.FooSitePage_DynamicProxy' which does not inherit required type 'EPiServer.Core.ContentFolder'

Content id 5 is in fact our top level site node of type "FooSitePage".

Looking through the forums others have come across this before but with no real solution given that seems to be related to what we're trying to do.

We've also referenced Quan's article here : https://vimvq1987.com/export-catalog-with-linked-assets/ which also causes the same error, interesting there is a comment on Quan's blog post with someone with similar issue but no resolution.

Any help would be great

Full stack trace:

EPiServer.Core.Internal.DefaultContentLoader.ThrowTypeMismatchException(ContentReference link, Type actual, Type required)
EPiServer.Core.Internal.DefaultContentLoader.Get<T>(ContentReference contentLink, LoaderOptions loaderOptions)
Mediachase.Commerce.Assets.ImportExport.CatalogItemAssetImportExport.BuildAssetPath(MediaData imageAsset)
Mediachase.Commerce.Assets.ImportExport.CatalogItemAssetImportExport.CreateEntryCatalogItemAsset(CatalogItemAssetRow assetRow)
Mediachase.Commerce.Assets.ImportExport.CatalogItemAssetImportExport.GetEntryAssets()
Mediachase.Commerce.Assets.ImportExport.CatalogItemAssetImportExport.PrepareCatalogItemAssetCollection()
Mediachase.Commerce.Assets.ImportExport.CatalogItemAssetImportExport.ExportXml(XmlWriter exportWriter)
Mediachase.Commerce.Catalog.ImportExport.CatalogImportExport.Export(string catalogName, Stream output, string baseFilePath)
Foo.Features.Tools.ProductImportExport.ProductImportExportController.GetFile(string catalogName) in ProductImportExportController.cs
+
                    _importExport.Export(catalogName, entryStream, Path.GetTempPath());
Foo.Features.Tools.ProductImportExport.ProductImportExportController.Export(string catalogName) in ProductImportExportController.cs
+
            return GetFile(catalog.Name);
lambda_method840(Closure , object , object[] )
Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor+SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, object controller, object[] arguments)
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker)
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Authorization.Policy.AuthorizationMiddlewareResultHandler.HandleAsync(RequestDelegate next, HttpContext context, AuthorizationPolicy policy, PolicyAuthorizationResult authorizeResult)
Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
Mediachase.Commerce.Anonymous.Internal.AnonymousIdMiddleware.Invoke(HttpContext httpContext)
SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext httpContext, bool retry)
Geta.NotFoundHandler.Infrastructure.Initialization.NotFoundHandlerMiddleware.InvokeAsync(HttpContext context, RequestHandler requestHandler)
Foo.Website.Initialization.Middleware.AdminSafeListMiddleware.Invoke(HttpContext context) in AdminSafeListMiddleware.cs
+
        await _next(context);
Foo.Website.Startup+<>c+<<Configure>b__4_0>d.MoveNext() in Startup.cs
+
            await next();
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
#320979
Apr 24, 2024 20:36
Vote:
 

Hi JSpencer,

Which base class of FooSitePage? It is inherited from PageData, right? And currently, you are using this one in link assets for certain products?

#321021
Apr 25, 2024 8:58
Vote:
 

Hi Binh,

Yes, inherited from PageData.

I don't believe any products have a content reference or asset link to FooSitePage

#321024
Apr 25, 2024 9:31
Vote:
 

BuildAssetPath expect the parent link of an asset to be a ContentFolder. In this case it is not which is why it is throwing an exception.

Maybe BuildAssetPath could be more flexible, but can you locate which media it is throwing error at? 

#321025
Apr 25, 2024 9:32
Vote:
 

Hi Quan,

How would you be able to tell as its not in a loop to capture/break.

If I decompile and load symbols for 

Mediachase.Commerce.Catalog.ImportExport.CatalogImportExport

And look through

public void Export(string catalogName, Stream output, string baseFilePath)

Which is what is called, I cannot see in this method any BuildAssetPath?

#321026
Apr 25, 2024 10:36
Vote:
 

It's CatalogItemAssetImportExport, not CatalogImportExport :)

I think if you turn off Just My Code option, you might be able to see the exception which might contain the imageAsset information.

#321027
Apr 25, 2024 10:58
Vote:
 

Turning off Just My Code didn't produce any more information.

Using dotpeek/loading the symboils for CatalogItemAssetImportExport and setting a break point I was able to hit the "first" error and get more information.

Looks like we have an asset called "EW11_WeatheredHickory_OH.jpg" this has a content link id = 335, it has a parent content link id = 332.

The BuildAssetPath takes the parent link and sets this as the contentfolder and sets the id to 22

Tt then checks if this equals the this._assetsRootFolder which in this case = 3

If it doesn't then it takes contentfolder parent link id and tries to set this as the ContentFolder and this Parent Link Id is 5

And this is where we get the above error.

I have checked the usage of this asset and looking at where its used its only used inside catalog items and never touchers Id=5

#321029
Apr 25, 2024 12:41
Vote:
 

BuildAssetPath will work recursively until it find the AssetRootFolder. What's 332 content type and where does it lead to?

Just to be clear we are trying to make the export work for you. once we figure out how it goes into problem we will find a way for BuildAssetPath to work better/be more tolerant  

#321030
Apr 25, 2024 12:58
Vote:
 

Hi Quan,

332 (parent link id of the asset) is a ContentFolder

22 is the "root" folder which is the "For This Site" folder and again is a ContentFolder

3 is the "root" folder which is "For All Sites" Folder and again is a ContentFolder.

Yeah totally understand a way to get it working for us now would be great.

#321031
Apr 25, 2024 13:41
Vote:
 

That is very strange.

Could you run this code - inside a controller probably, passing the (problematic) mediadata as parameter 

        private string BuildAssetPath(MediaData imageAsset)
        {
            var parentFolders = new List<string>();
            var parent = _contentLoader.Get<ContentFolder>(imageAsset.ParentLink);
            while (!parent.ContentLink.Equals(SiteDefinition.Current.GlobalAssetsRoot))
            {
                parentFolders.Insert(0, parent.Name);
                parent = _contentLoader.Get<ContentFolder>(parent.ParentLink);
            }

            return parentFolders.Count > 0 ? String.Join(_assetPathSeparator, parentFolders) : String.Empty;
        }

does it throw exception ? if yes, for what content 

#321038
Edited, Apr 25, 2024 16:09
Vote:
 

FWIW, here's what's used in Foundation, which is working for me. It's based on Quan Mai's blog, so it may not solve your current issue, but maybe there's an update in here that helps? https://github.com/episerver/Foundation/blob/main/src/Foundation/Features/Api/CatalogExportController.cs

To trigger the export, I go to: site.com/CatalogExport/episerverapi/catalogs?catalogName=Fashion

(Where the name of the catalog to be exported is "Fashion")

That downloads a zip file: catalogs.zip

Which includes a single XML file: catalog.xml

The export includes references to the assets (like in Quan Mai's blog), like this:

<CatalogItemAssetCollection><CatalogItemAssets><CatalogItemAsset><CatalogItemType>Entry</CatalogItemType><CatalogItemCode>P-40707713</CatalogItemCode><AssetPath>episerver/Catalogs/Mosey/Women/Jackets/Andrea Coat</AssetPath><AssetName>25758-1433-s19-3-m-b-h.jpg</AssetName><AssetType>mosey.cms.media.imagemediadata</AssetType><MimeType>image/jpeg</MimeType><GroupName>default</GroupName><SortOrder>2</SortOrder></CatalogItemAsset> ......

For the actual asset files themselves, I export them via the standard CMS content export.

#321040
Apr 25, 2024 16:50
Vote:
 

Hi Quan,

Took your code and popped it into a controller. I did modify it so I didn't have to pass in the media asset and instead, as I knew the parent link id just created a new contentreference:

        var parentFolders = new List<string>();
        var parent = _contentLoader.Get<ContentFolder>(new ContentReference(332));
        while (!parent.ContentLink.Equals(SiteDefinition.Current.GlobalAssetsRoot))
        {
            parentFolders.Insert(0, parent.Name);
            parent = _contentLoader.Get<ContentFolder>(parent.ParentLink);
        }

        return parentFolders.Count > 0 ? String.Join("/", parentFolders) : String.Empty;

The same error happened, it does a few loops of the while loop as its traverses up the folder tree but in the end its the same.

Looks like to gets to the id = 22 which is the "For This Site" Asset folder which is of type ContentFolder and then that Parent Id = 5, which is our site top root node. I am guessing this is happening because its at the top of the "For this site" tree, but as its 22 and not 3, which is the GlobalAssetsRoot it tries to go one level up:

Because its not a Global Asset folder, but instead a site one, this seems to indicate that the BuildAssetPath should be using SiteAssetsRoot, unless we're saying that the media folder structure is incorrect and the For This Site should still sit inside "For All sites"

#321091
Edited, Apr 26, 2024 16:21
Vote:
 

Thanks for testing it. that's strange. the content folder should not be child of the start page. I will look into it when time permits

#321130
Apr 27, 2024 10:55
Vote:
 

Hi JSpencer,

As I understood then the page id 5 with type FooSitePage is the start page for your site, right? You are using  "Use site-specific assets" as bellow?

With this above setting, editors could add medias for only site level below "For This Site" folder and this root is child of Start Page by default

So if you are using media contents below "For This Site" root for commerce assets then I must say that system will throw error as same as you got when I reviewed CatalogItemAssetImportExport code.

I am quite sure that it is a bug from framework. While loop is only stopped if parent link is Global Assets Root => If ancestor is not Global Assets Root then there is no condition for breaking While loop. We will get error from "Get<ContentFolder>" if parent link is not Content Folder. And "For This Site" with parent is Start Page is a case.

I suggest some solutions for it:

  1. Raise this bug to support team and ask for hotfix if it is rush with you
  2. Move media from "For This Site" to "For All Sites" temporary
#321216
Apr 29, 2024 6:38
Vote:
 

Thank Bình 

I have created bug COM-17997 so it will handle assets better.

#321217
Apr 29, 2024 6:43
Vote:
 

Yes Binh, these are exactly my findings too

#321219
Apr 29, 2024 8:11
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.