Vulnerability in EPiServer.Forms
Using EPiServer.Commerce 12.17.2 and EPiServer.CMS 11.11.2:
We have been talking with Microsoft Azure support team for a while now because of a random app pool reycle that takes place on one of our client's website. The client has about 180K products in their online catalog. The plan is to keep adding products up to 1 Million.
Looking into the memory dumps from a high memory prespective, these were the results:
-Bytes in All Heaps 6,804,301,128 <== Managed heap is nearly 7GBGCHandles 30,665GEN 0 Collections 21,086GEN 1 Collections 7,970GEN 2 Collections 454# Induced GCs 24 <== GC.Collect() calls
There are millions of EPiServer.* objects flooding the managed heap:00007fff38fad170 451149 39701112 EPiServer.SpecializedProperties.PropertyXhtmlString00007fff29dfe840 677449 43356736 EPiServer.Core.PropertyBoolean00007fff38eecb78 1289409 51576360 EPiServer.Core.ContentReference00007fff29dfea78 900046 64803312 EPiServer.Core.PropertyFloatNumber00007fff38fa92e0 677173 70425992 EPiServer.SpecializedProperties.PropertyContentArea00007fff29c842d0 1823778 72951120 EPiServer.Util.Internal.AutoCompressString00007fff29c83dc8 1351222 86478208 EPiServer.Core.PropertyDataCollection00007fff38fc63c0 1125445 108042720 EPiServer.Core.PropertyContentReference00007fff38f223e8 2025090 162007200 EPiServer.Commerce.SpecializedProperties.PropertyDecimal00007fff38f248b8 3376284 216082176 EPiServer.Core.PropertyNumber <== 3 million objects00007fff2a157f78 1351160 217956944 EPiServer.Core.PropertyData00007fff38f20fc0 12381093 990487440 EPiServer.Core.PropertyLongString <== 12 million objects
Any recomenndations on how to locate the modules responsible of this high memory consumption? Whether it is a custom code or an EPi module?
The website has P3V2 App Service Plan.
180k products are not a very big number . It sounds like the site loads too many products than it needs. There is no easy way to spot that, but if you has captured memory dumps before the site restarts then Application support should be able to look into it and provide some guidance
@Quan Mai: I cannot thank you enough for taking the time to respond to our posts!
The issue was related to our custom products indexing job where we load the whole catalog to custom index it in FIND.
Using your tutorial here of traversing the catalog contributed in resolving the issue: https://leanpub.com/epicommercerecipes/read_sample
This brings another question to my mind related to memory management. I understand there is an expiry time for VariantContent cached in the memory.
But shouldn't the memory forced to free these cached objects if it reaches its maximum capacity? I guess cached objects should have higher priority than any other objects in memory.
Thanks again for the help.
You are very welcome.
Yes in theory asp.net will trim the cache if there is a memory pressure. However you might have setting that keep the threshold high (e.g the cache will not be trimmed until it reaches 95% ) and you might have auto heal rule that restarts the instance if it reaches, says, 90%. Is it the case?
if you are running Cms11+ and are using scheduled job you have a way to reduce cache time
We turned off auto heal for troubleshooting this issue. The issue was still occuring.
Yes, we do use CMS 11.11.2. My understaning is contents loaded by schedule job have very short expiration time if I am not mistaken
I misread it a little bit, so you are trying to use the recipe to resolve the problem, not that it caused the problem :) . As long as you load content in batches AND you don't have too aggressive memory management thresholds (I would suggest to start with all default settings), it should be fine.
And yes, CMS 11 allows you to reduce the caching time, as if you are indexing the content you just need them once, then discard.
Wait a second, why don't you have the blue badge below your name?
Do you need help with commerce certifiication?!! :)