This is an interesting post and a really difficult technical challenge.

Serving a personalised page is always going to require more CPU than serving the same page to every user. For each and every request the server has to work out who you are and what specific content to serve. This takes CPU time.

I doubt this is caused by a bug, so there won't be an 'easy' fix. What I'd suggest is you look slightly further up at your page and system design, and try and optimise there.

1. Can you reduce the server load for other non-personalised assets / content, allowing your CPU time to be focussed on the personalised areas, with everything else served via cache. Donut caching is an interesting technique here, along with properly caching static files.
2. Could you limit the amount of personalised content on each page. This might not be possible, but there has to be a trade off between performance and personalised features, especially when aiming for a one second render time.
3. Could you redesign the pages to make them simpler? Some user interface designers don't consider performance. They should, especially on an e-commerce site.
4. Could you throw more tin at it. I'm not sure what load your site operates under, but if you need fast response times and heavily personalised pages then you will need a 'beefy' set of servers. Time to get the site in the cloud, so you can elastically scale :)

Good luck!

Could you run the page with personalized content under a performance profiler? Hopefully that would indicate which code that takes time, e.g. to figure out if it is a specific critera or similar.

Hi,

Sorry for taking so long updating this post.

Mark, thank you very much for your suggestions, they were very useful input!

This is what we have done so far:

1. We doubled (from 2 to 4) the numbers of CPU's on both frontend servers, and that took the average response time (during a normal load scenario), from 13,1seconds to about 3-3,5 seconds for a page with 4 content areas and 36 personalizations in total within these 4 content areas. With no load it takes 1 user about 3 seconds to load the same personalized page. Trippling the amount of CPU's had no additional effect. 

Now the servers can at least handle the amount of pages it needs to serve per hour without page response times going through the roof. The problem that still remains is getting the response time for 1 page request from 3 seconds to below 1 second.

2. I ran a performance profiler on a similar page in my development environment, and from the result i could see that the following call took alot of time, once per content area:

EPiServer.Web.Mvc.Html.PropertyFor
->ASP.episerver_cms_views_movetoplatform_displaytemplates_contentarea_ascx.__Render__control1
    ->EPiServer.Cms.Shell.MoveToPlatform.PersonalizedContentAreaExtensions.RenderContentArea

Johan, I have sent the performance profiler file to support.

Limiting the amount of personalized content on a page might be an option. My next step is to see how many personalizations it takes before reaching the 1 sec page response time requirement, then running a load test on that page. That would give an indication as to where the trade off between performance and personalization is.

I have done a preliminary upgrade of the solution to EPiServer CMS 7.5, which has added a cache layer to personalization/visitor groups. The HTML request of a page with 25 personalizations dropped from 3-3,5second to 400-800ms.