Jens Lundström
Apr 29, 2011
  3688
(0 votes)

Crippled Relate+ 2 R2 web site, low CPU hang, DebugDiag, Windbg and a happy ending

Whether you want it or not, sooner or later you'll probably wind up in front of a crippled production environment without any meaningful logs to hold hand with. The pressure is high, let this heavily shortened story begin..

This particular website is hosted on four different web servers, where two are located on a DMZ. After upgrading to Relate+ 2 R2 the websites hosted on the DMZ servers randomly starded to show the unwanted symtoms of low CPU hangs. When browsing the web site you'll end up with a "ReadResponse() failed: The server did not return a response for this request.". And the eventlogs, EPiServerlogs, httperrlogs all say - Nothing!

Tortured by the memories of earlier unexplainable web server errors I realized that it once again was time to grab DebugDiag and try to get a memory dump from at least one of the servers. With the dump on disk it was time to start WinDbg and put on my Tess Ferrandez googles.

After some dirty work and precious time lost (just to get WinDbg to work with a dump from a different OS version is a challange) I stumbled on this webpage:

http://blogs.msdn.com/b/dsvc/archive/2008/06/24/troubleshooting-startup-issues-with-managed-application-having-authenticode-signature.aspx

To shorten the story even more (time is precious):

The root cause to the low cpu hang was  the assemblies configured in the compilation section in web.config. From the webpage linked above:

"When the CLR loads an assembly which has an Authenticode signature, it will always try to verify the signature by connecting to the Internet to generate Publisher evidence for the assembly. This is true even if there is no internet connectivity available."

In this particular case the url requested was http://crl.thawte.com/ThawtePrem. Since our hosting partner has a very strict firewall configuration for outgoing http traffic (and more), the unexpected request was blocked which in turn triggered the startup process to halt completely and wait for the verification call to timeout.

To fix this problem you either have to modify the Windows\Microsoft.Net\FrameworkXX\2.0...\Aspnet.config file with

<configuration>
  <runtime>
    <generatePublisherEvidence enabled="false"/>
  </runtime>
</configuration>

or open the firewall for outgoing request.

If you bump in to this low CPU hang error, I'll problably can give you some more concrete advices in the debug process. This post was just an attempt to clear my thoughts, the details are left out.

Apr 29, 2011

Comments

Please login to comment.
Latest blogs
Update on .NET 8 support

With .NET 8 now in release candidate stage I want to share an update about our current thinking about .NET 8 support in Optimizely CMS and Customiz...

Magnus Rahl | Oct 3, 2023

Adding Block Specific JavaScript and CSS to the body and head of the page

A common requirement for CMS implementations includes the ability to embed third party content into a website as if it formed part of the website...

Mark Stott | Oct 3, 2023

Performance optimization – the hardcore series – part 1

Hi again every body. New day – new thing to write about. today we will talk about memory allocation, and effect it has on your website performance....

Quan Mai | Oct 3, 2023 | Syndicated blog

Next level content delivery with Optimizely Graph

Optimizely introduced a new product called Optimizely Graph earlier this year. We were one of the first partners to adopt this new service in a...

Ynze | Oct 2, 2023 | Syndicated blog

IDX21323 - RequireNonce is true, Nonce was null

We have multiple clients configured with Azure Active Directory (Microsoft Entra) for requiring authentication when accessing their website. The...

David Drouin-Prince | Oct 1, 2023 | Syndicated blog

Minimum Detectable Effect in Optimizely Web Experimentation

Understanding Minimum Detectable Effect Minimum Detectable Effect (MDE) is a core statistical calculation in Optimizely Web Experimentation. In...

Matthew Dunn | Oct 1, 2023 | Syndicated blog