November Happy Hour will be moved to Thursday December 5th.

Troubleshooting Remote Events issues

Vote:
 

I'm working on implementing a new website with Episerver CMS 10.10.4 and Episerver Commerce 11.2.6.

We have 2 servers into the load balancer and the CMS interface is available on both.

We’ve been having issues with setting-up the Episerver remote events: when CMS updates are published on one server, they are not getting published on the second server as well.

We’re using the default configuration for UDP multicast:


    
    
      
        
      
    
    
      
    
    
      
        
      
    
    
      
        
          
          TODO: the option should be only in test environment true in the production should be false
        
      
    
    
      
        
          
          
        
      
    
  

The only error we’re seeing into the Windows Events Viewer when updates are published are related to WebSocket Protocol Exceptions:

Event time: 12/8/2017 12:46:18 AM
 
    Exception type: WebSocketException 
    Exception message: The WebSocket instance cannot be used for communication because it has been transitioned into an invalid state.
   at System.Web.WebSockets.AspNetWebSocket.ThrowIfAborted()
   at System.Web.WebSockets.AspNetWebSocket.CloseAsyncImpl(WebSocketCloseStatus closeStatus, String statusDescription, CancellationToken cancellationToken, Boolean performValidation)
   at System.Web.WebSockets.AspNetWebSocket.CloseAsync(WebSocketCloseStatus closeStatus, String statusDescription, CancellationToken cancellationToken)
   at EPiServer.Shell.UI.Messaging.Internal.DefaultWebSocketHandler.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EPiServer.Shell.UI.Messaging.Internal.PushMessageHandler.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.WebSocketPipeline.d__9.MoveNext()

As per the infrastructure team, UDP is enabled and port 5000 is open. Also Web Socket protocol is enabled.

 We’ve tried updating the Web.config to use UDP unicast or TCP protocol, but neither of them worked.

 

Could someone please advise whether the configuration is correct and how can we troubleshoot the remote events issues?

We've successfully used the same configuration for other projects in the past.

Background Information:

We are using VMs and hosting is done by the client

Servers:

  • 2 web servers for Test environment
  • 2 web servers for Production environment
  • Note: Issue reproduces in both environments
#186401
Edited, Dec 19, 2017 11:38
Vote:
 

Hi!

Your config above is using TCP protocol, not UDP. Just to avoid confusion, is the above your current config? If yes, do you have the following config on the other server?

<system.serviceModel>
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
    <services>
      <service name="EPiServer.Events.Remote.EventReplication">
        <endpoint name="RemoteEventServiceEndPoint" contract="EPiServer.Events.ServiceModel.IEventReplication" bindingConfiguration="RemoteEventsBinding" address="net.tcp://10.255.11.194:5000/RemoteEventService" binding="netTcpBinding" />
      </service>
    </services>
    <client>
      <endpoint name="10.255.11.195" address="net.tcp://10.255.11.195:5000/RemoteEventService" binding="netTcpBinding" bindingConfiguration="RemoteEventsBinding" contract="EPiServer.Events.ServiceModel.IEventReplication" />
    </client>
    <behaviors>
      <serviceBehaviors>
        <behavior name="DebugServiceBehaviour">
          <serviceDebug includeExceptionDetailInFaults="true" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <bindings>
      <netTcpBinding>
        <binding name="RemoteEventsBinding" portSharingEnabled="true">
          <security mode="None" />
        </binding>
      </netTcpBinding>
    </bindings>
  </system.serviceModel>
#186405
Dec 19, 2017 12:44
Vote:
 

Hi Mattias,

Sorry, I pasted the TCP config by mistake. The curent config we're using is the UDP one. I updated my initial post.

Regards,

Roxana

#186406
Dec 19, 2017 12:49
Vote:
 

Hi again,

Do you have multiple installations on the same network (are Test and Production on the same network)? Then you need to make sure you use unique port numbers for each Episerver installation. I would also suggest reading this blog post if you haven't already:

https://www.nimeshjm.com/episerver-remote-events-troubleshooting/

#186407
Dec 19, 2017 13:12
Vote:
 

You can also try to use TCP.

Let's assume web server 1 in Test has IP address 192.168.0.100 and web server 2 in Test has IP address 192.168.0.101.

Config for server 1 would look like this if you would use port number 13000:

<system.serviceModel>
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
    <services>
      <service name="EPiServer.Events.Remote.EventReplication">
        <endpoint name="RemoteEventServiceEndPoint" contract="EPiServer.Events.ServiceModel.IEventReplication" bindingConfiguration="RemoteEventsBinding" address="net.tcp://192.168.0.100:13000/RemoteEventService" binding="netTcpBinding" />
      </service>
    </services>
    <client>
      <endpoint name="192.168.0.101" address="net.tcp://192.168.0.101:13000/RemoteEventService" binding="netTcpBinding" bindingConfiguration="RemoteEventsBinding" contract="EPiServer.Events.ServiceModel.IEventReplication" />
    </client>
    <behaviors>
      <serviceBehaviors>
        <behavior name="DebugServiceBehaviour">
          <serviceDebug includeExceptionDetailInFaults="true" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <bindings>
      <netTcpBinding>
        <binding name="RemoteEventsBinding" portSharingEnabled="true">
          <security mode="None" />
        </binding>
      </netTcpBinding>
    </bindings>
  </system.serviceModel>

Config for server 2 would look like this:

<system.serviceModel>
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
    <services>
      <service name="EPiServer.Events.Remote.EventReplication">
        <endpoint name="RemoteEventServiceEndPoint" contract="EPiServer.Events.ServiceModel.IEventReplication" bindingConfiguration="RemoteEventsBinding" address="net.tcp://192.168.0.101:13000/RemoteEventService" binding="netTcpBinding" />
      </service>
    </services>
    <client>
      <endpoint name="192.168.0.100" address="net.tcp://192.168.0.100:13000/RemoteEventService" binding="netTcpBinding" bindingConfiguration="RemoteEventsBinding" contract="EPiServer.Events.ServiceModel.IEventReplication" />
    </client>
    <behaviors>
      <serviceBehaviors>
        <behavior name="DebugServiceBehaviour">
          <serviceDebug includeExceptionDetailInFaults="true" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <bindings>
      <netTcpBinding>
        <binding name="RemoteEventsBinding" portSharingEnabled="true">
          <security mode="None" />
        </binding>
      </netTcpBinding>
    </bindings>
  </system.serviceModel>
#186408
Dec 19, 2017 13:19
Vote:
 

No, Test and Production don't have access to each other. Actually, the issue was present before we set the Production environment.

I went through the document, but didn't try to use any of the tools to troubleshoot the issue. I will try and let you know the results.

#186409
Dec 19, 2017 13:29
Vote:
 

Hi Mattias,

No, Test and Production don't have access to each other. Actually, the issue was present before we set the Production environment.

I went through the document, but didn't try to use any of the tools to troubleshoot the issue. I will try and let you know the results.

I already tried the TCP config and UDP unicast as well, but no luck. 

#186410
Dec 19, 2017 13:31
Vote:
 

The Exception you pasted, I might be wrong but that seems not to be related to event system but rather for websocket updates in UI.

Do you get any other information in the logs regarding event system when you update a page? Does it work in any direction?

What you could do is to verify that service is running on the server on that port with "netstat -a" in commando prompt.

In the windows firewall depending on your profile. Make sure that both inbound and outbound is enabled for UDP packets on port 5000. This is often the case that you have to add it manually.

#186413
Dec 19, 2017 14:07
Vote:
 

Also if you have multiple NICs on the server make sure the packets are sent on the correct network. See my blog post:

https://world.episerver.com/blogs/Sebastian-Lundh/Dates/2011/7/Multicast-configuration-with-multiple-NICs/

#186414
Dec 19, 2017 14:13
Vote:
 

Update: We fixed the issue by using the TCP configuration. We also added an inbound & outbound rule on each server, to allow TCP packets on port 5000.

We couldn’t make UDP multicast work even with the firewall down.

Thank you for your suggestions, Mattias and Sebastian.

Regards,

Roxana

#187001
Jan 10, 2018 7:54
This topic was created over six months ago and has been resolved. If you have a similar question, please create a new topic and refer to this one.
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.