Scheduled jobs setup in DXC Service
Scheduled jobs is one of the essential features of Episerver platform. The main purpose of scheduled jobs is to execute in background repeatable long-running tasks within Episerver context, but outside of a web request. You can find more information about this feature here.
A long-running and resource-intensive scheduled job might negatively affect application performance when it runs on the same web server that also serves site visitors requests. There is a known solution for this - when the application is hosted on multiple web servers one of the web servers is dedicated to scheduled jobs by applying a configuration change (see Multi-sever scenario in the scheduled jobs documentation) and exclude the server from the load balancers list. This setup works fine for the hosting scenarios when the site administrator has full access to the web servers. However, in case of multi-tenant hosting solution like Azure App Service, there is no way to control configuration for individual web server instances. Here is how we solve this in DXC Service.
Since each web server instance in Azure App Service has identical application codebase and configuration, there is no way to configure the enableScheduler attribute for a particular instance. Instead, we configure enableScheduler=”false” for the whole WebApp that serves site visitor requests, and create a separate WebApp dedicated to scheduled jobs, that has the same codebase and configuration as the primary one, except enableScheduler is set to ”true.” To avoid any performance impact on the primary WebApp, the scheduled jobs WebApp is hosted on a separate App Service. Each App Service in Azure is essentially a set of web workers that can host one or multiple web applications (more in-depth information about this topic is available here). By using a separate App Service, we can ensure that scheduled jobs will run on dedicated web workers. The scheduled jobs WebApp and App Service is provisioned as needed in DXC Service and updated automatically by DXC deployment flow together with the primary WebApp, so this process is completely transparent for DXC customers and doesn’t require any code changes or configuration updates. The DXC deployment flow will also configure the enableScheduler attribute automatically. Note that dedicated scheduled jobs WebApp and App Service are created only on Pre-production and Production environments, but not on Integration.
When developing a scheduled job for a site hosted in DXC-S, it is important to define the job as stoppable - set the IsStoppable property to true and implement the Stop method. This will give a chance for your job to perform a graceful shutdown when the web worker instance is recycled during jobs execution. Another important consideration, especially for the long-running jobs and jobs processing massive amounts of data, is to configure the job as Restartable. This will tell the scheduler service to start the job again if the job was shut down during the previous execution. It’s up to the jobs implementation to store some checkpoint to be able to resume the processing when restarted.
separate app service for the job host is all great. just my 2 humble cents to hint on looking also at how azure functions or wejobs are implementing their "bookkeeping" and tracking which job is ran and which needs to be executed. they are using shared azure storage blob to issue and release locks that are synchronization mechanism for the function host itself. otherwise they could not ensure massive scale out and other interesting features. it then would open up some other interesting features for scheduled jobs like - "oh, dedicated app service is all taken up by this massive/aggressive commerce product import process, but I have to spin up another" ;)
If we would like to have this setup ourselves in Azure not running DXC-S, I guess that means we need an additional license for the Scheduler Web App? Or is a license not required if the app is not publically available?
I have the simular setup to get standard Episerver Search to work in "ordinary" Azure (read more here: https://world.episerver.com/blogs/Henrik-Fransas/Dates/2015/8/setup-episerver-search-in-a-future-safe-and-stable-way/) and for that I have not felt the need for aditinal license since it is not showing anything to visitors.
How exaclty do we do set up other App Services within the DXC? I've only ever been given access to the Azure portal for Integration and have no control of the resources on the DXC? Are you expecting Episever to set this up? Does this require paying more?
@Scott Reed the App Service for scheduled jobs can be added by Managed Services if the sites performance metrics clearly indicate that there is a heavy scheduled job that affects site visitors experience. Currently, there is no separate price tag for this feature, its a part of the service.
@Sergii - what about on-premises case?
@Johan Petersson, @Valdis according to the license terms each web app instance requires a license, so the scheduler instance requires it too.
Hi @Sergi, Thanks for the blog post giving direction for long running scheduled jobs on DXC. Is there something similar we can do for long running CatalogImportExport.Import method without using scheduled job? We have an external PIM that synchronize the complete catalog with episerver site. And WebApp continously times out because this import takes hours to complete. I was curious, if there is something we can do run this Import on a different dedicated App Service instead?
If Scheduled job is the only way to go, is there a way for us to run specific scheduled jobs on dedicated App Service and some jobs on any webapp?
Regards,
Sujit
@Sujit, scheduled jobs are designed for long-running tasks, so I think it should work for the catalog import as well. Currently, there is no way to select a host to run a specific scheduled job, the only option available is to enable or disable all jobs in the web.config. So once you have an App Service dedicated for scheduled jobs, all jobs will run there.