Mark Hall
Aug 9, 2018
  3612
(7 votes)

Simple Content Synchronization

I recently received a question from a partner trying to implement a requirement for having no live edits on the production server.  I have received this question a few other times so I thought I would see if I could come up with a solution that would work.  This example is a really simple solution and should probably be extended for production use. For example a service bus should be used for the change set and maybe azure logic apps or scheduled job to read the queue and process it.

The first thing I did was create an initialization module for listening to the publish events.  There is some code to check if these events should run.  This appsetting should only be set on the staging server that is to be used for content changes.

using EPiServer.Core;
using EPiServer.Core.Transfer;
using EPiServer.Enterprise;
using EPiServer.Framework;
using EPiServer.Framework.Initialization;
using EPiServer.Logging;
using EPiServer.ServiceLocation;
using Newtonsoft.Json.Linq;
using System;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading;
using System.Threading.Tasks;

namespace EPiServer.Reference.Commerce.Site.Infrastructure.Initialization
{
    [InitializableModule]
    [ModuleDependency(typeof(Web.InitializationModule))]
    public class PublishSync : IInitializableModule
    {
        private ServiceAccessor<IDataExporter> _dataExporterAccessor;
        private IContentLoader _contentLoader;
        private const string SettingPageFiles = "ExportPageFiles";
        private const string SettingRecursively = "ExportRecursively";
        private const string SettingPageLink = "ExporterPageLink";
        private const string SettingIncludeContentTypeDependencies = "ExportIncludeContentTypeDependencies";
        private string _username;
        private string _password;
        private string _stagingUrl;
        private ILogger _logger = LogManager.GetLogger(typeof(PublishSync));

        public void Initialize(InitializationEngine context)
        {
            if (!bool.TryParse(ConfigurationManager.AppSettings["contentStaging:Enabled"], out var stagingEnabled))
            {
                return;
            }

            _stagingUrl = ConfigurationManager.AppSettings["contentStaging:ProductionUrlBase"];
            _username = ConfigurationManager.AppSettings["contentStaging:Username"];
            _password = ConfigurationManager.AppSettings["contentStaging:Password"];
            _dataExporterAccessor = context.Locate.Advanced.GetInstance<ServiceAccessor<IDataExporter>>();
            _contentLoader = context.Locate.Advanced.GetInstance<IContentLoader>();

            var events = context.Locate.Advanced.GetInstance<IContentEvents>();
            events.PublishedContent += Events_PublishedContent;
        }

        public void Uninitialize(InitializationEngine context)
        {
            var events = context.Locate.Advanced.GetInstance<IContentEvents>();
            events.PublishedContent -= Events_PublishedContent;
        }

        private void Events_PublishedContent(object sender, ContentEventArgs e)
        {
            if (e.Content is PageData || e.Content is BlockData || e.Content is MediaData)
            {
                Task.Run(() => ExportItem(e.Content, _dataExporterAccessor(), _contentLoader));
            }
        }

        private async Task ExportItem(IContent content, IDataExporter exporter, IContentLoader contentLoader)
        {
            var exportedFileLocation = Path.GetTempFileName();
            var stream = new FileStream(exportedFileLocation, FileMode.Create, FileAccess.ReadWrite, FileShare.None);
            var settings = new Dictionary<string, object>();
            settings[SettingPageLink] = content.ContentLink;
            settings[SettingRecursively] = false;
            settings[SettingPageFiles] = true;
            settings[SettingIncludeContentTypeDependencies] = true;

            var sourceRoots = new List<ExportSource>();
            sourceRoots.Add(new ExportSource(content.ContentLink, ExportSource.NonRecursive));

            var options = ExportOptions.DefaultOptions;
            options.ExcludeFiles = false;
            options.IncludeReferencedContentTypes = true;

            IContent parent;
            contentLoader.TryGet(content.ParentLink, out parent);

            var state = new ExportState
            {
                Stream = stream,
                Exporter = exporter,
                FileLocation = exportedFileLocation,
                Options = options,
                SourceRoots = sourceRoots,
                Settings = settings,
                Parent = parent?.ContentGuid ?? Guid.Empty
            };

            if (state.Parent == Guid.Empty)
            {
                return;
            }

            try
            {
                exporter.Export(state.Stream, state.SourceRoots, state.Options);
                exporter.Dispose();
                await SendContent(state.FileLocation, state.Parent);
            }
            catch (Exception ex)
            {
                exporter.Abort();
                exporter.Status.Log.Error("Can't export package because: {0}", ex, ex.Message);
            }
        }


        private async Task SendContent(string file, Guid parentId)
        {
            var token = await GetToken();
            if (string.IsNullOrEmpty(token))
            {
                return;
            }
            using (var client = new HttpClient())
            {
                client.BaseAddress = new Uri(_stagingUrl);
                client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
                var content = new MultipartFormDataContent();
                var filestream = new FileStream(file, FileMode.Open);
                content.Add(new StreamContent(filestream), "file", "Import.episerverdata");
                var response = await client.PostAsync($"/episerverapi/import/cms/content/{parentId}", content);
                if (response.StatusCode != HttpStatusCode.OK)
                {
                    _logger.Error(response.Content.ReadAsStringAsync().Result);
                }
            }
        }

        private async Task<string> GetToken()
        {
            using (var client = new HttpClient())
            {
                client.BaseAddress = new Uri(_stagingUrl);
                var fields = new Dictionary<string, string>
                {
                    { "grant_type", "password" },
                    { "username", _username },
                    { "password", _password }
                };
                var response = await client.PostAsync("/episerverapi/token", new FormUrlEncodedContent(fields));
                if (response.StatusCode == HttpStatusCode.OK)
                {
                    var content = response.Content.ReadAsStringAsync().Result;
                    var token = JObject.Parse(content).GetValue("access_token");
                    return token.ToString();
                }
            }
            return null;
        }

        private class ExportState
        {
            public string FileLocation { get; set; }
            public IDataExporter Exporter { get; set; }
            public ExportOptions Options { get; set; }
            public Stream Stream { get; set; }
            public IList<ExportSource> SourceRoots { get; set; }
            public Dictionary<string, object> Settings { get; set; }
            public Guid Parent { get; set; }
        }
    }
}

Next I added an endpoint to be able to update the content with the changeset.  This makes use of some service api classes so a reference to the package will be needed to create the endpoint.  

using EPiServer.Core;
using EPiServer.Enterprise;
using EPiServer.ServiceApi.Configuration;
using EPiServer.ServiceApi.Extensions;
using EPiServer.ServiceApi.Util;
using EPiServer.ServiceLocation;
using EPiServer.Web.Internal;
using System;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using System.Web.Http;
using System.Web.Http.Description;

namespace EPiServer.Reference.Commerce.Site.Features.Rest
{
    public class ImportController : ApiController
    {
        private static readonly ApiCallLogger _logger = new ApiCallLogger(typeof(ImportController));
        private readonly PermanentLinkMapper _permanentLinkMapper;
        private readonly  ServiceAccessor<IDataImporter> _dataImporterAccessor;

        private const string InvalidMediaTypeMessage = "Wrong Media Type";

        public ImportController(PermanentLinkMapper permanentLinkMapper, ServiceAccessor<IDataImporter> dataImporterAccessor)
        {
            _permanentLinkMapper = permanentLinkMapper;
            _dataImporterAccessor = dataImporterAccessor;
        }

        [Route("episerverapi/import/cms/content/{id:guid}", Name = "mh_UpdateContent")]
        [HttpPost]
        [ResponseType(typeof(Guid))]
        [AuthorizePermission(Permissions.GroupName, Permissions.Write)]
        public virtual async Task<IHttpActionResult> PostCmsImport(Guid id)
        {
            if (!Request.Content.IsMimeMultipartContent())
            {
                _logger.Error(InvalidMediaTypeMessage, Request.CreateResponseException(InvalidMediaTypeMessage, HttpStatusCode.UnsupportedMediaType));
                throw Request.CreateResponseException(InvalidMediaTypeMessage, HttpStatusCode.UnsupportedMediaType);
            }
            
            var file = await Request.GetUploadedFile(UploadPaths.IntegrationDataPath);
            var destinationRoot = _permanentLinkMapper.Find(id);
            if (destinationRoot == null)
            {
                return Ok(id);
            }

            var importerOptions = ImportOptions.DefaultOptions;
            importerOptions.KeepIdentity = true;
            importerOptions.ValidateDestination = true;
            importerOptions.EnsureContentNameUniqueness = true;
            importerOptions.IsTest = false;
            var importer = _dataImporterAccessor();
            Security.PrincipalInfo.RecreatePrincipalForThreading();
            var state = new ImporterState
            {
                Destination = destinationRoot.ContentReference,
                Importer = importer,
                Options = importerOptions,
                Stream = new FileStream(file.LocalFileName, FileMode.Open)
            };

            var message = await ImportFileThread(state);
            if (string.IsNullOrEmpty(message))
            {
                return Ok(id);
            }
            else
            {
                throw Request.CreateResponseException(message, HttpStatusCode.InternalServerError);
            }
        }

        private Task<string> ImportFileThread(ImporterState state)
        {
            return Task.Run(() =>
            {
                try
                {
                    state.Importer.Import(state.Stream, state.Destination, state.Options);
                    return "";
                }
                catch (Exception ex)
                {
                    _logger.Error("Can't import data because, ", ex);
                    return ex.StackTrace;
                }
            });
        }
    }

    public class ImporterState
    {
        public ContentReference Destination { get; set; }
        public IDataImporter Importer { get; set; }
        public Stream Stream { get; set; }
        public ImportOptions Options { get; set; }
    }
}
Aug 09, 2018

Comments

Aug 9, 2018 01:00 PM

Nice - mirroing version 3.0 :)

Paul Gruffydd
Paul Gruffydd Aug 9, 2018 03:07 PM

Much as I always disliked mirroring, I like the idea here, particularly if you go with your suggestion of queueing the content and transferring via a separate process.

I was thinking of similar a while back but hit issues where publish events weren't called for scheduled publishes and expiry. Thankfully it looks like that's been resolved now and the PublishedContent event is called in most of the situations where I'd expect it to be. Beyond the publish event it would be good to tie in to the MovedContent event too otherwise, when pages get deleted in thet CMS, the production site wouldn't reflect that. Maybe also the IContentSecurityRepository.ContentSecuritySaved event if ACLs are important.

Good stuff Mark - I suspect I'll be referencing this post in the future.

valdis
valdis Aug 20, 2018 09:52 AM

this is nais approch. but as advocate of dependency injection - I would recommend to extract all appSettings access code from the job and ask for the `IContentSyncOptions` or similar interface to be injected. and then I could configure container to instruct how to create implementation of `IContentSyncOptions` and from where to get config values.

these are just my humble 2 cents :) no offense

Please login to comment.
Latest blogs
Translating Optimizely CMS 12 UI components

Optimizely CMS 12 have been out for a while now, but still some elements haven't been properly translated resulting in a GUI defaulting to english....

Eric Herlitz | Jan 26, 2023 | Syndicated blog

Image preview in Optimizely CMS12 all properties view

With these simple steps, you can now see an Image and its Metadata, including size and dimensions, when editing an Image property in Optimizely...

Tomas Hensrud Gulla | Jan 26, 2023 | Syndicated blog

Setting up the ImageEditor in Optimizely CMS 12

Setting up certain configurations on Opimizely CMS 12 differs quite a bit from prior versions of (Episerver CMS 11 and older). Here's a small guide...

Eric Herlitz | Jan 25, 2023 | Syndicated blog

Happy Hour Returning in February

Hi everyone! It's been a while and we're excited to resume our Happy Hour in February for more learning, sharing, connecting, relaxing, and just to...

Patrick Lam | Jan 24, 2023