Targeting multiple environments and machines - part 1/2
A classic problem, or challenge if you are a glass half-full type of person, in software development is how to target multiple environments. By environments, I really mean different machines. Machines for development, for testing, for staging and for production, e.g.:
In some areas of software development, this is a problem of supporting different hardware setups, but in web development we (most often) only have to worry about different database connection strings, mail server setup, paths to e.g. upload folders, and similar configuration differences on different environments.
Several solutions to this problem have been suggested, but common for all solutions is a goal to automate this process of supporting multiple environments without having to manually figure out which configuration bits that needs to be flipped.
Furthermore, as developers we are used to working by contracts. At a very lowest level we have an unspoken contract with the compiler, making sure that we keep to the rules of our language of choice. Our types and methods too define contracts on what we are allowed to do. Webservices rely heavily on contracts, just as any services in the real world do. Surprisingly though, while our applications and websites more often than not rely on configuration files to keep them running, our configuration files are just flat files. In other words, there is no contract preventing us or at least warning us from making errors that in worst case scenarios could bring everything crashing down around our ears. Solving this part of the problem is hard. While you can build procedures to automatically check that everything looks nice, actually validating the data is a topic worthy of a book in itself.
Automatically building environment specific configurations is nothing new, in fact a tool is built right into Visual Studio 2010.
The Web.config transformation tool in Visual Studio allows you to:
- set attributes
- remove attributes
- delete nodes
- replace nodes
- insert nodes
in your Web.config file. This is great.. as long as all you require is the ability to modify your Web.config file from environment to environment but if either of the following statements are true:
- I keep certain configuration bits outside the Web.config file
- I’m not .NET web developer using Visual Studio
this solution is not for you. Not to mention, this does not ensure that aconfig file for a specific environment is not missing a vital property.
While our work primarily involves ASP.NET web development, the projects we work on require a much high degree of configuration customization than the Web.config transformation tool described above facilitates.
Our goal was to create a configuration framework that would provide us with a high degree of customization and at the same time be statically compiled, preventing us from building and then deploying a bad environment configuration.
We ended up with a solution looking something like the figure below:
In the figure above, the boxes on the left (“Templates” and “Environment specific properties”) are especially interesting.
“Templates”, surprisingly, contains templates. For example, it might contain a Web.config template in which connection string values are not defined. It could also contain a license file template, in which the license key was not defined. Instead of defining the environment specific values, that is connection string and license key values, a uniquely named property is written. This property will then be substituted for an environment specific value by the configuration framework when the project is built.
At build time, we then invoke the configuration framework, which collects all the templates and attempts to populate them with properties from a specific environment configuration. Now, if a property is not found, the build will fail and warn us about the missing property. This is great, since we only have to maintain one configuration file (the template), but can still modify and build it across multiple environments and at the same time ensure that all configuration files contain all required properties.
The figure below illustrates this process:
Hopefully this has piqued your interest. If so, stay tuned for the next post in which we will go into the technical details of our solution and best of all, offer it to you free of charge.
Great post, thanks for sharing!
Have you looked at http://ctt.codeplex.com/?
I let you do web.config transformations on any file, outside Visual Studio, using the same framework. The transformation syntax is well documented by Microsoft, and easy to use. The CTT is the missing piece of the puzzle. I use it in my build scripts from my build-server.
It also allow you to do keyword substitution from the command line.
Just for the record. Web.config transformation works fine outside Visual Studio. Uses MSBuild behind the scenes.
You can also, like Steve mentioned above, use a tool like http://ctt.codeplex.com/ to transform all files not just web.config. This can also be achieved by modifiying the target files that web.config transform uses quite easily (one of the web.config transform MS team member has a blog post about it).
I didn't know of CTT, so thanks for sharing.
CTT definitely solves the transformation part too, but I have to say I'm not overly impressed by the syntax/readability. Also (maybe I'm missing something here so please correct me if I'm wrong) it seems like CTT only supports 1:1 transformations, by which I mean one source file must have one and only one transformation file. Thirdly, it doesn't really solve the problem of static files that don't need to be transformed, but are still different from machine to machine.
I do like that it's based on msbuild though, but it's not really a selling point for me since the solution I describe really is just a Visual Studio project - which of course can be built using msbuild.
@Frederik: My gripe was not aimed at Visual Studio, but the fact that it only allows for a Web.config transformation.