This post is part of the Environment templates thread.

In part one I show how to crate a template environment.

In part two we used such template to re-create environment and run Integration Tests on it

In this last section we will use the same template to produce and deploy a working environment of our purposes.

As stated in the last post, we have to create a separate maven project for out purposes: its name will be mycompany-dellivery, such project will act again as a parent-pom for the final environment project.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

 <parent>
   <groupId>com.mycompany.delivery</groupId>
   <artifactId>mycompany-base</artifactId>
   <version>1.0.0-SNAPSHOT</version>
 </parent>

 <artifactId>mycompany-env-base</artifactId>
 <packaging>pom</packaging>
 <name>mycompany environments base pom - ${project.groupId}:${project.artifactId}:${project.packaging}</name>
 <description>mycompany environment base build</description>

 <properties>
   <middleware>jboss</middleware>
   <final.build.home>${project.build.directory}/${middleware}</final.build.home>
   <config.templates.dir>${project.build.directory}/templates</config.templates.dir>
   <config.filter.dir>${project.build.directory}/filters</config.filter.dir>

   <deploy.home>${deploy.path}</deploy.home>
 </properties>
</project>

OK, so we defined all our base parameters plus some useful properties do define which will be our templates and filters folder (check previous post if you do not remember what I’m referring to) as well al out final build folder.

I’ve add another parameters called deploy.home which is the path where you expect the final environment to be installed. 

Such properties is crucial since it must be used in any placeholder referring to an absolute path for the installation configuration.

The only relevant dependency for our build is for sure our template artifact zipfile.

<dependencies>
 <dependency>
 <groupId>com.mycompany</groupId>
 <artifactId>mycompany-env-template</artifactId>
 <scope>runtime</scope>
 <type>zip</type>
 </dependency>
</dependencies>

Remember that we are creating a parent-pom project, so all application dependencies (ears and wars) are not expected to be added here, but on the leaf project which will be the one providing final applications to be deployed over out environment template.

If, on the other hand, you have some additional “core” dependencies to be always added to the build regardless other dependencies which may be declared in the leaf project, my suggestion is to inject another level of inheritance between mycompany-base and mycompany-delivery in order to separate environment build management from core dependencies management.

This is a very interesting topic since a well-organized dependency inheritance allow very effective releases baseline but, for the moment, it’s out of the scope of this post, so just be aware of the potential issue.

As you may expect the next phase is as usual

1. Unpack template artifact in template folder
2. Overlay with any content available from src/main/resources (such resourced are expected to be present in the leaf project, not here in the parent-pom one)
3. Copy everything back with placeholder resolution via maven profile selection.

We have discussed the 3 point above in details in the two previous post, so I will not replicate all the plugins code here, you should now be template-profiling gurus 😉

The additional step needed to have everything in place is to deploy our application code together with the environment, so since our Wildfly middleware can handle both WARs and EARs, let’s decide that any dependency of such kind added to the leaf project will be considered an application to be deployed and therefore copied to standard${final.build.home}/standalone/deployments location.

<plugin>
 <artifactId>maven-dependency-plugin</artifactId>
 <executions>
   <!-- put any EARs-depenencies in place -->
   <execution>
   <id>jboss-ear-copy</id>
   <phase>package</phase>
   <goals>
     <goal>copy-dependencies</goal>
   </goals>
   <configuration>
     <outputDirectory>${final.build.home}/standalone/deployments</outputDirectory>
     <includeTypes>ear</includeTypes>
     <stripVersion>true</stripVersion>
   </configuration>
 </execution>

 <!-- put any WARs-depenencies in place -->
 <execution>
   <id>jboss-war-copy</id>
   <phase>package</phase>
   <goals>
     <goal>copy-dependencies</goal>
   </goals>
   <configuration>
     <outputDirectory>${final.build.home}/standalone/deployments</outputDirectory>
     <includeTypes>war</includeTypes>
     <stripVersion>true</stripVersion>
   </configuration>
 </execution>

 </executions>
</plugin>

Notice that such application artifact copy takes place in package phase, so after all the template unpack and placeholder resolution has been performed so the destination folder is ${final.build.home} instead of ${config.templates.dir}

Applications to be deployed are surely not involved in placeholders resolution since they are binary code, so it’s useless (or even risky) to copy them in template folder and then back in final one unaffected. Let’s alway try to keep build time and needed space as small as possible.

Ok, now our environment is ready to be deployed, if anyone creates a leaf project with the proper parent, he has only to add any wished application as standard maven dependency to have everything in place, here a short sample of such project:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>

 <parent>
 <groupId>com.mycompany.delivery</groupId>
 <artifactId>mycompany-deplivery</artifactId>
 <version>1.0.0</version>
 </parent>

 <groupId>com.mycompany.env.jboss.myservice</groupId>
 <artifactId>mycompany_myservice_env</artifactId>
 <version>1.0.0-SNAPSHOT</version>

 <packaging>pom</packaging>
 <name>mycompany myservice environment - ${project.groupId}:${project.artifactId}:${project.packaging}</name>
 <description>Packaging jboss myservice project</description>

 <dependencies>

 <dependency>
 <groupId>com.mycompany.myservice</groupId>
 <artifactId>myservice-webapp</artifactId>
 <version>2.0.0</version>
 <type>war</type>
 </dependency>

 <dependency>
 <groupId>com.mycompany.myservice</groupId>
 <artifactId>myservice-core</artifactId>
 <version>1.2.0</version>
 <type>ear</type>
 </dependency>

 </dependencies>

</project>

As you can see the only declaration needed are the war/ear to be deployed and the parent inheritance.
If you defined any core dependency in your parent pom, you could however declare a different artifact version here and maven hierarchy will made your customization win over the default one.

Ok, everything is read on the build side, now we have to deploy our wonderful folder on the desired server.
Since I’m an Ansible (sorry this links to an old italian-only post) fan, we’ll write a couple of playbook to perform this.

The main goal here is not to write a real playbook about our jboss sample, since it would be useless to any other scenario, but to understand how to use Ansible template to inject an additional level of abstraction to our templates and deploy procedures.

So let’s just simplify everything by changing our working sample, let’s forget for a while the jboss template and think just to an apache proxy server configuration.
Having just a configuration file and a docroot folder a simple rsync task would work as a deploy playbook.

- name : Rsynch Env data
 local_action: command rsync --delete -acrtvlE --delete-after {{ deploy_src }} {{ deploy_path }}

We already have maven filters placeholder replacement in our build and this is fine, but this approach can work only with all parameters you already know at build time, what about any parameter you may know only at deploy time?
Servers IPs or database passwords may differ from one server from another depending on your IT policy or maybe IT simply won’t share such information with developers to be stored in a clear-text properties files on a GIT repository.
Another typical usage is for balanced delivery, where the same environment build must be deploy on several server with different base configuration.

For all these scenarios we will replace all runtime-variables with jinja2 syntax which is

{{ variable_name }}

Such syntax is not managed by maven resource filtering, so no replacement will occur at build time.

We will apply this logic to our apache configuration file which will become something like this:

{% if apache_protocol=="https" %}
<VirtualHost {{ apache_host }}:443>
{% else %}
<VirtualHost {{ apache_host }}:80>
{% endif %}

 ServerName ${httpd.servername}
 DocumentRoot ${deploy.path}/httpd/docroot
 ... 
 ProxyPass / http://{{ jboss_host }}:${jboss.connector.http.port}/ ttl=10 max=100 retry=5 timeout=90 
 ProxyPassReverse / http://{{ jboss_host }}:${jboss.connector.http.port}/

</VirtualHost>

As you can see we have a mix of build-time variables like the ServerName , DocumentRoot path and Jboss connection port, which will be configured in our profile-properties files and resolved by maven build while IPs of apache and jboss, as well as whether the configuration must use SSL or not have been delegates to deploy-time variables which will be resolved by ansible.
To do so we will introduce a template task in our playbook to manage all files containing this kind of variables.

- name: Deploy configuration templating
  template: src={{ deploy_src }}/{{item}} dest={{ deploy_path }}/{{item}}
  with_items:
   - conf/httpd.conf
   - another/conf/file2.conf

This will tell your Ansible deploy to overwrite previously synched files with a jinja2-template-resolved version.
Sadly jinja templates file have to be listed by hand but the with_items directive helps in keeping the task code clean.

Now is time to to assign also values to our placeholders and since we want this resolution to happen at host level we decide to put them directly into the Ansible inventory file

[prod-apache]
 192.168.0.10 apache_host=192.168.0.10 jboss_host=192.168.1.10 apache_protocol=https
 192.168.0.20 apache_host=192.168.0.20 jboss_host=192.168.1.20 apache_protocol=https
[prod-jboss]
 192.168.1.10 apache_host=192.168.0.10 jboss_host=192.168.1.10
 192.168.1.20 apache_host=192.168.0.20 jboss_host=192.168.1.20
[prod:children]
 prod-apache
 prod-jboss

[devel-apache]
 192.168.10.10 apache_host=192.168.10.10 jboss_host=192.168.11.10 apache_protocol=http
[devel-jboss]
 192.168.11.10 apache_host=192.168.10.10 jboss_host=192.168.11.10
[devel:children]
 devel-apache
 devel-jboss

As you can see we created an inventory file with two different build profiles, since we are re-using the same approach taken with maven build we want profiles names to match in both maven(build) and ansible(deploy) phases.

Prod inventory group will deploy application on 4 servers, 2 apache and 2 jboss applying https configuration, while devel profile will use just 2 servers without SSL.

So in the second scenario our apache template would be resolved and deployed in this way:

<VirtualHost 192.168.10.10:80>
 ServerName myservername.mycompany.org
 DocumentRoot /homt/mycompany/httpd/docroot
 ... 
 ProxyPass / http://192.168.10.11:8080/ ttl=10 max=100 retry=5 timeout=90 
 ProxyPassReverse / http://192.168.10.11:8080/
</VirtualHost>

While with the very same configuration but the prod profile, the template procedure will generate two different configuration files, one for each server of the prod-apache group and deploy them accordingly all from the same single maven build we run before.

By the way, Ansible provides a very high level of variables resolution and inheritance, so while your deploy project will grow bigger you will probably want to split variables resolution at different levels between  inventory, group/hosts files and command line invocation.
This allow also to split security-sensitive information on different layers, since some of these file could be stored (crypted if needed) on the build machine instead of the git project respository and therefore developers and IT can access them at different levels

This also allow you to centralize all your deploy on a single build server using tools like bamboo or jenkins to trigger your build allowing final users to deploy environment by themselves with controlled degree of customization over parameters.

Ok, let’s do some recap of all we have done up to now:

1. we have created a reusable template for our environments starting from a default distribution and customizing it at will
2. we have used such template to recreate a 100% production equivalent integration test environment and provided developers an easy way to embed and automate it in their work
3. we have used the very same template to create the final deploy environment and provided a easy way to create several projects with different application embedded on them.
4. We provide different level of abstraction in our deploy procedure to separate build-time and deploy-time customizations
5. We organize all the process over 2 simple steps, a build-time managed by Maven and a deploy-time managed by Ansible with a profile naming convention as a trait-d’union between them. All can easily be integrated in a web build orchestrator tool.
6. we are now really happy 😉

With this, I can consider completed this first series of Environment templating posts.
Obviously there are hundred of details and tips & tricks I left behind, but I will probably manage (some of) them in the future with smaller posts on very specific topic.
I think the biggest part of the topic has been explained, I hope to get some reader’s feedback and questions (you surely have a lot of) to decide how to proceed for next in-deep posts, thank you for your patience if you read all of them 😉