Ok, time for a big topic. Environment templates is a really useful practice which can improve many Continuous Integration areas, so I will probably split its description over several posts.
This first post involves, obviously, template creation.
The second one will show you how to reuse such template for integration tests automation.
Third one will add some detail over deploy abstraction and management.

But, first of all, WHY would you need environment templates?
Supposing you are already performing CI over your product libraries and applications, you will probably face the problem on how such application have to be deployed over your different middlewares (apache, tomcat, jboss, etc) and on different kind on environments (dev, qa, preprod, prod, etc).

The first step you will probably have to adopt is to create configuration agnostic libraries/applications so that you can use the very same artifact on dev as well on prod environment (and your CM can sleep fine for this!).
Any hardcoded configurations about working directories, IP addresses, users et similia have to be removed from the application itself and delegated elsewhere.

BUT if the application artifact is unaware of its configuration it’s up to the middleware or even to the underlying environment to know all information and to communicate them to the running code.
You may therefore say that you are not solving a problem but just moving it on another point of the chain: this is partially true, since we are centralising environment configuration.
You may have plenty of libraries and web applications running on your environment, you may have also several different middleware talking to each other and managing shared configuration among all of them can be a mess, but you will always have one environment to be deployed at once.
On all these environments we want to run the very same artifact, built just once by our CI chain and propagated with no differences on all relevant installations.

To achieve all of these, you have to extend your CI chain from library and application, to full environments automated builds.

Since we are all Java fans (really?) we will use as usual Maven to create our template and to properly inject all our configurations stuffs.
To make all more clear we will do a very simple example, we will build a Jboss Wildfly template where we will create a parametrized configuration to add a JNDI datasource for our application and be able to change memory allocation.
For sure we want to be able to use different database reference accordingly to the kind of environment we are going to deploy (prod, preprod, qa, test) and also to allocare different RAM size in the same way.
You will see that all we need is already present in any maven distribution, with just few plugin addition for some extra features.

OK, let’s start, first of all you will have to download the standard Wildfly 8.1 archive. You can get if from the official site, a good idea is to store it (with no changes!) also into your internal artifact repository to have it immediately available within your intranet for any successive build and save time and bandwidth.

You should always start creating your template from an official distribution package for a couple of very simple (but extremely useful) reasons.

1. If you start from scratch, your template maven build will contains all the know-how regarding the customization you applied to the official version, so you will never miss a step. If you upload to your artifactory an already-patched version you may forgot why and what you patched in a very close future.
2. Starting from an official version makes upgrade to future minor version easier, since you will have to change only the sections of your build affected by the upgrade, none at all if you are lucky. Moreover, versioning over the build script will preserve backward compatibility if you will have to reproduce an old template for any reason.

While you read my 5 cents on this topic your package download (and re-upload to repo) should be complete, so we can move forward.
Now we must create a new maven project with the following features:

• project must be of pom type
• a src/main/resources folder must be available
• a src/main/filter folder must be available
• our official package is declared as a runtime dependency.

more or less something like this (there are a couple of additional properties I already added to save time, they will come in handy soon):

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
 <groupId>com.mycompany</groupId>
 <artifactId>mycompany-base</artifactId>
 <version>2.7.2-SNAPSHOT</version>
 <relativePath>../pom.xml</relativePath>
 </parent>

 <groupId>com.mycompany</groupId>
 <artifactId>mycompany-jboss-tpl</artifactId>
 <packaging>pom</packaging>

<name>Jboss env template - ${project.groupId}:${project.artifactId}:${project.packaging}</name>

<properties>
 <filter.dir>${project.build.directory}/templates/filters</filter.dir>
 <template.dir>${project.build.directory}/templates</template.dir>
 <final.build.home>${project.build.directory}/templates</final.build.home>
 <jboss.artifact>wildfly</jboss.artifact>
 <jboss.version>8.1.0.Final</jboss.version>
 <mysql.module.path>${template.dir}/modules/system/layers/base/com/mysql/main/</mysql.module.path><jboss.path>wildfly-${jboss.version}</jboss.path>
</properties>

<dependencies>
<!-- ========== base jboss package ========== -->
<dependency>
 <groupId>org.wildfly</groupId>
 <artifactId>wildfly</artifactId>
 <version>${jboss.version}</version>
 <type>zip</type>
 </dependency>
</dependencies>

</project>

PS: check my post on hierarchical maven organization if you are wandering about the <parent> tag.

Now we have to use maven dependency:unpack-depedency plugin to unpack our middleware archive under target/template folder, which will become our temporary working directory. We skip the docs directory since we do not expect to need it in a running environment.

<build>
 <plugins>
 <plugin>
 <artifactId>maven-dependency-plugin</artifactId>
 <executions>
 <execution>
 <id>unpack-jboss</id>
 <phase>generate-resources</phase>
 <goals>
 <goal>unpack-dependencies</goal>
 </goals>
 <configuration>
 <outputDirectory>${template.dir}</outputDirectory>
 <includeArtifactIds>${jboss.artifact}</includeArtifactIds>
 <includeTypes>zip</includeTypes>
 <excludes>**/docs/</excludes>
 </configuration>
 </execution>

Now what we want to do is to add a mysql JNDI resource to the jboss, so we will have to change the standalone/configuration/standalone.xml configuration file.
We want also to change memory setting so the other file to change is bin/standalone.conf file.
So not forget that we also need a mysql connector module, so we will add it under the ${mysql.module.path} we declared in the initial properties.

Since we are starting from the official distribution we will copy both files from the package to our src/main/resources folder preserving the relative path, so they will be:

• src/main/resources/standalone/configuration/standalone.xml
• src/main/resources/bin/standalone.conf

our mysql jboss module will need its path with a module xml file as well:

• src/main/resources/modules/system/layers/base/com/mysql/main/module.xml

Inside our standalone.xml we will add our JNDI connection

<subsystem xmlns="urn:jboss:domain:datasources:2.0">
 <datasources>
 <datasource jndi-name="${myappds.ds.server.jndi.name}" pool-name="${myappds.ds.server.pool.name}" enabled="true" use-java-context="true">
 <connection-url>${myappds.db.url}</connection-url>
 <driver>${db.driver.name}</driver>
 <transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
 <pool>
 <min-pool-size>10</min-pool-size>
 <max-pool-size>30</max-pool-size>
 <prefill>true</prefill>
 </pool>
 <security>
 <user-name>${myappds.db.user}</user-name>
 <password>${myappds.db.password}</password>
 </security>
 <statement>
 <prepared-statement-cache-size>32</prepared-statement-cache-size>
 <share-prepared-statements>true</share-prepared-statements>
 </statement>
 </datasource>
 <drivers>
 <driver name="${db.driver.name}" module="${db.driver.module}" />
 <driver name="h2" module="com.h2database.h2">
 <xa-datasource-class>org.h2.jdbcx.JdbcDataSource</xa-datasource-class>
 </driver>
 </drivers>
 </datasources>
 </subsystem>

While in our standalone.conf we’ll add the memory management:

#
 # Specify options to pass to the Java VM.
 #
 if [ "x$JAVA_OPTS" = "x" ]; then
 JAVA_OPTS="-Xms${myappds.java_opts.xms} -Xmx${myappds.java_opts.xmx} -XX:MaxPermSize=${myappds.java_opts.max_perm_size} -Djava.net.preferIPv4Stack=true"
 JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=$JBOSS_MODULES_SYSTEM_PKGS -Djava.awt.headless=true"
 JAVA_OPTS="$JAVA_OPTS -Djboss.server.default.config=standalone.xml -Dfile.encoding=UTF-8 -Dcom.sun.jersey.server.impl.cdi.lookupExtensionInBeanManager=true"
 JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled"
 else
 echo "JAVA_OPTS already set in environment; overriding default settings with values: $JAVA_OPTS"
 fi

Finally here is our mysql module.xml

<?xml version="1.0" encoding="UTF-8"?>
 <module xmlns="urn:jboss:module:1.0" name="com.mysql">
 <resources>
 <resource-root path="mysql-connector-java-${mysql_driver.version}.jar"/>
 </resources>
 <dependencies>
 <module name="javax.api"/>
 </dependencies>
 </module>

As you can see in all these file we just created the configuration structure, but never hardcoded any deploy-dependend parameter (IPs, Users, Password, etc)
In replace of all of them, we used maven-like placeholders as ${myappds.ds.server.pool.name}
We will manage them in few minutes, now let’s understand what to do with our configuration files.

We have unpacked our original wildfly zip in target/template folder with our first maven plugin, we have replicated the same structure with our customization in our src/main/resources folder, now we have to copy these files in the same folder as the distribution, overwriting the original one.
Maven plugin to perform this is maven-resource-plugin:copy-resources, we must check that it’s execution runs after the first one or our custom file will not replace the original ones, package phase is fine.

Note: proper plugin execution may become a critical issue when modifying templates, mainly over high level of pom inheritance, always check everything is running in the order you are expecting.

<plugin>
   <artifactId>maven-resources-plugin</artifactId>
   <executions>
     <execution>
       <id>copy-local-unfiltered</id>
       <phase>package</phase>
       <goals>
         <goal>copy-resources</goal>
       </goals>
       <configuration>
         <outputDirectory>${template.dir}</outputDirectory>
         <overwrite>true</overwrite>
         <resources>
           <resource>
           <directory>src/main/resources</directory>
           <filtering>false</filtering>
          </resource>
         </resources>
       </configuration>
     </execution>
   </executions>
 <plugin>

We also have to add the mysql jar with proper version in the same folder as the module.xml configuration file.
Since we can reuse the already declared maven-dependeny-plugin, this time with the copy goal, we can just ad a different execution to the previous section:

<execution>
 <id>copy-mysql-modules</id>
 <phase>package</phase>
 <goals>
 <goal>copy</goal>
 </goals>
 <configuration>
 <artifactItems>
 <artifactItem>
 <groupId>mysql</groupId>
 <artifactId>mysql-connector-java</artifactId>
 <version>${mysql_driver.version}</version>
 <type>jar</type>
 <overWrite>true</overWrite>
 <outputDirectory>${mysql.module.path}</outputDirectory>
 </artifactItem>
 </artifactItems>
 </configuration>
 </execution>

Ok, now all our files are in place, we just have to resolve all placeholder with final values providing a way to distinguish different environment.
It’s time to create in src/main/filters folder a new properties file for each profile we want to implement, e.g.

• src/main/filters/local.properties will contains all placeholders resolution for developers local environment
• src/main/filters/devel.properties will contains same resolutions for a development integration environment
• src/main/filters/prod.properties will do the same for production environment
• …

A sample development file ( so devel.properties ) with all placeholders we used in out code above could be:

## Standalone.xml placeholders
db.server=localhost
db.port=3306
db.driver.name=com.mysql
db.driver.module=com.mysql
db.driver.class.name=com.mysql.jdbc.Driver
myappds.db.schema=myAppDb
myappds.db.user=myDbUser
myappds.db.password=myDbPassword
myappds.ds.server.pool.name=myDatasourceName
myappds.ds.server.jndi.name=java:jboss/datasources/${myappds.ds.server.pool.name}
myappds.db.url=jdbc:mysql://${db.server}:${db.port}/${myappds.db.schema}

## Standalone.conf placeholders
myappds.java_opts.xmx=2G
myappds.java_opts.max_perm_size=2G

while its production counterpart prod.properties:

## Standalone.xml placeholders
db.server=192.168.3.10
db.port=3306
db.driver.name=com.mysql
db.driver.module=com.mysql
db.driver.class.name=com.mysql.jdbc.Driver
myappds.db.schema=myAppProductionDb
myappds.db.user=myProductionDbUser
myappds.db.password=myProductionDbPassword
myappds.ds.server.pool.name=myDatasourceName
myappds.ds.server.jndi.name=java:jboss/datasources/${myappds.ds.server.pool.name}
myappds.db.url=jdbc:mysql://${db.server}:${db.port}/${myappds.db.schema}

## Standalone.conf placeholders
myappds.java_opts.xmx=2G
myappds.java_opts.max_perm_size=8G

As you can see there are very few differences (also because we are using a very simple example) but they are typical of the devel/prod scenario we are analyzing.

Now in order to have all these profile files being processed depending on our build scope we have to introduce a new maven profile for each of them:

<profile>
 <id>devel</id>
 <properties>
 <profile.name>devel</profile.name>
 </properties>
 <build>
 <filters>
 <filter>${config.filter.dir}/devel.properties</filter>
 </filters>
 </build>
 </profile>

... add missing ones here, one for each properties files you want to manage.

<profile>
 <id>prod</id>
 <properties>
 <profile.name>prod</profile.name>
 </properties>
 <build>
 <filters>
 <filter>${config.filter.dir}/prod.properties</filter>
 </filters>
 </build>
 </profile>

Last step is to replace all configuration placeholder with the values of our favorite properties file.
That’s why we prepared everything in a temporary working directory called templates: with our last plugin we will copy everything in the final environment location applying resource filtering to all relevant files type:

<plugin>
 <artifactId>maven-resources-plugin</artifactId>
 <!--==== move from templates to final folder applying filters ==== -->
 <executions>
 <execution>
 <id>copy-template-into-final-filtered</id>
 <phase>package</phase>
 <goals>
 <goal>copy-resources</goal>
 </goals>
 <configuration>
 <outputDirectory>${final.build.home}</outputDirectory>
 <includeEmptyDirs>true</includeEmptyDirs>
 <overwrite>false</overwrite>
 <!-- do not filter binaries -->
 <nonFilteredFileExtensions>
 <nonFilteredFileExtension>swf</nonFilteredFileExtension>
 <nonFilteredFileExtension>jar</nonFilteredFileExtension>
 <nonFilteredFileExtension>lpkg</nonFilteredFileExtension>
 <nonFilteredFileExtension>gif</nonFilteredFileExtension>
 <nonFilteredFileExtension>png</nonFilteredFileExtension>
 <nonFilteredFileExtension>jpg</nonFilteredFileExtension>
 <nonFilteredFileExtension>gpg</nonFilteredFileExtension>
 <nonFilteredFileExtension>ttf</nonFilteredFileExtension>
 <nonFilteredFileExtension>pdf</nonFilteredFileExtension>
 </nonFilteredFileExtensions>
 <resources>
 <resource>
 <directory>${config.templates.dir}</directory>
 <filtering>true</filtering>
 <includes>
 <include>**/*.properties</include>
 <include>**/*.xml</include>
 <include>**/*.conf</include>
 <include>**/*.cfg</include>
 <include>**/*.sh</include>
 <include>**/*.yml</include>
 </includes>
 <excludes>
 <exclude>binaries/**</exclude>
 <exclude>data/**</exclude>
 </excludes>
 </resource>
 <resource>
 <directory>${config.templates.dir}</directory>
 <filtering>false</filtering>
 <excludes>
 <exclude>**/*.properties</exclude>
 <exclude>**/*.xml</exclude>
 <exclude>**/*.conf</exclude>
 <exclude>**/*.cfg</exclude>
 <exclude>**/*.sh</exclude>
 <exclude>**/*.yml</exclude>
 </excludes>
 </resource>
 </resources>
 </configuration>
 </execution>
 </executions>
 </plugin>

This plugin is quite verbose since we want to apply resource filtering only to relevant files types, that is the ones containing placeholders.
Applying resource filtering to binaries or other non-text based files will result in corrupted file which will cause havoc on our applications, so be careful.
Obviously the filter file used for resolution will be the one activated by the maven profile declaration!

So a quick recap, if now we are going to submit our CI build with a very simple

mvn clean package -Pdevel

our build will:

1. Download original Wildfly distribution from our internal repository.
2. Unpack it into template folder
3. Overlay and overwrite any template content with the one provided into our src/main/resources folder
4. Copy everything back into the environment folder, replacing all placeholders with values coming from the devel.properties filter from src/main/filters directory

Well, it seem to me we have now a CI-ready build for our jboss middleware, with a totally pre-environment customizable configuration set.
Notice that the result of this build is not something you are expected to store on a maven repository (that’s why we triggered a package and not a deploy lifecycle) but an environment ready to be deployed on your live server, would it be a Qa, Prod or Development one.

Ok, this is enough for now, since this is just a quick sample of the environment template approach you can put in place on your CI system.

In the next posts we will try to extend this quick sample to cover application integration tests and real environment deploying with Ansible.