This post is part of the Environment templates thread, you can find part 1 here.

If you just read the previous post, I expect you to notice that,up to now, I didn’t create a real template, but just an environment build since I’m not reusing anything of my work elsewhere.

That’s the focus of this second post where we’ll try to modify our Jboss Wildfly sample in order to create a real template and reuse it to provide our application developer with an automatic Integration Test environment to be included into their maven build.

Using the same template to perform Integration Test and Environment Builds is really a powerful combo since you are 100% sure that no differences exists between them, giving an higher degree of confidence to your tests.

Before to start we have to do some logic modification to our project, since we have to split it into several submodule:

• first submodule will create the real template and package it as a zip file to be uploaded to our repository for future reuse.
• If you remember our previous post we told that the final environment has not to be deployed but installed on the final server, but this module will be just the environment skeleton, without any placeholder resolution nor deployed application
• Another submodule will act as a parent-pom for application development, it will provide out of the box relationship with the template plus all plugin needed to start the environment and deploy the built artifact on it.
• A third submodule will simulate an application deployment and Integration tests execution. It has the purpose to act as a “Test of the Test” to be sure the full process of building such modules works fine before breaking developers real applications in event of failures.
• Final environment build has to be split on a separate maven project, not a submodule one.
• The reason of this separation is that the parent-pom submodule above has to be deployed before application development build occurs, since you will have to provide with parent artifact to development team. 
• Environment build,on the other hand , must include all application artifacts, so it must take place after a new application version has been provided by dev team and included into the env pom. 

So this will be our sample workspace folder structure:

• mycompany-base first maven project, parent-pom for all development project which DO NOT need integration tests
  • mycompany-jboss-tpl template creation and packaging
  • mycompany-jboss-base parent-pom for all development project which will implement integration tests over the template
  • mycompany-jboss-base-it test project for parent-pom above
• mycompany-delivery second maven project, environment build

mycompany-jboss-tpl is more or less the same project described in part one, you still have to:

• include your official wildfly package as a dependency
• create src/main/resources folder with your customization
• create src/main/filters with placeholder resolution
• unpack package to template folder
• overwrite template content with resources one

ok, time to do some change, in the previous post this was time for placeholder resolution to create the final environment folder, but this time we are creating a real template so we will zip all template and filters folder in a separate artifact and upload it into our repo for later use.

We can use maven assembly plugin to do so, so add the following declaration in your pom file

<plugin>
  <artifactId>maven-assembly-plugin</artifactId>
  <executions>
    <execution>
      <id>assembly-zip</id>
      <phase>package</phase>
      <goals>
        <goal>single</goal>
      </goals>
      <configuration>
        <attach>true</attach>
        <descriptors>
          <descriptor>assembly.xml</descriptor>
        </descriptors>
      </configuration>
    </execution>
  </executions>
</plugin>

And create assembly.xml file at the same level of pom.xml

<assembly>
    <formats>
        <format>zip</format>
        </formats>
    <includeBaseDirectory>false</includeBaseDirectory> 
    <fileSets>
        <fileSet>
            <directory>${project.build.directory}/templates</directory>
            <outputDirectory></outputDirectory>
            <fileMode>0644</fileMode>
        </fileSet>
        <fileSet>
            <directory>${project.build.directory}/filters</directory>
            <outputDirectory></outputDirectory>
            <fileMode>0644</fileMode>
        </fileSet>
</fileSets>
</assembly>

Our template is now packaged and ready to use, let’s move to the juicy part, mycompany-jboss-base

This module is a pom type too, since it will be the parent-pom for all developer projects as well as for our test project mycompany-jboss-base-it.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
  <groupId>com.mycompany</groupId>
  <artifactId>mycompany-base</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <relativePath>../pom.xml</relativePath>
</parent>

<groupId>com.mycompany</groupId>
<artifactId>mycompany-jboss-base</artifactId>
<packaging>pom</packaging>
<name>Jboss projects base pom - ${project.groupId}:${project.artifactId}:${project.packaging}</name>

<properties>
  <!-- These properties controls if liferay must be started for integration tests -->
  <maven.jboss.skip>true</maven.jboss.skip>

  <!-- home folder for intregration test jboss -->
  <myJboss_home>${project.build.directory}/myJbossIT</myJboss_home>
  <myJboss.conf.path>${myJboss_home}/conf</myJboss.conf.path>
  <deploy.home>${myJboss_home}/src/test/resources</deploy.home>

  <!-- Integration test properties -->
  <!-- host to be used for integration tests -->
  <integration-test-server-name>${host.myJboss}</integration-test-server-name>

  <!-- debug port for integration tests jboss -->
  <integration-test-debug-port>8787</integration-test-debug-port>

  <!-- http port for integration tests jboss - dynamic -->
  <integration-test-server-port>${myJboss.socket-binding.http.port}</integration-test-server-port>

  <!-- context of resource under test -->
  <integration-test-application-name>${project.artifactId}</integration-test-application-name>

  <!-- path to invike server-side tests -->
  <integration-test-server-side-rest-service-invoker>rest/testrunner/execute</integration-test-server-side-rest-service-invoker>
  
  <!-- path to deploy cargo webapp -->
  <jboss.deploy>${myJboss_home}/standalone/deployments</jboss.deploy>

  <!-- path and timeout to ping cargo webapp for succesful deploy -->
  <maven.jboss.pingURL>http://${integration-test-server-name}:${integration-test-server-port}/${integration-test-application-name}/index.jsp</maven.jboss.pingURL>
  <maven.jboss.pingTimeout>20000</maven.jboss.pingTimeout>
  
  <!-- path to look for webapp to be deployed by cargo -->
  <maven.jboss.location>${project.build.directory}/${project.artifactId}.${maven.jboss.type}</maven.jboss.location>
  
  <!-- default artifact type to be deployed by cargo -->
  <maven.jboss.type>ear</maven.jboss.type>
  <cargo.jvmargs.mem>-Xmx2G -XX:MaxPermSize=512m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled</cargo.jvmargs.mem>

  <!-- name to be used to deploy artifact on cargo -->
  <cargo.artifactName>${project.artifactId}</cargo.artifactName>
  <maven.jboss.extraparams></maven.jboss.extraparams>

  <!-- where look for gloabl filter properties -->
  <jboss.filters.path>${project.build.directory}/extracts/filters</jboss.filters.path>

  <!-- Override of global local.properties -->
  <db.server>localhost</db.server>
  <deploy.path>${myJboss_home}/standalone/deployments</deploy.path>
</properties>
</project>

I know I’ve put really a lot of properties here, please be patient, it’s just to avoid going back to properties section on every plugin addition, just read the comments for the moment they will become more clear while going on…

Now we have to add the dependency for our brand-new customizable jboss template (be careful, it’s a dependency to mycompany-jboss-tpl this time, not the official package! ) and add a plugin to extract it into our target folder in order to have our target/template and target/filters folder immediately available.

Also in this case we want to grand developer the chance to add customized files to override template ones, since in this case the source directory will be no more src/main/resources (reserved for developer use) we will use src/test/overlay

Also in this case we have to perform the placeholder resolution step, you can do it using one of the already available profiles (I typically reuse local.properties one).

Up to now the pom plugins do not differs much since our previous post:

<plugins>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<!--===== Unpack jboss templates ======== -->
<execution>
<id>unpack-jboss-templates</id>
<phase>generate-test-resources</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/extract/template</outputDirectory>
<artifactItems>
<artifactItem>
<groupId>com.mycompany</groupId>
<artifactId>mycompany-jboss-tpl</artifactId>
<version>${parentversion}</version>
<type>zip</type>
</artifactItem>
</artifactItems>
<skip>${maven.jboss.skip}</skip>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>copy-overlay-into-template-unfiltered</id>
<phase>package</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/extract/template/myJboss</outputDirectory>
<overwrite>true</overwrite>
<resources>
<resource>
<directory>src/test/overlay</directory>
<filtering>false</filtering>
</resource>
</resources>
</configuration>
</execution>
<execution>
<id>copy-template-into-final-filtered</id>
<phase>package</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${myJboss_home}</outputDirectory>
<includeEmptyDirs>true</includeEmptyDirs>
<overwrite>true</overwrite>
<!-- do not filter binaries -->
<nonFilteredFileExtensions>
<nonFilteredFileExtension>swf</nonFilteredFileExtension>
<nonFilteredFileExtension>jar</nonFilteredFileExtension>
<nonFilteredFileExtension>lpkg</nonFilteredFileExtension>
<nonFilteredFileExtension>gif</nonFilteredFileExtension>
<nonFilteredFileExtension>png</nonFilteredFileExtension>
<nonFilteredFileExtension>jpg</nonFilteredFileExtension>
<nonFilteredFileExtension>gpg</nonFilteredFileExtension>
<nonFilteredFileExtension>ttf</nonFilteredFileExtension>
<nonFilteredFileExtension>pdf</nonFilteredFileExtension>
</nonFilteredFileExtensions>
<resources>
<resource>
<directory>${project.build.directory}/extract/template/myJboss</directory>
<filtering>true</filtering>
</resource>
</resources>
<filters>
<filter>${jboss.filters.path}/local.properties</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
<plugins>

Really important: keep in mind that any properties re-declared into your pom will be higher priority than the one declared into the properties files so both this parent-pom project as well as your final developer one are free to use this technique to further customize any parameter they want to override.

Some of them (e.g. a mysql db password which may change on each developer machine) may need an even more volatile solution, like override them via CLI maven declaration or .m2/settings.xml user file. It’s up to you to guide your dev team on the more comfortable solution, just remember them not to commit such configuration in project file.

Ok up to now we have more or less replicated our generate-environment-from template standard procedure:unpack, overlay, resolve placeholder, we are ready to start our environment for tests.

To do so we will use Maven Cargo Plugin which will start the customized jboss, deploy our application, ping it to know it’s ready and finally give us the green flat to start integration tests. Let’s see how to achieve all this!

Cargo plugin itself offers the capability to start a standard application server environment for any purpose, but our focus is to start our very own environment, not a third party one.

Here comes our plugin configuration:

<pluginManagement>
  <plugins>
    <plugin>
      <groupId>org.codehaus.cargo</groupId>
      <artifactId>cargo-maven2-plugin</artifactId>
      <configuration>
        <skip>${maven.jboss.skip}</skip>
        <container>
          <containerId>wildfly8x</containerId>
          <type>installed</type>
          <!-- <log>${myJboss_home}/standalone/log/cargo.log</log> -->
          <!-- <output>${myJboss_home}/standalone/log/container.log</output> -->
          <home>${myJboss_home}</home>
        </container>
        <configuration>
          <type>existing</type>
          <home>${myJboss_home}/standalone</home>
          <properties>
            <cargo.jvmargs> ${cargo.jvmargs.mem} -Dfile.encoding=UTF-8 ${jacoco.agent.it.arg} ${maven.jboss.extraparams}</cargo.jvmargs>
            <cargo.logging>high</cargo.logging>
            <cargo.jboss.management-http.port>${myJboss.management.http.port}</cargo.jboss.management-http.port>
            <cargo.servlet.port>${myJboss.socket-binding.http.port}</cargo.servlet.port>
          </properties>
        </configuration>
        <deployer>
          <type>installed</type>
        </deployer>
      </configuration>
    </plugin>
  </plugins>
</pluginManagement>

Let’s analyze it in details:

• We provide a maven.jboss.skip property to totally turn on/off (default off in properties section) cargo phase, this has to be used together with maven.tests.skip if you want to totally skip test phase for any (bad 🙂 ) reason
• containerId must match one oft the official cargo templates (check documentation link), this is just a reference to let it know which kind of base configuration we are creating.
• type installed tag is the directive to told cargo we will provide him a valid wildfly8x installation instead of using it’s default one
• the 2 commented lines are about log management, if you leave them commented application server log will be displayed in maven standard output/error, if you uncomment them they will go to the separated logfile provided. I personally prefer to have them all together on my CI system build log to ease troubleshooting without having to access a separate file, but all depends on your tools and habits.
• home tag is the base path for the wildfly installation, basically the final path where you stored the placeholder-resolved output.
• in the properties section you will find all additional properties you can pass to the application server as port binding, memory and jvm parameters, etc…
• Most of these parameters are stored as maven properties to allow end-users (this is a parent-pom project always keep this in mind) to override them if needed, to increase memory or do add additional parameters needed by their application.
• For Jacoco section see my post on Sonar integration test coverage.

As you can notice all this configuration section is into a pluginManagement section instead into a plugins one, this is due to the fact that wildfly container can handle both war and ear applications, so in my project I’ve created this section for shared configuration plus a dedicated one for each deployable into two separate profiles with different activation rules.

Since we are just illustrating an example I do not want to mess up things too much, so we will see only the ear scenario, but I kept section separated to just show you this alternative. 

The ear plugin configuration is:

<plugin>
  <groupId>org.codehaus.cargo</groupId>
  <artifactId>cargo-maven2-plugin</artifactId>
  <configuration>
    <deployables>
      <deployable>
        <groupId>${project.groupId}</groupId>
        <artifactId>${project.artifactId}</artifactId>
        <type>${maven.jboss.type}</type>
        <location>${maven.jboss.location}</location>
        <pingURL>${maven.jboss.pingURL}</pingURL>
        <pingTimeout>${maven.jboss.pingTimeout}</pingTimeout>
        <properties>
          <name>${cargo.artifactName}</name>
        </properties>
      </deployable>
    </deployables>
  </configuration>
  <executions>
    <execution>
      <id>jboss-start</id>
      <phase>pre-integration-test</phase>
      <goals>
        <goal>start</goal>
      </goals>
      <configuration>
        <skip>${maven.jboss.skip}</skip>
      </configuration>
    </execution>
    <execution>
      <id>jboss-stop</id>
      <phase>post-integration-test</phase>
      <goals>
        <goal>stop</goal>
      </goals>
      <configuration>
        <skip>${maven.jboss.skip}</skip>
      </configuration>
    </execution>
  </executions>
</plugin>

• deployable section contains all details about the application to be deployed on the container, since this is expected to be the application developer by end user, groupId, artifactId, type and location can all be taken from the project properties itself.
• pingURL and pingTimeout are a really useful configuration since you can tell cargo to ping a custom probe page once it has started the container to check that also the application has successfully tarted before going on with next maven steps. 
• You have to illustrate this step very carefully to developers using your parent-pom, since such page should return a 200 OK only when all application services are ready to undergo integration tests, so ask them to provide something like a servlet url with has dependencies with all main application services. Starting test before all services has properly initialize may lead to false negative test failures.
• finally name property is just needed to know which context from the ear deploy has to be considered for testing.
• executions section is the one which tell maven to initialize our container in pre-integration-test phase, just before triggering failsafe tests, and to turn everything off in post-integration-test when all tests just completed

now everything is in place, we just need to add the failsafe declaration

<plugin>
  <artifactId>maven-failsafe-plugin</artifactId>
  <executions>
    <execution>
      <phase>integration-test</phase>
      <goals>
        <goal>integration-test</goal>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

to have all integration test being fired just after cargo as performed it’s magic.

So, a quick recap before the last step.

Any developer application project, by simply inheriting your parent-pom will automagically:

1. download our mycompany-jboss-tpl zip file with our customized distribution and unpack it
2. overlay it will all files present in src/test/overlay folder
3. copy it in its final destination using profile properties files to do placeholder replacement
4. tell cargo to start the resulting environment 
5. deploy developer ear over our container
6. ping a probe page to wait for the application to be up-and running
7. fire all integration tests
8. shutdown container to free all resources.

Not bad for a maven project with just a parent-pom a not even a plugin declared by developers, isn’t it ? (here you should show developers your very-proud-CM-pocker-face when illustrating all this out-of-the box stuffs 😉 )

But wait! It’s not yet time to go for a win-beer! We wrote a lot of code and configuration, automation is nice but things are getting complicated.

Before delivering such a wonderful piece of art to end users we must be sure it works as expected, and that any future change will not break automation chain: we need to test the integration test!

So here comes mycompany-jboss-base-it pom project: 

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
  <groupId>com.mycompany</groupId>
  <artifactId>mycompany-jboss-base</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <relativePath>../mycompany-jboss-base/pom.xml</relativePath>
</parent>

<artifactId>mycompany-jboss-base-it-ear</artifactId>
<packaging>ear</packaging>
<name>Jboss ear integration test - ${project.groupId}:${project.artifactId}:${project.packaging}</name>
<description>Test project to check IT correctness over Jboss ear</description>

<properties>
  <!-- integration must be run -->
  <maven.jboss.skip>false</maven.jboss.skip>

  <!-- I don't want this project to be uploaded to repo -->
  <maven.deploy.skip>true</maven.deploy.skip>

  <!-- Declare which is the context to be used for ping and testing -->
  <integration-test-application-name>mycompany-jboss-base-it-war</integration-test-application-name>
</properties>

<dependencies>
  <dependency>
    <groupId>com.mycompany</groupId>
    <artifactId>mycompany-jboss-base-it-war</artifactId>
    <version>${project.version}</version>
    <type>war</type>
  </dependency>
</dependencies>

<build>
  <finalName>${project.artifactId}</finalName>
</build>

</project>

As you can see the parent-pom is now our mycompany-jboss-base and project type is ear instead of pom.

We set maven.jboss.skip to false since integration test is exactly what we want to perform.
I’ve added another properties, maven.deploy.skip, which refers to another plugin configuration to prevent this test artifact to be deployed to our internal repo. This is out of scope for our environment template management, just remember you can save some space by skipping deploy/install lifecycle for this kind of self-test project.
Finally integration-test-application-name is configured to point to webapp mycompany-jboss-base-it-war which is basically a test war (in my project is another submodule) with a single “Hello world” API class and a single JSP page invoking such API to produce an html web page.
Such war will be included into our ear and tested by the integration test class included into our project:

public class HelloWorldJspIntegrationTest {
    private static String URL = "http://localhost:8080/index.jsp";
    private static final int TIMEOUT = 5 * 1000;
    @Test
    public void testSayHello() {
        String res = null;
        try {
            System.out.println("Get Page:" + URL);
            res = getPage(URL);
            assertNotNull(res);
            System.out.println("Get Page Result:" +res);
            assertTrue(res.length() >0 );
        } catch (Exception e) {
            Assert.fail(e.getMessage());
        }
    }
    private String getPage(String httpurl) throws IOException {
        ... omissis, utility method to invoke an url and retrieve its content ...
    }
}

My test simply invokes the war JSP page and check its content is non-empty, not very smart but we are not testing the application itself but the automation cycle which will cause the application to answer correctly.

Ok, now now invoking a

mvn clean verify -U

over our mycompany-base project we will:

1. produce a zipped artifact containing our reusable environment template
2. deploy a parent-pom project which sill provide integration test automation with cargo over such template
3. test chat such automation is working with a sample project reproducing final developer job

Now we are still missing our spin-off mycompany-delivery project, but since this other one will focus on environment packaging and delivery we’ll task about this in part three.

Now a small disclaimer: all the code samples I provide in these post are a very simplified version of the problems and their solutions.
I skipped many other aspect like DB management (we had a mysql datasource/module in our template, hadn’t we? ), ports conflicts, etc…
Maybe I will talk about some of these in future posts, but do not expect to have a 100% working template just our of the sample code above.
Moreover environment templating must match your business needs, not mine, so take my samples exactly as they are: samples, a stating point to create your own work over your own issue and needs.
And, last but not least, this approach is my best practice, it fits perfectly with all my current and past needs, but I would be very glad to discuss any alternative, in-deep, to change suggestion to my approach, feel free to comment anytime!

Part 3 will illustrate how to use the same template to mange deploys!