Today I got a very interesting problem to solve: I had to delivery a bunch of libraries to an external developer to let him be able to fully compile a very complex build.
Tasks wasn’t easy since he could not simply connect to our local Artifactory to perform the single maven build he would need to have everything in place and packaging all libraries in a single zip and send it to him was a really error-prone procedure, even taking advantage of maven dependency plugin.
After a small brainstorming we got a really nice idea, I he cannot connect to Artifactory directly, why not providing him Artifactory itself as a Vagrant box?
Let’s do a small step behind: what is Vagrant? It can be considered a wrapper of a virtualization provider (Virtualbox, Vmware) together with some scriptable automatic provisioning tools (Ansible, Chef, Puppets) which allow you to create extensible and portable virtual machine via scripting.
A perfect example of the power of Vagrand will be illustrated in this article.
First of all I had to choose a base box to start from, since Artifactory is provided in a self-extracting RPM package, I looked for a minimal Centos box from the ones available at VagrantBox : centos65-x86_64-20140116.box
Now it was time for some Ansible’s magic.
I created a small YAML script to fully install some needed bash tools (just wget to download needed package), a Java JDK, download Artifactory RPM from the official site, install it as a startup service and finally start it!
Here it comes:
Java jdk install is delegated to a separate YML file since it’s a useful routine I reuse in a lot of other ansible script, here it comes (packages are taken from our internal artifactory to speedup setup time)
Ok, now everything is ready to be used with a Vagrant file to create our portable Artifacoty Box, let’s prepare a Vagrantfile this way:
As you can see I’m basically:
- Telling vagrant I want to use the Centos box and the url where to find it (it will take care to download just at first usage)
- Mapping my virtual box (guest) port 8081 to my own laptop (host) 8081 port so that i can access it via localhost or provide other people access via my IP address
- Telling vagrant to use my Ansible script to provision the machine with all I want.
- Tuning some CPU/memory settings
ok, everything on the launch pad, now its time to run it, just perform a vagrant up and see what happens.
As you can see from the output vagrant brought up the virtual machine, configured the network interface as specified, mapped the ports and finally triggered ansible to run the script which installed and started Artifactory.
If you now open your browser and navigate to http://localhost:8081/artifactory/ you will get Artifactory homepage.
Ok, almost done, now we jut have to populate it with all needed library ( the screenshot refers in fact to the already populated instance since you can see the “839 artifacts” count ).
Before doing so we want to perform a dump of the machine in a clean state to be able to reuse it from scratch at will.
So we will perform a vagrant package –output artifactory.box to have vagrant produce a new .box file containing the full installed server.
now that we have your milestone box backupped we can go on: bring the machine up again with a vagrant up (packaging causes the machine to be turned of for dumping).
You will notice that this time all the Ansible provisioning will not run since it’s performed just at first boot.
You will instead get the following trace.
==> default: VM already provisioned. Run `vagrant provision` or use `–provision` to force it
Now we have to populate the newly created repository with all (and just) the needed libraries.
We also want to exclude any source or javadoc since we are dealing with a closed source project.
We need to perform the following steps:
- configure our official artifactory instance as a remote repository of the boxed one, with proper replication and caching configuration
- prepare a different user maven setting to use our boxed instance as a new mirror, so that all request goes through it instead of the official one (see “Artifactory – repository setup and configuration” post for details)
- configure a different local repository location in global setting to be sure no artifact is taken by our local cache instead of being requested to the remote boxed server.
The url of the remote repository must be the one of your official one, containing all your maven libraries.
We exclude sources and javadoc to avoid source code being available into the box we will redistribute.
Finally we configure advanced setting to be sure that all requested artifact will be stored locally into our box repo.
Our repo is now ready to serve incoming requests and cache them locally.
Next steps are on our own machine, where we want to perform the main artifact build.
First we have to force each request going just to the boxed repo, which will then proxy any other remote repo configured, mainly out official one we just added.
To do so create a copy of your user setting file (usually ~/.m2/settings.xml ) as ~/.m2/settings-vagrant.xml and add the mirror directive pointing to our boxed repo
Now our build is almost ready to run, we just have to be sure nothing is cached on our local maven repository, the dirty way could be simply delete the ~/.m2/repository folder, but this would means that any maven build afterward will have to re-download any library.
A more safe way is to create also a copy of the global maven settings (usually $MAVEN_HOME/conf/settings.xml ) providing a different , empty location for your local repo:
save it as something like $MAVEN_HOME/conf/settings-nolocal.xml
Ok, we are ready to go, now all you have to do is to perform your maven build as usual but telling maven to use the user and global setting file we just created.
mvn clean package –settings ~/.m2/settings-vagrant.xml –global-settings /opt/apache-maven-3.0.5/conf/settings-nolocal.xml
If everything has been done properly you will notice your build re-downloading any needed dependency from the boxed artifactory, while, checking repo log, you will see that each request will be forwarded to the official one and cached locally.
Once the been has been completed your artifactory will be populated with all needed libraries to perform your build.
If you get any error on missing artifact during this phase you probably have to add some more remote repository to you boxed instance to match the ones available on your official one.
Once everything is build you just need to perform a new vagrant package –output artifactory-full.box to dump all your work.
Now you have two boxes, a artifactory-full.box with all your artifact to be delivered to end user and which can be integrated with new libraries at each new build release,and a smaller artifactory.box with a clean installation to be reused every time you need to re-populate your full-box from scratch.
Every time you want to run the new boxes you can simply use a Vagrantfile similar to the one used for creating the box from scratch, simply changing the box name at will:
Now just perform some cleanup with a vagrant destroy to deallocate running virtual instance and keep just you boxes for future reuse.