Building bndtools projects with Gradle

Headless builds in bnd and bndtools

Bndtools is by far the easiest way to develop OSGi applications. The extremely fast code-save-test development cycle would be reason enough. If you're new to Bndtools, take a look at this video first to see it in action.

Bndtools always generated ANT build files for each project to support "headless" builds, for example on a build server. While ANT seems to be a long forgotten build tool for some, it actually works quite nicely because the build files are generated and in most cases you don't need to ook at them much. That is until you want to add some custom steps in your build. Think about code coverage tools, JavaScript minification etc.

Selecting a better build tool

To start looking at an alternative build tool we first need to define what we need from a build tool:

• Building bnd projects without additional setup
• Running bnd integration tests
• Easily adding custom build steps/plugins/scripts
• Integration with bnd dependency management

That last step is important. Bnd already manages our build dependencies in the .bnd files; we don't want to repeat this in a build tool specific way! This makes our selection process easier; we are only looking for a build tool, not for a dependency management tool.

The two most obvious candidates in the Java ecosystem are Maven and Gradle. Let's evaluate both.

Maven is both a build tool and a dependency management tool. Dependency Management works well (although it has many flaws) but as a build tool Maven isn't exactly perfect. Building a standard project with Maven is easy, but adding your own build steps is a quite horrible experience. Even the most trivial script would require you to write (or use) a Maven plugin, which requires a lot of boilerplate steps. This is also the reason that many (non-OSGi) projects are currently migrating from Maven to Gradle. We don't need Maven's dependency management. Bnd already does this for us in a much more natural way for OSGi, using OSGi metadata instead of POM files. 

Gradle is a more generic build tool with optionally dependency management similar to Maven. Gradle build files are Groovy scripts with a powerful DSL. It's trivial to add your own build steps and hook into the default life-cycle. Groovy turns out to be an extremely powerful language to write build scripts. Specially when compared to writing ANT scripts in XML or Maven's completely broken plugin development. Another nice bonus is that we can re-use existing ANT plugins directly from Gradle. 

Gradle obviously matches our requirements better, so let's take a look how we can integrate with bnd!

Integrating Gradle with bnd

Bnd is not only a tool (as most people know it), but also has a powerful Java API. Using this API we can perform builds and test runs from code. This is exactly what Bndtools (which is built on top of bnd) is doing as well. Because Gradle build scripts are Groovy code, we can use the bnd API directly from our build script, instead of launching bnd as an external process. 

As an example for this post I have taken the modularity-showcase which is part of our book, and set up a Gradle build. Let's take a look at the build file and walk through it step by step.

Setting up dependencies
Because the build script is using bnd, we need the bnd jar to be on the build script classpath.

Generating settings.gradle
A multi project build in Gradle requires a settings.gradle file, which lists the projects part of of the build. Since each project in the workspace is a bnd project, we can create a task that creates the settings.gradle file for us. This is exactly what the generatesettings task is doing. If you want to exclude certain projects from the build, you could filter those right there.

Building projects
Because our build script is Groovy code, we can use the bnd API directly. Before we can build anything we have to initialise the workspace.

Next we have to build each individual project. Gradle's DSL uses the subprojects syntax to declare tasks that run on each project. For each project we enable the Java plugin; this gives us a Java compiler. Next we configure our source and target directories to match bndtools defaults.

The next step is an important one. We declare that each compileJava task (the default compile task) depends on the bndbuild task of all it's project dependencies. A project dependency is a dependency on another bundle build by another bndtools project.

Next we add junit as a default dependency to the testCompile step, in case someone added junit only as an Eclipse library instead of adding it to the build path. This is common because Eclipse does this automatically.

In the bndbuild step we finally perform the build itself. This is as easy as calling the bndProject.build() method and making sure that errors and warnings are reported correctly.

Testing projects
Bnd's excellent integration testing support can also be used headless, generating junit XML output which can be parsed by most build servers. We can recognise a test project by checking the Test-Cases header in the bnd files. For each test project we simply invoke bndProject.test() which will run the tests and generate the XML files.

There is one thing to be careful with! Do not add an OSGi shell in your integration tests. While this works within the IDE, it will terminate the tests when running in headless mode.

Packaging a release
After performing a headless build it's useful to collect generated jar files and external dependencies, so that we can install them on a server, for example by using Apache ACE. Collecting generated bundles is easy, but how do we collect external dependencies that we need for a deployment?

In bndtools we use .bndrun files to run projects. In these files we specify which bundles (both from the workspace and externally) we want to install. A while ago I created an ANT task in bnd that parses these files and copies all bundles to a directory. From that directory you can easily copy the files to whatever deployment environment you use.

This ANT task is reused in the release task. You could create multiple run configurations and export all of them.

Generating a Gradle wrapper
A convenient Gradle feature is the possibility to generate a wrapper. This will generate a gradlew script (both for Windows and MacOS/Linux systems) that download Gradle and run the build. This way you can run Gradle builds even when a build server doesn't "support" Gradle.

Running the build

Before we can run a build, we fist need to generate settings.gradle. Remember to repeat this when you add new projects to the workspace!

> gradle generatesettings

Now we can run our build. Let's run a clean build and integration test.

> gradle clean bndbuild bndtest

Finally we can collect bundles for a deployment.

> gradle release

Multi project and single project builds

With the build file that we have seen we can run both multi project and single project builds. This is a standard Gradle feature and works for our builds as well. For a single project build you simply invoke gradle bndbuild within the project's directory instead of the workspace level.

Gradle builds in the real world

Setting up a build for a small project is always easy. How does this scale to large projects? For the last few months the PulseOn project that I'm working on has been running on Gradle builds. To give you an idea of size, we generate ~300 bundles and run ~1500 integration tests each build. This runs on a Bamboo server for each feature branch in Git. Our code base includes both Java and Groovy code.

Our builds also include code coverage metrics based on Clover, JavaScript optimisation using Require.js and the Closure compiler and Sonar metrics. This shows that Gradle works perfectly together with bnd to perform large, real-life builds.