OSGi doesn't suck - You're just using it wrong

James Ward published a nice post "Java Doesn't Suck - You're Just Using It Wrong" recently. If you didn't read it yet, please do so first! I completely agree with James and thought it would be fun to see how this applies to the Amdatu stack, and how we used this on different projects in the past few years.

10 Page Wikis to Setup Dev Environments Suck

Even on our largest projects the dev environment setup is just a git clone away. All our projects are based on Bndtools, and a cloned workspace just needs to be imported in Eclipse. Headless builds based on Gradle are supported out of the box for each Bndtools workspace without any additional setup. A freshly cloned workspace contains all the configuration to run on the developer's machine.

Most developers install Mongo manually on their machines, but there's a hosted Mongo cluster for development as well.

For UI development we use tools from the JavaScript ecosystem such as Grunt, and those are installed automatically from the Gradle build.

Incongruent Deployment Environments Suck

Our build server does a fully automated deployment at each merge to master in git. In general master is pretty stable, because all work is done and reviewed on feature branches. Promoting a build to the more stable test server is done by starting a build on Bamboo. This will create a tag in git so that the release is reproducible and does the automated install on the test cluster. The same process applies to promote a build from test to production, it's just a single click on the build server.

When using OSGi you don't actually need an application server. The OSGi framework is part of the application, and you can start the application as an executable jar file. Things like a web server is just another bundle in the application (we use Jetty). On the cluster nodes (in any environment) we don't deploy the application directly either. A clean node starts a very bare bones OSGi application containing a Management Agent. The agent connects to a provisioning server, Apache Ace. The build server deploys bundles to the provisioning server, and when a new cluster node connects, it will receive the latest version of the application's bundles. This makes installations light weight, completely automated and reproducible. The important thing is, everything is completely automated, and there are no servers to maintain. When cluster nodes become unhealthy, a new one will be started automatically (by AWS AutoScaling) and will connect to the provisioning server to install the software. The videos below show more about deployments and Apache ACE.

Deploying with Apache ACE from Luminis Technologies on Vimeo.

Creating runnable JARs from Luminis Technologies on Vimeo.

Servers That Take More Than 30 Seconds to Start Suck

We don't even HAVE an application server to startup, so there's no waiting either... Starting an OSGi framework is extremely light weight (remember it was once designed for embedded environments). The only startup there is, is the actual application code. Even for very large applications (more than 500 bundles) this is a few seconds at max, while most applications start pretty much instantly. 

Besides that, you don't actually restart your application often during development. Using Bndtools you get hot code deployment; each time you compile code, the bundle containing that code is rebuilt and updated in the running framework. This process is so fast that you won't even notice it, coding feels like working in a dynamic language like Groovy or Ruby. 

Manually Managed Dependencies Suck

The very best way to manage dependencies is using bnd (which Bndtools is built on top of). Just like in Maven you declare dependencies by name and version. Because bnd is built for OSGi it understands things like semantic version ranges. You don't need POM files; OSGi bundles already contain all the metadata a POM file normally contains. This means that conceptually dependency management in bnd are not that different, but bnd is a lot easier because it's closer to OSGi. Similar to Maven there are online repositories, and you can host your own repositories.

There is one difference, bnd doesn't do transitive dependencies. Transitive dependencies are dangerous and can cause a lot of trouble. Also, the resolver in the OSGi framework will help you to make sure that all dependencies your application need are installed.

Bnd does integrate very well with Gradle, which we use for headless builds. But again, dependencies are managed by bnd, not by Gradle.

For UI development we often use Bower to manage dependencies. 

Unversioned & Unpublished Libraries Suck

Versioning is an important topic in OSGi. All the tools discussed understand semantic versioning, which helps a lot when checking for compatibility with newer versions of (external) bundles. API's can be baselined automatically, which will force correct semantic versioning of your packages.

baselining from Luminis Technologies on Vimeo.

Long Development / Validation Cycles Really Suck

Of all the things in OSGi, this is probably what I love most; an instant feedback cycle during development. Just check out this video.

Bndtools code reload from Luminis Technologies on Vimeo.

Monolithic Releases Suck

Completely automated deployments are required when releasing often, but as discussed above, all the tools are in place to do this. We deploy multiple times per day to development and often also to test, and multiple times per week to production. This definitely requires a different mind set in the team (and not just the development team), but once everyone got used to it, it's perfect. Fast feedback for the win!

James makes another excellent point about the need for monitoring. When deploying new code all the time, things can always go wrong once in a while. Make sure to monitor for this! We implemented health checks on all our (OSGi) services. The Load Balancer will check these health checks every few minutes. When there's something wrong, we receive notifications immediately. 

Another really useful tool that we use is New Relic. Specially when performance issues occur it gives a lot of helpful information.

Sticky Sessions and Server State Suck

All web user interfaces are based on AngularJS. Pretty much all session state is on the client side and this greatly reduces the need for server side sessions. The only state the server has is related to authentication on the RESTful resources, which is stored in Mongo. This gives horizontal scalability, and makes topics like failover and auto scaling a lot less complex.

Useless Blocking Sucks

Clients use a mix of RESTful web services and Web Sockets, where Web Sockets are used for asynchronous responses from the server. This makes asynchronous communication easy for wherever it applies. Also we use RabbitMQ and RX at several places. This all doesn't have much to do with OSGi, but it works perfectly together. 

The Java Language Kinda Sucks

I did agree with this for a long time. We used a mix of Groovy and Java for a while, but this turns out to introduce quite a few problems. The Groovy tooling in Eclipse is horrible, which is the main reason I'm not so eager to use it in new projects any more. Java got a LOT better with Java 8 as well. Once you go stream() you never go back (or something like that ;-) ). I'm really not sure anymore if polyglot really makes that much sense, even when I do enjoy playing with alternative languages. Groovy and Scala do work without any problem in OSGi, it's just another bundle...