Sunday, October 21, 2007

An OSGi Experience

Last Wednesday, I had a chance to attend the local Java Special Interest Group meeting at the Google Engineering building, here in New York City. This month's topic was the OSGi standard and the technology behind it. The presenters were Dr. Richard Nicholson and David Savage from Paremus.

Unfortunately, while the slides were good and the presenters were clearly very smart, their presentation style left quite a bit to be desired. (Hint: Know your slides, know your audience, don't read from the slides, etc. Credit to my very smart coworker, JF, for those helpful suggestions.)

If you're not familiar with OSGi (which I wasn't, really, until recently), it is basically a framework for structuring applications as loadable bundles which can be dynamically managed. Of course, that's a simplification; I suggest reading the OSGi group's explanation for a better understanding. One person at the meeting who sits on the OSGi board from a major web app server company, with whom I had a chance to talk prior to the presentation, summed it up as class loaders on steroids.

During the presentation, Dr. Nicholson discussed some of the failings of service oriented architectures and how it hadn't really lived up to the hype. Probably true, but what does? It made me wonder how one can judge something like that, but I suppose (and according to my presentation saavy cohort, JF) making a controversial statement during a presentation keeps people paying attention. It worked on me, to be sure.

OSGi, in theory, allows one to build application bundles as independent components and wire them together at runtime. I'm still shady on some of the details around how that wiring takes place, but it's certainly comparable to the idea of software services. As I understood it, OSGi is local, within a JVM. If you want to extend the concept to encompass distributed applications, you need another layer on top of this. During the presentation, Dr. Nicholson demonstrated a product from Paremus called InfiniFlow which he described as an Enterprise Service Fabric. Names aside (because that's a little, shall we say, marketing inspired?), he used 12 Apple Mac Minis running the InfiniFlow agent and his Mac Book Pro to dynamically build a fractal rendering farm. Not too shabby, Doc.

With this, he loaded a number of OSGi bundles into (presumably) some InfiniFlow management application, deployed it to all 12 machines (getting instant feedback the entire time), and showed us that the network (or fabric) was set to render a Mandelbrot set. Kicking off the rendering showed that each Mac Mini was performing calculations and sending the results back to the laptop where pieces of the image were being drawn. That was neat.

What was more impressive is that he, next, loaded a new OSGi bundle and almost immediately, the client had new options for the Julian set. Starting the rendering process showed that the 12 nodes now supported and were executing a new function with no down time, no restarts, and no administrative work (other than the initial loading of the bundles).

The idea behind OSGi is very cool. Dr. Nicholson and Mr. Savage told us that some projects such as Eclipse already use OSGi functionality and that Apache is releasing an OSGi R4 implementation as open source in the form of Felix. Additionally, they said that they have plans to try and push OSGi for other languages, specifically mentioning PHP and others (sorry, don't remember all the details). This could be nice to see, especially in environments where multiple languages are supported in one runtime (like with language independent VMs like Parrot) such that Java OSGi bundles could be loaded in Perl or Ruby. This is where I'd mention the CLR if I were a Windows guy.

Who knows if OSGi will gain the kind of traction they're hoping for, but it does have some major players involved including Eclipse, Apache, BEA, IBM, Interface21 (Spring guys), SAP AG, and many others. Either way, the ideas behind this technology are really exciting, from an architectural point of view for obvious reasons.

As an aside, it's worth considering the rather yucky learning curve to understanding how these bundles are wired together. With systems such as OSGi (or anything like it), it can easily become a debugging nightmare and a maze of abstraction. I'd be remiss if I didn't at least make mention of this aspect. You need smart tools, smart people, and a very clear understanding of this kind of technology before deciding to go down this road. Here's to hoping these guys do it right (and keep it as open source)!

8 comments:

mcculls said...

Hi, we're developing various OSGi tools and bundles over at OPS4J - for example, Pax-Construct which lets you create and manage Maven based OSGi projects:

http://www.ops4j.org/projects/pax/construct

Our development model is very open (think of it as "wiki brought to coding")

David Savage said...

Hi Eric,

Thanks for your comments and glad we were able to further your understanding of OSGi especially with regard to its use in the enterprise.

Firstly to clarify the multi language support is being driven by the OSGi Alliance as a whole vs Paremus in particular - though we will certainly be making use of it and may lend our development expertise to make it happen.

Other languages being considered (by us at least) are Scala and Fortress - basically anything that runs as byte code in a vm is an immediate candidate for bundling. However given the excellent native support in OSGi perhaps other languages will follow after?

Secondly on the learning curve of wiring these bundles together. I admit that there is an initial hurdle in the form of considering the classpath for a bundle as unique to that bundle and defined via the imports that it specifies.

However, I'm not sure it is such a large hurdle and I almost think this is easier to understand than the mess that is hierarchical classloading.

We as developers have been so conditioned to the hierarchical model over the past decade or so that we have forgotten what a tangled mess it leaves us in. Namely the problems associated with managing classpaths at runtime - where how a class is resolved depends on when and where a jar is added to the classpath.

Overall I think the peer classloading model used in OSGi is simpler and easier to understand and develop with.

You are right to say that tools are key though - the benefit of OSGi is that tools are able to introspect the model and provide help to the developer to suggest improvements or locate dependencies. There is a lot of good work going on on the Eclipse project under the banner of the PDE (plugin development environment).

Also for your reference the foundation layer of Infiniflow is available as an open source project called Newton (licensed under GPL) available at http://newton.codecauldron.org. This gives you the container and
provisioning features but security, management and ui functions are held back for the paid for product. Hey we've got to eat too :) But hopefully you should be able to get more of an idea where we're going and may even like to contribute?

Finally, on a personal note I'll do my best in the future to take on board your comments on presentation style. I have to admit I am relatively new to large scale presentations. I'm generally much happier tapping away on a keyboard than standing in front of a crowd - so sometimes my nerves still get
the better of me. Though I know I shouldn't read the slides it feels like something to hold onto when the nerves kick in - though actually its probably just making things worse - relax... Apologies if it got in the way of the information I was trying to get across.

Regards,

David Savage

E. Sammer said...

David:

Thanks so much for your comments! It was really great to get a better understanding of OSGi and I do agree with you that it makes more sense (peer class loading, et al). This is one of those evolutions in the way that we do things that will take some adjusting on all of our parts, but I think will benefit us all in the end.

Also, thanks for the clarification on Infiniflow and its availability. I personally think that releasing the base as OSS and keeping some of the advanced features and tools as pay-for is a great model. It's fantastic to see commercial companies finding ways to take advantage of OSS and the community while still putting food on the table (I like to eat too). :)

I was probably too harsh on your presentation. You did fine. :)

Thanks again and I hope to hear more from Paremus and the OSGi group.

WarpedJavaGuy said...

@David,

> We as developers have been so conditioned to the hierarchical model over the past decade or so that we have forgotten what a tangled mess it leaves us in.

I couldn't agree more with that. I haven't had the luxury of working with OSGi yet, but it does seem like the ultimate solution to the current paradox that is dynamic discovery in the hierarchically class loading JEE world.

@ E. Sammer,

The ability to hot deploy modular bundles that autowire at runtime is just phenomenal and definitely the way of the future. Regarding learning curves, I have to expect that the tools will get better with time.

A great and inspiring post. Thanks :)
I wish I was there for the presentation.

E. Sammer said...

Warped:

I don't know if the NY JavaSIG group has podcasts or the like. I don't think they do, but I don't see why it wouldn't be possible. If you want to poke around the site, you might find something. Usually at least the slides are available along with additional information.

NY JavaSIG

On the other hand, if you're in the New York City, NY area, sign up and come! It's free and there's lots of great people to meet and talk to. I'm the guy not wearing a button up shirt, with the tattoos. :)

Eric said...

I am surprised the presenters didn't mention the current activities at OSGi to extend the standard toward enterprise computing requirements, including a distributed model. It's true you have to be an OSGi Alliance member to participate, but both the Eclipse and Apache open source project folks are participating in the effort.

Eric

AlBlue said...

For a basic bit of background to OSGi, you can have a look at the presentation I gave for JSig:

http://www.eclipsezone.com/files/jsig/

There are the slides, along with some demo code that I narrated.

David Savage said...

Hi All,

Just to confirm we did mention the OSGi Enterprise Expert Group (EEG) and its activities at the talk.

You can read the specifics in the slides available from the NY JSIG website here.

Regards,

Dave