organic thoughts

When Google purchased YouTube back in 2006, the Flash video (FLV) format got an astronomical boots almost overnight. The media format became even more popular, the Flash/Flex pair gained notoriety as a viable platform, Adobe fortified its arsenals in the battle for media dominance. This single act shadowed other players such as Real, Microsoft, and to a certain extent Apple's QuickTime (when was the last time you embedded a Microsoft Media Player on your website).

Today (in TechChruch), it was announced that Google is purchasing On2, the media compression technology company behind many of the most popular media codecs including FLV. This has the potential to change the course of online video technologies once again. One can only imagine some of the implications this purchase will have

• It moves Google higher in the media compression food chain and gives it complete control of the most popular codecs used on the web.
• Google may open source the current proprietary VP6 (on which Flash video is based) & VP7 codec formats that On2 licenses to companies. They may decide to provide needed support for the Ogg Theora format and make it first class citizen in the compression stack.
• With Google's push of HTML5 which includes the notion of built-in video/audio support, one can rightly guess that we will see On2 technologies in Chrome Browser, Chrome OS, and Android and anywhere else Google control (which is wide and far).
• As Ogg (Vorbis and Theora) becomes the open source format of choice for media delivery, Google will wield its influence on the format and direction of media deliver on the web.

Why am I rambling about On2, Ogg, and video codec formats? Well, it all has to do with JavaFX. One of the selling point of JavaFX is portable playback of Video/Audio basedon VP6 (consequently FLV). With the purchase of On2, Google will have a direct control of the destiny of media codec supported by JavaFX. Depending on how Google proceeds, we, in the Java community, may see positive developments for JavaFX and this is how:

• Google open sources the On2 codec stack: big win for JavaFX (to a certain extent Adobe). This will imply that JavaFX applications can take part in the online media revolution with lowered entry points.We probably will see Java bindings to the codec's API's which means building both encoders and players in Java/JavaFX.
• If the stack is open source, this may also mean eventual support on JavaFX mobile platforms.
• Google may make On2 encoding tools available for free: another win for JavaFX! More videos being created with the supported On2 formats means more opportunities for JavaFX developers to create players.

Or, Google may decide to just sit on the technologies use them to further their ambitions in Mobile, desktop OS, and web markets. Only time will tell.

I just updated my netbeans to 6.7.x on the Mac and the difference is vivid.
Even faster start time
Snappier GUI
On the Mac, the new look makes it look native
Plus all the other goodies (Groovy, JavaFX, etc)

For past month, I have been working (on/off) on JmxLogger, a logging API which lets you monitor your Java Logging or Log4J application log events in real-time using JMX. I have had the idea for a while, but decided to finally capture it as project. So, here it is.

http://code.google.com/p/jmx-logger/

JmxLogger
JmxLogger makes it easy to broadcast your Java Logging or your Log4J log events as JMX notifications. As such, you can easily monitor your application's activity logs in real-time. Using familiar logging API and configuration mechanism (i.e. log4j.xml or Java Logging properties file), developers can quickly integrate realtime application log monitoring into their existing deployed code. All it takes is a logging configuration change.

The JmxLogger API provides a Java Logging Handler and a Log4J Appender which can be used to integrate between your favorite logging framework and JMX. JmxLogger hides the complexity of dealing with JMX. You simply configure your logging framework as you normally do, and that's it. You are ready to broadcast your logging events as JMX notification events.

Features

• Support for Java Util Logging API
• Support for the Log4J logging API
• Easy integration with your favorite logging framework
• No coding required, simply configure your logging framework to get started.
• Specify and control the level of event to broadcast
• Monitor event through a console or listener scripts
  
• Realtime monitoring of your application activity logging via JMX.
  

Getting Started

• Download the jar (generic or with log4j support)
• Add jar to your classpath (in case of Log4J, you will need log4j.jar on your classpath)
  
• Configure your favorite logging framework
• Use JMX console (JConsole or VisualVM) to monitor your log actvities.

JmxLogger Events

When you log your application events using either Java Logging or Log4J (see project site for more detail), your log events will get routed to the JmxLogger Java Logging Handler or Log4J Appender defined in your application's logging configuration file.

Below you can see a configuration that sends log messages to the Console and a JmxLogger Logging Handler.

Using a command-line Console you can see your log message messages as they are logged in your application.

The same log events are also emitted as JMX notifications. Using JConsole, you can see these events as they are logged in your application in real-time.

That's it! You have seen how JmxLogger can effortlessly integrate JMX and the two standard logging API's available (Java Logging and Log4J).

In my previous blog, I provided a summarized view of what OSGi is and how it impacts Java developers. I got some interesting feedback. So, in this blog entry, I want to take a detour to examine modularity itself and its importance in creating robust systems that can scale in size and functionality. This blog post does not discuss any specific Java technologies or platform directly. However, it looks at how modular design has affected industries as diverse as manufacturing to software engineering. Future posts will continue to look at OSGi and other modularization technologies.

Modularity
In the Merriam-Webster dictionary, being modular is defined as "... composed of standardized, self-contained units for easy construction and flexible use ..." The notion of modularity is universal and has found usage in process (policy), manufacturing, business, finance, engineering, etc. Those who study the natural sciences have presented arguments about the modularity of nature. Biological studies have shown that the evolution of complex organisms are due in part to functional modularity (composition of smaller independent components with clear bound & functions) and behavioral modularity (having traits and attributes that can evolve independently). To stay closer to the discernible realms, let's examine how contemporary usage of modularity and has affected our modern lives.

Product manufacturers figured out the way to increase output, increase quality, and reduce cost pressures was in modularity. At the turn of last century, industrialist Henry Ford introduced the notion of modularity in his assembly line with standardized and interchangeable automobile parts. He was able to reduce his production cycles and costs to achieve mass production of his Model-T automobiles.

The adoption of modular architecture, not processor speed, has been argued to be the quintessential force behind rapid growth and adoption of computing technologies. Today's hardware, while ever decreasing in size, offers cheap information processing power that has permeated through all industries. Achieving such complex and powerful systems has been possible through the use of modularity. Manufacturers independently create standardized, interchangeable subsystems (microprocessor, memory chips, input/output devices, etc) that can function together to create complex devices. They have been able to build systems with high level of complexity by combining hardware modules (subsystems) into innumerable configurations.

In term of longevity, Unix is arguably one of the most successful operating system today. Created in 1969, Unix has been used in all conceivable computing environment from small embedded to research class supercomputers and everything in between. Unix's success is attributed to its flexibility and depth made possible by its simple, clean, and orthogonal design. As complex an operating system as Unix is, it is composed of simple, distinct, interacting modules (small programs) with clean interfaces that do one thing and does it well (and nothing more).

In the world of Java, there are numerous examples of successful modular designs. One of the earliest and most popular was the JBoss application server. The JBoss designers realized early on the power of modularity. They were one of the first to build a modular architecture on the top of the JMX (though JMX was designed for management, not a general purpose lightweight component server, its standardized API provided the tools needed to build a modular systems). Other examples of well-known, successful modular systems include the Eclipse and the NetBeans desktop platforms. Both frameworks let developers build complex desktop applications (including IDE's) using small pluggable modules that interact at runtime.

Modularity in Software
So, the question remains: how does one develop modular code? In disciplines such as engineering and manufacturing, for example, there is a natural affinity to design modularity along the physical attributes of materials. In software development, however, there are little or no affordances to provide guidance to shape code into standardized modules. As such, developers tend to take the path of least resistance and write software systems organically (i.e. monolithic) where the code produced has little or no clear boundaries. The next few paragraphs review provide some points on creating modular software. If you already create your software as modules great, otherwise, you can use the following text as guidelines for your next projects.

Logical Modularity
Generally in software, modularity can be categorized as logical or physical. Logical Modularity is associated with the internal organization of code into logically-related units. In modern high level languages, logical modularity usually starts with the class, the smallest code group that can be defined. In languages such as Java and C#, classes can be further aggregated into packages (namespace in C#) which allows developers to organize code into group of related classes (languages such as Ruby, Scala, Smalltak, Java7 support similar organizational structure with additional semantics for managed components) .

In the Art of Unix Programming, author Eric S. Raymond provides an iconic rule for modularity that states "... write simple parts connected by clean interfaces." Depending on the context, a module can be implemented as a single class, several classes in a package, or an entire API (a collection of packages). Regardless of the implementation scale of your module, you should be able to describe its functionality in a single sentence (i.e. this module calculates tax per zip code). Your module should expose its functionality as simple interfaces that shield callers from all implementation details. The functionality of a module should be accessible through a published interface that allows the module to expose its functionalities to the outside world while hiding its implementation details (it is way beyond the scope of this blog entry to provide a comprehensive treatments of design patterns and rules for modularity in software).

Physical Modularity
Physical Modularity, is probably the earliest form of modularity introduced in software creation. Physical modularity is comprised of two main components 1) a file that contains compiled code and other resources 2) an executing environment that understand how to execute the file. Developers build and assemble their modules into compiled assets that can be distributed as single or multiple files. In Java for instance, the jar file is the unit of physical modularity for code distribution (.Net has the assembly). The file and its associated meta data is designed to be loaded and executed by the run time environment that understands how run the compiled code.

Physical modularity can also be affected by the context and scale of abstraction. Within Java, for instance, the developer community has created and adopted several physical modularity strategies to address different aspects of enterprise development 1) WAR for web components 2) EJB for distributed enterprise components 3) EAR for enterprise application components 4) vendor specific modules such as JBoss Service Archive (SAR). These are usually a variation of the JAR file format with special meta data to target the intended runtime environment. The current trend of adoption seems to be pointing to OSGi as a generic physical module format (see previous post). OSGi provides the Java environment with additional functionalties that should allow developers to model their modules to scale from small emddeable to complex enterprise components (a lofty goal in deed).

Why Be Modular?
We have seen numerous disciplines have benefited by adopting modularity in process and development efforts. So, why should you invest time in implementing a modular architecture for your next project? After all, adopting a modular design introduces new overhead such as change in design practices, module files management as project grows, design- and build-time dependency management, and each group working on a module has a narrow knowledge of the code. The following is a list of benefits that you gain that outweigh the headaches you may encounter in modular design.

• Scalable Development: a modular design allows a project to be naturally subdivided along the lines of its modules. A developer (or groups of developers) can be assigned a module to implement independently which can produce an asynchronous project flow.
• Testable Code Unit: when your code is partition into functionally-related chunks, it facilitates the testing of each module independently. With the proper testing framework, developers can exercise each module (and its constituencies) without having to bring up the entire project.
  
• Build Robust System: in the monolithic software design, as your system grows in complexity so does its propensity to be brittle (changes in one section causes failure in another). Modularity lets you build complex system composed of smaller parts that can be independently managed and maintained. Fixes in one portion of the code does not necessarily affect the entire system.
• Easier Modification & Maintenance: post-production system maintenance is another crucial benefit of modular design. Developers have the ability to fix and make non-infrastructural changes to module without affecting other modules. The updated module can independently go through the build and release cycle without the need to re-build and redeploy the entire system.
  
• Functionally Scalable: depending on the level of sophistication of your modular design, it's possible to introduce new functionalities with little or no change to existing modules. This allows your software system to scale in functionality without becoming brittle and a burden on developers.
  

Conclusion
In this post, I wanted to provide a broad overview of modularity as a concept and its effect on software system. We have seen how modularity is intrinsically entrenched in non-computing disciplines and how they all benefit from the practice. Software is no different, complex systems from desktop applications to enterprise behemoth such as application servers benefit from modularity in design. It is not the intent of this entry to insinuate that modular design means an end to buggy code. On the contrary, developers will always write buggy code as long there's code to be written. However, it is the intent of this entry to show that modularity is a step toward making software development, deployment, and management easier to scale and maintain.

References
1. Merriam-Webster, http://www.merriam-webster.com/dictionary/Modular
2. Mark Dodgson, David Gann and Ammon Salter, Think, Play, Do: Innovation, Technology, and Organization
4. Carliss Young Baldwin and Kim B. Clark, Design Rules: the Power of Modularity
4. Werner Callebaut, Diego Rasskin-Gutman, Herbert A. Simon, Modularity: understanding the development and evolution of natural complex systems
5. Eric S. Raymond, The Art of Unix Programming, 1999
6. Mark Fleury, Juha Lindfors, Enabling Component Architectures with JMX, http://www.onjava.com/pub/a/onjava/2001/02/01/jmx.html, 2001
7. P. Jalote, An Integrated Approach to Software Engineering, 1997
8. Tim Boudreau, Jaroslav Tulach, Geertjan Wielenga Rich Client Programming: Plugin into the NetBeans Platform

I am excited: the notion of modularity is once again taking center stage in Java land.

All technology platforms (OS's, languages, applications, etc) eventually face the struggle of providing an adequate level of abstraction to encapsulate modular code unit. On the MS Windows platform, for instance, there is myriad of technologies including VCL, VBX, OCX, ActiveX, COM, DLL, etc, that emerged over the years designed to provide modular, sharable, and distributable code units for specific domains.

On the Java platform, the story is similar. From the beginning, the JAR file format provided the means to physically package Java code and resources as modules. The file format has served its original purpose well giving the means to distribute and reuse the large number of Java API's that is available today. However, while the JAR file format works great for physical modularity, it falls short, however, of supporting a logical component model. (NOTE: vendors have developed their own component models to address their specific needs, for instance SpringSource's Spring Framework, JBoss SAR, Geronimo GBean, JMX MBeans, EJB's, etc).

OSGi
You have probably heard about OSGi and wonder how it fits in the Java landscape. In short, OSGi extends JAR's physical modularity into a logical component model (known as bundle). In OSGi, the bundle can be a living entity that provides services, not a mere container for code API. As such, OSGi affords system developers the following capabilities:

• A Framework: This is an execution environment for bundles at run-time. Bundles are loaded, executed, and managed (see OSGi Frameworks below).
  
• Service Registry: OSGi provides a container where bundles are deployed and can expose services to other registered bundles.
  
• Life-cycle Services and Events: Allows addition, removal, and update of bundles dynamically at run-time. The framework publish lifecycle events as notifications that can be captured for appropriate reactions.
  
• Service Model: Ability to expose and discover service interfaces at run-time.
  
• Bundle Isolation: OSGi adheres to strict isolation rules. Each bundle gets its own class loader. Visibility of a bundle is specified at the package level, not class. A Bundles can export its internal packages or indicate its dependency on existing packages hosted by other bundles running in the framework.
  
• Version Dependency: Expressive and flexible bundle versioning semantics with automatic dependency resolution.
  
• Multi-version Coexistence: Multiple versions of same bundle can co-exists without any shadowing effect. Client A can have a dependency on bundle X.1 while client B has a dependency on a different version of bundle X.2 with no risks of class path collisions (commonly known as Jar Hell).
• Dynamic Availability: part of OSGi's component model is the notion of bundle dynamic availability. A bundle is not guaranteed to be available. This is good because you can remove/reload/restart a bundle at runt time (without restarting the framework). This is not so good because your bundle has to have mechanism to handle dynamic availability of bundles it depends on.
  

The OSGi Bundle
Because OSGi was designed (from the ground up) to be a component-based platform, an OSGi application is a collection of interacting bundles running within the framework. As stated earlier, an OSGi bundle is based on the traditional JAR file. However, OSGi uses special meta-data in the META-INF/MANIFEST.MF file of the bundle to specify configuration information used by the OSGi framework at runtime. Below is an example of a MANIFES.MF file for a bundle


Some important items to notice in the manifest meta data:

• Bundle-Activator: specifies a class that will react to bundle life-cycle events (when the bundle starts and when the bundle stops). You can use this event as a starting point for functionalities implemented by your bundle (sort of a main() for your OSGi application).
• Bundle-ClassPath: establishes a classpath for your bundle's internal library of classes (and jar files).
• Import-Package: this entry specifies the name of a package that the bundle depends on (and expected to be available on the platform). Notice the version directive specified in the Import-Package declaration. It uses an interval notation to express dependency on versions 1.0.0 <= X < 2.0.0
  
• Export-Package: declares a package in the bundle that should be made available at runtime. When the bundle is installed in a framework, package will be visible to other installed packages running.

OSGi Frameworks
You may be wondering where/how do I run an OSGi bundle? You will need a framework. A framework is an implementation of the OSGi service platform specifications that has the ability to execute bundles. The framework itself is a small core with system level logic to load bundles. All of the framework's functionalities come from installed bundles on the framework. There are several framework implementations available today, the three most popular (and open source) are:

• Apache Felix - A small embeddable, developer friendly port of OSGi service platform. It's the basis for many popular projects including Glassfish and JOnAS App Servers.
• Eclipse Equinox - This is the base for Eclipse IDE (which a collection of bundles known as plugins in Eclipse parlance). It's used in many popular products such as SpringSource DM Server and IBM's WebSphere.
• Knopflerfish - Knopflerfish is one of the earliest implementations from the commercial side (gone open source) with presence in automotive, telecom, and home automation.

Why Should I Care?
You will certainly wonder how OSGi impacts you. Well, if you spend most of your day writing enterprise applications, then OSGi will mean little to you. As a matter of fact, vendors are working hard to keep developers away from OSGi-specific concerns. You will continue to use enterprise semantics that you are already familiar with and produce enterprise artifacts (EJB, WAR, EAR, etc) that you produce today. Life is good, keep coding!

However, if you are system level developer or simply interested in component-based development, OSGi may be of great value. Already, many vendors are decomposing their products into OSGi-ready jars to allow developers easy deployment on OSGi runtimes. There's also a movement of porting commonly used API's to OSGi by creating a bundle repository (i.e. Felix Commons and Equinox Orbit).

Another approach to OSGi integartion is to embed an OSGi framwork into your own development to create a component model. Eclipse has become the posterchild for this approach using OSGi to provide a plugin architecture for the IDE. Felix, however, has become the favorite embeddable OSGi framework because of its relative small size, easy licensing, and easy API. One great example of embedded Felix is the SIP-Communicator desktop application which uses OSGi-bundles as plugins to extend the application (I am sure there are others).

Conclusion
I hope this entry got you, at least, interested in OSGi. I think OSGi is a great start toward a unified component model for Java. It will certainly get developers to (hopefully) start viewing their systems in terms of components and not just classes. In future posts, I will look at how to use OSGi practically and integrate into your development efforts.

Even if you don't plan to adopt OSGi right away, at least check it out .

References
About OSGi - http://en.wikipedia.org/wiki/OSGi
OSGi Alliance – http://osgi.org/
OSGi Articles - http://www.osgi.org/Links/HomePage
OSGi on InfoQ - http://www.infoq.com/osgi
Free OSGi Book By Neil Barlertt (PDF) - http://neilbartlett.name/downloads/osgibook_preview_20090110.pdf
EclipseCon Presentation by Richard S. Hall (PDF) - http://felix.apache.org/site/presentations.data/osgiapachecon-2005121…
Eclipse Equinox - http://www.eclipse.org/equinox/
Apache Felix - http://felix.apache.org/
Knopflerfish - http://www.knopflerfish.org/
SpringSource OSGi - http://www.springsource.org/osgi
Sip-Communicator Presenation - http://archive.fosdem.org/2007/slides/lightningtalks/SIPcommunicator.pdf