Monday, February 22, 2010

WLDF and Spring-generated Custom MBeans

In last week's blog, I mentioned a problem which occurs when trying to configure the WebLogic Diagnostic Framework (WLDF) to reference Spring-generated custom MBeans. Here I will describe how this issue can be addressed in a fairly simple way.

Originally, after deploying a web-app containing a Spring-generated MBean, I had attempted to configure a WLDF module using WebLogic's Admin Console to harvest an attribute of this MBean. However, as shown in the screen-shot below, the WLDF tooling was not detecting this custom MBean type as something that could be monitored.

(click image for larger view)

The Spring-generated custom MBean type is not listed (it should begin with test.management...).

After doing a little digging around, I realised that WebLogic's WLDF documentation at section "Specifying Type Names for WebLogic Server MBeans and Custom MBeans" hints to why this should not work. Specifically it states: "...if the MBean is a ModelMBean and there is no value for the MBean Descriptor field DiagnosticTypeName) then the MBean can't be harvested".

Basically, when you try to define a Collected Metric or Watch Rule in a WLDF Diagnostic Module, WLDF needs to know the MBean's implementation class type for the way WLDF categorises MBeans of interest. Even if we don't use the Admin Console and instead use WLST or hack the XML for the domain config diagnostic module directly, we still have this problem because we have to declare the MBean implementation type.

In Sun's JMX standard, 3 of the 4 possible JMX MBean types (Standard, Dynamic and Open) are implemented directly by a Java class, which WLDF can automatically detect. However, for the 4th type (Model MBean), no direct Java class is used to define the MBean. Instead, the MBean is defined using metadata. As there is no MBean implementation class for WLDF to base its configuration on, it needs a hint for what the implementation class should be assumed to be. Spring-generated custom MBeans are Model MBeans and thus are affected by this issue. The WLDF documentation states that the 'implementation hint' should take the form of an explicitly declared MBean Descriptor field called DiagnosticTypeName.

The left-hand side of the screen-shot below shows the Spring-generated custom MBean, as seen via JConsole. The Model MBean descriptor is missing the WebLogic-specific field and hence the MBean can't be used by WLDF. The right-hand side of the screen-shot shows the generated MBean after the WLDF-required field DiagnosticTypeName has been included (the rest of this blog will show you how to achieve this).

(click image for larger view)

So, for an adequate solution, I really needed a way for Spring to generate MBeans with this field automatically set to an appropriate value. Looking at the documentation for Spring's MBeanExporter (which I described in my last blog entry), I found that MBeanExporter has an optional property called assembler, which, if not defined, defaults to org.springframework.jmx.export.assembler.SimpleReflectiveMBeanInfoAssembler. This default assembler uses reflection to generate Model MBeans by introspecting the simple Java-bean style classes. I didn't really want to lose the power and simplicity of this, but needed some way to ensure that the generated MBeans included the extra descriptor field. Then I hit upon the idea of extending this Spring assembler class and overriding its populateMBeanDescriptor() method to add the extra field after first calling the overridden method to have the other descriptor fields created as normal. So I implemented the following one-off class.
package customjmx;

import javax.management.Descriptor;
import org.springframework.jmx.export.assembler.SimpleReflectiveMBeanInfoAssembler;

public class WLDFAwareReflectiveMBeanInfoAssembler 
                    extends SimpleReflectiveMBeanInfoAssembler {
   private static final String WLDF_MBEAN_TYPE_DESCPTR_KEY = 
                                           "DiagnosticTypeName";
   private static final String NAME_MBEAN_DESCPTR_KEY = "name";
   private static final String MBEAN_KEYNAME_SUFFIX = "MBean";

   @Override
   protected void populateMBeanDescriptor(Descriptor descriptor, 
                           Object managedBean, String beanKey) {
      super.populateMBeanDescriptor(descriptor, managedBean, 
                                                       beanKey);
      descriptor.setField(WLDF_MBEAN_TYPE_DESCPTR_KEY, 
                descriptor.getFieldValue(NAME_MBEAN_DESCPTR_KEY) 
                                        + MBEAN_KEYNAME_SUFFIX);
   }
}
In my example code, I just take the original Spring POJO class's name and add the suffix 'MBean' to come up with a name which I feel best conveys the way the MBean is implemented, for the benefit of WLDF. In my example, the DiagnosticTypeName descriptor field is created for the MBean with a value of test.management.TestManagerBeanMBean. You could easily implement this sub-class code differently to generate a field value using your own convention.

In my Spring bean definition file (WEB-INF/applicationContext.xml) I declared my custom assembler class which extends the Spring default assembler class, and then modified the exporter Spring bean to explicitly reference this assembler, as shown in bold below.
<bean id="assembler" class="customjmx.WLDFAwareReflectiveMBeanInfoAssembler"/>

<bean id="exporter" class="org.springframework.jmx.export.MBeanExporter"
                                                      lazy-init="false">
   <property name="beans">
      <map>
         <entry key="com.test:name=TestMgr" value-ref="testMgrBean"/>
      </map>
   </property>
   <property name="server" ref="jmxServerRuntime"/>
   <property name="assembler" ref="assembler"/>
</bean>

This time when I re-deployed my web-app to WebLogic and used JConsole to view it, the extra DiagnosticTypeName field was present in the MBean (see the right-hand side of the screen-shot above).

I tried again to create a custom Harvested Metric for an attribute in my Spring generated custom MBean, using WebLogic's Admin Console, and this time I was able to find and select my MBean type as shown in the screen-shot below.

(click image for larger view)

On the next page, I was then able to see the available attributes of the MBean type to monitor and choose the 'PropA' one, as shown below.

(click image for larger view)

Finally, I was given the option to select the instance of the MBean to monitor, before pressing finish to save the WLDF module, as shown in the screen-shot below.

(click image for larger view)

Once my WLDF module was activated, I then waited a few minutes before using JConsole to change, dynamically at runtime, the value of PropA on my custom MBean. I then went to the Admin Console | Diagnostics | Log File page, and selected HarvestedDataArchive as the log file, to view the WLDF harvested data. The screen-shot below shows the value of the harvested 'PropA' attribute, taken every 30 seconds, with the new value shown at the base of the page.

(click image for larger view)

In summary, it is possible to use WLDF to monitor Spring-generated custom MBeans as long as a WebLogic-specific descriptor field is defined for the generated MBean. In this blog, I have shown one way to achieve this, using a simple class that only has to be written once and then re-used and applied for every MBean required in a deployed app.


Song for today: Saints Around My Neck by Come

Tuesday, February 16, 2010

Creating Custom MBeans for WebLogic using Spring

In my previous blog I discussed how WebLogic provides some features to better integrate Spring-enabled apps into the app-server, including WebLogic auto-generated MBeans to monitor Spring specific elements of an application. In this blog, I instead focus on Spring's built-in ability to let developers create their own custom MBeans and how these can then be published to WebLogic's Runtime MBean Server rather than the underlying JVM's default Platform MBean Server.

Why would a developer want to create a custom MBean? Well the developer may want to provide a standards-based management capability for his/her deployed application, to enable third-party tools to manage and monitor the application remotely.

Why would the developer want to publish these MBean's to WebLogic's Runtime MBean Server? Well, in addition to being visible to generic remote JMX client tools (eg. Sun's JConsole tool), the MBeans are then easily accessible from WebLogic specific tools such as the WebLogic Scripting Tool (WLST) and even the WebLogic Diagnostic Framework (WLDF). Also, WebLogic's capabilities can be leveraged to secure the custom MBeans when associated with WebLogic's MBean Servers. For example, one could secure access to the MBeans using the t3s protocol with a valid username and password.

A few months ago, Philippe Le Mouel wrote a great article on how to create custom MBeans and register them with WebLogic, using pretty much standard JavaEE code. As you can see from his article, it takes quite a lot of effort and boilerplate Java code to define an MBean and then register it with the server at start-up.

In contrast, Spring makes it really easy to generate an MBean as part of your developed JavaEE app and avoid a lot of this coding effort. The Spring Reference (chapter 20) describes how to do this in detail. In essence, just include a Java-bean style POJO in your JavaEE web-app, like the following, for example:
package test.management;

public class TestManagerBean {
   public String getPropA() {
      return propA;
   }
  
   public void setPropA(String propA) {
      this.propA = propA;
      System.out.println("PropA set to: " + propA);
   }

   public int getPropB() {
      return propB;
   }

   public void setPropB(int propB) {
      this.propB = propB;
      System.out.println("PropB set to: " + propB);
   }
 
   private String propA;
   private int propB;
}
This example provides two properties to be JMX-enabled (one String, one int). Obviously, in a real-world application, the getter and setter code would reach inside the rest of the application's code to obtain data or change settings.

In our application's Spring WEB-INF/applicationContext.xml file, we can then define our Spring bean in the normal way with some initial values to be injected into the bean's two properties, e.g.:
<bean id="testMgrBean" class="test.management.TestManagerBean">
   <property name="propA" value="Some text"/>
   <property name="propB" value="1000"/>
</bean>
The real add-value that Spring then provides, is the ability for Spring to auto-generate the MBean for us, for inclusion in our deployed app, by using Spring's MBeanExporter capability. This is enabled by adding the following definition to our applicationContext.xml file, for example:
<bean id="exporter" class="org.springframework.jmx.export.MBeanExporter"
                                                      lazy-init="false">
   <property name="beans">
      <map>
         <entry key="com.test:name=TestMgr" value-ref="testMgrBean"/>
      </map>
   </property>
</bean>
Now, when you deploy the app to WebLogic, the MBean is generated and registered with an MBean Server. However, by default, the Spring runtime only really knows about the standard Platform MBean server in the underlying JVM that Spring is running on. As a result, upon deployment, Spring registers this generated MBean with the JVM's built-in Platform MBean server only. Spring has no awareness of the possibility to use one of WebLogic's own MBean Servers. We can demonstrate that the JVM's Platform MBean server is currently hosting the deployed custom MBean, by launching the JDK's 'jconsole' tool, from the same machine as the WebLogic is running on, using the following commands:
$ . /opt/oracle/Middleware/wlserver_10.3/server/bin/setWLSEnv.sh
$ jconsole
We then select the local Java process corresponding to WebLogic to connect to - we don't specify a username/password:

(click image for larger view)

We can then traverse the JVM's Platform MBean Server, using JConsole's built-in MBean browser. As you can see below, in the MBean list, we can view the standard JVM java.lang MBeans, like the OperatingSystem MBean and the Memory MBean, together with our custom MBean which has an ObjectName of com.test:name=TestMgr.

(click image for larger view)

We can even click in the the shown field for one of the properties (e.g. PropA), change the value to "Hello World!", press enter and value is changed in the running MBean. If we view the system-output for the WebLogic Server JVM's O.S. process, we will see the following text logged:
   PropA set to: Hello World!
(we coded this println statement in our example POJO earlier)

What we really want to do though, is have this MBean registered with WebLogic's MBean Server, not the JVM's Platform MBean Server. To do this in Java code we'd have to add a lot of JMX code at application start-up and shut-down time to register/un-register our MBean with the WebLogic MBean Server. However, because we're using Spring, its much easier. In our Spring applicationContext.xml file, we simply add a bean definition to tell Spring to use JNDI to locate the WebLogic Runtime MBean Server object at the well known WebLogic JNDI path. Then we modify our exporter bean definition to set an optional server property of Spring's MBeanExporter, giving it the handle to the JMX Server which Spring should export MBeans to. The additions to the Spring bean definition file are shown below in bold:
<bean id="jmxServerRuntime" class="org.springframework.jndi.JndiObjectFactoryBean">
   <property name="jndiName" value="java:comp/env/jmx/runtime"/>
</bean>

<bean id="exporter" class="org.springframework.jmx.export.MBeanExporter"
                                                      lazy-init="false">
   <property name="beans">
      <map>
         <entry key="com.test:name=TestMgr" value-ref="testMgrBean"/>
      </map>
   </property>
   <property name="server" ref="jmxServerRuntime"/>
</bean>
Once re-deployed, we can launch JConsole again, but this time we connect to the WebLogic Runtime MBean Server using T3 to communicate remotely, using the following commands:
$ . /opt/oracle/Middleware/wlserver_10.3/server/bin/setWLSEnv.sh
$ jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:
    $JAVA_HOME/lib/tools.jar:$WL_HOME/server/lib/weblogic.jar   
    -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote
For the connection URL we specify "service:jmx:t3://localhost:7001/jndi/weblogic.management.mbeanservers.runtime" and we provide the WebLogic administrator username/password:

(click image for larger view)

We can now see that our MBean is contained in the same JMX list as all WebLogic's server runtime MBeans.

(click image for larger view)

We can even launch WLST to access our custom MBean by looking up the MBean using the JMX APIs and its ObjectName:
$ /opt/oracle/Middleware/wlserver_10.3/common/bin/wlst.sh
 > connect('weblogic','welcome1','t3://localhost:7001')
 > serverRuntime()
 > myMBean = mbs.getObjectInstance(ObjectName('com.test:name=TestMgr'))
 > print myMBean
test.management.TestManagerBean[com.test:name=TestMgr]
If we want to get or set a property on the custom MBean using JMX in WLST, we can:
 > print mbs.getAttribute(ObjectName('com.test:name=TestMgr'), 'PropA')
Some text
 > mbs.setAttribute(ObjectName('com.test:name=TestMgr'), Attribute('PropA', 'Hello World!'))
 > print mbs.getAttribute(ObjectName('com.test:name=TestMgr'), 'PropA')
Hello World!
Or alternatively, we can use the more convenient WLST commands to navigate and manipulate our custom MBean rather than issuing complicated JMX commands:
 > custom()
 > ls()
drw-   com.test
 > cd('com.test')
 > ls()
drw-   com.test:name=TestMgr
 > cd('com.test:name=TestMgr')
 > ls()
-rw-   PropA        Some text
-rw-   PropB        1000
 > print get('PropA')
Some text
 > set('PropA', 'Hello World!')
 > print get('PropA')
Hello World!
(remember, when in the custom tree we can't use the WLST cmo variable)

In summary, when it comes to creating management interfaces for developed applications hosted on WebLogic, Spring comes into its own, making things much easier and less error prone. Developers can rely on the comfort of a POJO based programming model for development of application management logic in addition to core business logic.

One final word. At the start of this blog entry I stated that by exporting custom MBeans to WebLogic's MBean Servers, we can use WebLogic tools like WLST and WLDF for monitoring these custom MBeans. Whilst this is true for custom MBeans in general, it turns out that for WLDF and Spring generated MBeans specifically, there's a slight hitch which means you can't use WLDF to harvest a Spring-generated custom MBean's properties or define a Watch/Notify for these properties. In my next blog entry, I intend to explain why this problem occurs and highlight a simple and re-usable workaround to address this and thus enable Spring-generated custom MBeans to be fully usable with WLDF.


Song for today: Just Because by Jane's Addiction

Monday, February 1, 2010

WebLogic and Spring

In this blog topic I describe some of the integration points that WebLogic provides for Spring based applications.

I have mixed feelings about Spring. I definitely prefer its primary focus on the Dependency Injection pattern (DI) instead of the more typical JavaEE model of using the Service Locator pattern (think JNDI lookups). If used the right way, both patterns can help promote loosely coupled and easily unit-testable solutions. However, Spring's prescriptive DI approach makes the process of developing loosely-coupled solutions feel more intuitive and natural. Also, Spring offers an easy way to leverage Aspect Oriented Programming (AOP) when needed (and yes I emphasise the word when - excessive sprinkling of aspects can make applications hard to understand, debug and troubleshoot).

On the downside, it sometimes feels like Spring has evolved from a light-weight framework for making J2EE easier, into a vast and competing component model for building enterprise applications. It's not clear to me how much the balance in Spring has shifted from Innovating to Re-inventing, and whether this shift is a good thing or not.

Fortunately, WebLogic makes it pretty easy to deploy Spring based apps, whether your application uses Spring in a just-enough-dependency-injection way or in a use-nearly-every-Spring-feature-under-the-sun way. On most levels it doesn't really matter what version of Spring you use with a particular version of WebLogic. If you have an issue, you use the Oracle Support organisation for help with WebLogic specific problems and any Spring parts to your application are treated just like your own custom code is, from an Oracle Support perspective.

In addition, Oracle actually provides explicit certification for specific versions of Spring running on specific versions of WebLogic. For example, on WLS 10.3.x, Oracle explicitly certifies the use of Spring version 2.5.3 (and any later `double-dot' Spring releases). For the official certification matrix, see the spreadsheet titled System Requirements and Supported Platforms for Oracle WebLogic Server 10.3 on the supported configurations page. As a matter of interest, it's also worth noting that internally* WebLogic uses elements of Spring and its AOP capabilities to implement some of WebLogic's newer JavaEE features like EJB 3.0, by using the Spring Pitchfork codebase under the covers.

* WebLogic prefixes the package names of its internally bundled Spring classes to avoid potential class-loading clash issues with Spring classes bundled in any deployed applications. Application developers can also separately choose to bundle the classes from the Spring Pitchfork project to enable Spring beans to be injected into Servlets and EJBs in their own developed application.
When using an Oracle certified version of Spring with WebLogic, extra integration features are also available to help Spring based applications become first-class WebLogic citizens, in the same way that normal JavaEE applications are. This is described in WebLogic's help documentation on Spring. By including some boilerplate text in your web-app's Manifest.mf file to refer to an optional package and by ensuring WL_HOME/server/lib/weblogic-spring.jar is first deployed to WebLogic as a shared library, the following 3 WebLogic features are automatically enabled:


1. Spring MBeans. WebLogic automatically generates a set of Spring related MBeans, hanging off the normal WebLogic ApplicationRuntimeMBeans, into each server's Runtime Service JMX tree. Examples of these MBeans are SpringRuntimeMBean, SpringApplicationContextRuntimeMBean and SpringTransactionManagerRuntimeMBean. See the WebLogic MBean Reference for more info on these Spring MBeans. The Spring MBeans are read-only and enable administrators to have better visibility into what's going on inside the Spring parts of deployed applications. The screenshot below shows the use of WLST to inspect some of these MBeans. If the Manifest.mf file is not correctly defined, WebLogic does not detect the presence of Spring elements in the application and thus will not generate the Spring MBeans; SpringRuntimeMBean would not appear in the list of child MBeans shown in the screenshot.

(click image for larger view)

2. Spring Console Extension. WebLogic provides an Admin Console extension for Spring to provide administrators with visual tools for monitoring the Spring parts of deployed applications (first navigate to WebLogic Admin Console's Preferences | Extension menu option and and enable spring-console). This Spring console extension is basically a set of pages which are added amongst the normal pages of the standard WebLogic admin console, rather than being a separate console per se. The extension provides a view onto the values of the WebLogic generated Spring MBeans (see point 1). If you navigate to the deployed web-app in the Admin Console, select the Configuration tab, and then select the Spring Framework sub-tab, you will see a read-only view of the contents of the application's Spring application context(s), as shown in the example screenshot below.

(click image for larger view)

In the Deployment's Monitoring tab, if you select the Spring Framework sub-tab, as shown in the example below, you can drill into read-only views of the types and amounts of Spring beans that have currently been created in the deployed application's Spring application context(s). It also lets you view the WebLogic managed transactions that have been initiated via the Spring library code in the deployed application.

(click image for larger view)

By selecting one of the Spring application contexts listed in this table, you will see statistics showing how many beans have been created in the context, what scope they have and their performance metrics, as shown in the screenshot below.

(click image for larger view)

3. WebLogic Injected Spring Beans. During the start-up process for the Spring-enabled web-application (see point 1), WebLogic intercepts the creation of the web-app's normal Spring Application Context (using AOP under the covers), and transparently adds a parent context to the normal context. This parent context is pre-populated with the following 3 WebLogic specific beans, ready to be used by the application:
  • A WebLoigc Transaction Manager bean (ref="transactionManager") which extends org.springframework.transaction.jta.JtaTransactionManager
  • A WebLogic Edit Server MBean Connection bean (ref="editMBeanServerConnection") which implements javax.management.MBeanServerConnection
  • A WebLogic Runtime Server MBean Connection bean (ref="runtimeMBeanServerConnection") which implements javax.management.MBeanServerConnection
This is mainly just a convenience feature for application developers to use so that they can then refer to these WebLogic-specific beans (using the ref ids shown above) and have them injected into application code. For example, we may want to inject a reference to the WebLogic ServerRuntime JMX Server into a piece of our code, to enable the code to then use JMX to inspect the host server's runtime MBeans, using a Spring declaration similar to the following:
<bean id="myTestBean" class="com.acme.MyTestBean">
   <property name="mbeanSvrConn" ref="runtimeMBeanServerConnection"/>
</bean>
The exact set of WebLogic beans injected into the parent application context can be deduced by unzipping the file WL_HOME/wlserver_10.3/server/lib/weblogic-spring.jar and viewing the contents of the file weblogic/spring/beans/SpringServerApplicationContext.xml.

Using the Spring console extension discussed in point 2, we can actually navigate to and view the contents of the parent application context, at runtime, in addition to the normal app context, as shown in the first two screenshots in point 2.

During the application initialisation process, WebLogic also sets the application's default Spring Transaction Manager to be org.springframework.transaction.jta.WebLogicJtaTransactionManager to enable WebLogic's Transaction Manager to always be used for managing JTA transactions initiated in Spring code.

Note: A 4th bean (a WebLogic System Work Manager bean) is also populated in the parent context. However, this bean is meant for internal WebLogic system usage and is not for application developers to use, as it does not implement commonj.work.WorkManager. Developers who do want to utilise WebLogic Work Managers, should just declare the work managers they require in the WebLogic domain's configuration, in the normal way, and then use org.springframework.scheduling.commonj.WorkManagerTaskExecutor in their Spring Bean context XML files, to enable the work managers to be injected into application code.


In summary, in this blog topic I have described some of the ways that WebLogic handles applications that use Spring. Not all the integration points between WebLogic and Spring have been discussed. For example, I have not described how WebLogic can integrate with Spring Security (a.k.a. Acegi), how WebLogic's fast RMI/T3 binary wire protocol can be used for Spring Remoting and I have only mentioned in passing the ability to use Spring Dependency Injection into Servlets and EJBs.


Footnote: Brand new in WebLogic 10.3.2 is un-documented and un-supported tech-preview support for the Service Component Architecture (SCA) using Spring. This tech-preview uses Spring to wire up the POJOs inside components and composites, and to declaratively specify the invocation protocol bindings for these composites. The WebLogic-Spring capabilities discussed here in my blog entry are unrelated to the SCA tech-preview support.


Song for today: Rid of Me by PJ Harvey