New release of DomainHealth - 1.0

I've just released a new version of DomainHealth, that by virtue of being the next increment after 0.9, means that this is the grand 1.0 release! No great fanfare or massive new features but this should [hopefully] be a nice stable release to rely on and live up to its 1.0 billing! :D

You can download DomainHealth 1.0 from here: http://sourceforge.net/projects/domainhealth

One new feature in 1.0 that is worth highlighting though, is the new optional capability to collect and show Processor, Memory and Network statistics from the underlying host Operating System and Machine that WebLogic is running on. DomainHealth only enables this feature if you've also deployed another small open source JEE application that I've created, called WLHostMachineStats. Below is a screenshot of DomainHealth 1.0 in action, displaying graphs of some of these host machine statistics (in this case it's running on an Exalogic system).

(click image for larger view)

WLHostMachineStats is a small agent (a JMX MBean deployed as a WAR file) that runs in every WebLogic Server in a WebLogic domain. It is used to retrieve OS data from the underlying machine hosting each WebLogic Server instance. For more information, including deployment instructions, and to download it, go to: http://sourceforge.net/projects/wlhostmchnstats

Here's another screenshot, just for fun:

(click image for larger view)

Some things to bear in mind....

...the WLHostMachineStats project is still in its infancy and currently places restrictions on what specific environments are supported. Right now, WLHostMachineStats can only be used for WebLogic domains running on Linux Intel (x86) 64-bit based machines (including Exalogic) and only for versions 10.3.0 or greater of WebLogic. This is partly because WLHostMachineStats relies on the SIGAR open source utility, that uses native C libraries and JNI. I hope to widen the list of supported platforms for WLHostMachineStats in the future.

Song for today: Dynamite Steps by The Twilight Singers

New release of DomainHealth (v0.9.1)

I've just released a new version of DomainHealth (version 0.9.1). This is primarily a maintenance/bug-fix release.

DomainHealth is an open source "zero-config" monitoring tool for WebLogic. It collects important server metrics over time, archives these into CSV files and provides a simple web interface for viewing graphs of current and historical statistics. It also works nicely on Exalogic.

To download (and see release notes) go to the project home (select 'files' menu option) at: http://sourceforge.net/projects/domainhealth/

For DomainHealth help docs see: http://sourceforge.net/apps/mediawiki/domainhealth/index.php

Song for today: Ascension Day by Talk Talk

New release of DomainHealth WebLogic monitoring tool

I've just released the latest version of DomainHealth - version 0.9.

DomainHealth is an open source "zero-config" monitoring tool for WebLogic. It collects important server metrics over time, archives these into CSV files and provides a simple web interface for viewing graphs of current and historical statistics.

(click image for larger view)

This release includes a new look and feel and a new page navigation mechanism (based on some contributions from Alain Gregoire). Other new features included are Web-App monitoring, EJB monitoring and various minor improvements and tweaks. For a full list of changes see the Release Notes document listed alongside the DomainHealth download files.

You can download DH from the project home page at http://sourceforge.net/projects/domainhealth.

The help docs for DH are at http://sourceforge.net/apps/mediawiki/domainhealth.

I'd like to say a big thank you to Alain Gregoire for his valuable design and code contributions to this version of DomainHealth.


Song for today: Lets Go For A Ride by Cracker

Deciding to use DomainHealth

DomainHealth (DH) is an open source "zero-config" monitoring tool for WebLogic. It collects important server metrics over time, archives these into CSV files and provides a simple web interface for viewing graphs of current and historical statistics.

So when should you consider using DomainHealth?

When you don't have a full monitoring solution in place already for your WebLogic environment. Ideally an organisation will have standardised on using Oracle Enterprise Manager or a 3rd party management tool, for its comprehensive monitoring needs. Oracle Enterprise Manager also caters for various management requirements in addition to monitoring, which DH does not. The scope of DH's monitoring capabilities is purely focussed on tracking current and historic usage of certain key server resources. DH is not applicable for profiling your deployed applications (instead see Oracle AD4J) or intensively monitoring your JVM (instead see Oracle JRockit Mission Control). However, some organisations using Enterprise Manager may find that DH acts as a convenient and complementary addition to their monitoring solution.

The most comparable tool to DomainHealth is the WebLogic Diagnostic Framework Console Extension which ships as standard with WebLogic. I use both of these tools in my day to day work, where different situations and requirements dictate the use of one over the other.

I use DomainHealth (with its built-in WLDF harvesting mechanism) rather than the WLDF Console Extension, in situations where some of the following factors are advantageous:

1. Zero-configuration. An administrator does not have to first work out and configure the server objects to monitor. An administrator does not have to spend time creating a complex WLDF module with the correct object types, names and attributes in it. An administrator does not have to work out what graphs to plot and then configure specific graphs for each server in the domain for every new environment (eg. Test, Pre-Prod, Prod1, Prod2).
2. Minimal performance impact on Managed Servers. Obtains a set of statistics once and ONLY once, regardless of how many times you come back to view the same statistics in the graphs. The background statistics collection work is driven from the admin server, once per minute, lasting far less than a second.
3. Tools friendly storage of statistics in CSV files. Administrators can open the CSVs in MS Excel or Open Office for off-line analysis and graphing. Using CSV files rather than WebLogic Persistent File Stores on the admin server has no detrimental performance impact. It doesn't matter if it takes 10 microseconds or 100 milliseconds to persist the set of statistics - timeliness only has to be to the nearest minute. The file I/O for writing data to CSV files on the admin server is not in the 'flight-path' of transactions that happen to be racing through the managed servers.
4. Minimal administrator workstation pre-requisites. Doesn't require Java Applet support on the administrator's workstation; it's browser-friendly and just uses very simple HTML and PNG images to display graphs.
5. Hot deployable. Deployable to an already running domain for diagnosis of currently occurring problems, without needing to restart the admin server.
6. Statistics don't constantly scroll whilst trying to analyse them. Administrators can focus in on the current window of time or an historic window of time, in a graph, without it continuously refreshing and moving to a later time. A simple set of navigation buttons is provided to move backwards or forwards in time or just go to the most current time.
7. Statistics can be viewed for non-running Managed Servers. If a managed server has has just died, graphs of its recent statistics can still be viewed to help diagnose the failure cause, without first requiring the managed server to be recovered and re-started.

I use the WLDF Console Extension rather than DomainHealth when some of the following factors are advantageous:

1. Infinitely configurable. Administrators get to choose exactly what server resources they want to monitor.
2. Fine-grained statistics capture. Statistics are gathered and displayed at a much higher frequency than just once every minute.
3. Shipped as part of WebLogic. No need for an administrator to seek corporate approval to download and provision a 3rd party open source application into the organisation's WebLogic environment.
4. Statistics can be retrieved for the periods of time when the Admin Server was down. As long as an administrator has previously configured a WLDF module with the right harvested objects and attributes, statistics can still be retrieved retrospectively, by the Console's graphs, following a period of time when the admin server was down and unable to contact the managed servers.

So if DomainHealth sounds like it would be useful, give it a try and let me know your feedback in the forums provided on the project's site (you have to first click the Develop menu option for some reason that only SourceForge knows!).

DomainHealth project home page: http://sourceforge.net/projects/domainhealth

DomainHealth help documentation: http://sourceforge.net/apps/mediawiki/domainhealth


Song for today: Gimme Shelter by The Rolling Stones

New DomainHealth WebLogic monitoring tool version

I've just released the latest version of DomainHealth - version 0.8 (well actually 0.8.1 because of a fix for a last minute bug spotted by Kris).

DomainHealth (DH) is an open source "zero-config" monitoring tool for WebLogic. It collects important server metrics over time, archives these into CSV files and provides a simple web interface for viewing graphs of current and historical statistics.

You can download it from the project home page at http://sourceforge.net/projects/domainhealth.

The help docs for DomainHealth are at: http://sourceforge.net/apps/mediawiki/domainhealth.


This release includes many minor fixes and enhancements (see the Release Notes document listed alongside the DH download file), plus the following major additions:

• Now provides the ability to harvest and retrieve server statistics using a WLDF module (configured on the fly by DH), rather than using JMX to poll each server for statistics. This is now the default behaviour when running on WLS version 10.3.x or greater. For WLS versions 9.0 to 10.0.x, it still uses JMX Polling. If you prefer to use JMX Polling for the recent WLS versions, you can force this behaviour with a special parameter (see the Help docs). It is worth noting that, although I don't believe the load that the periodic JMX Polling puts on the managed servers (once a minute), is noticeable, I was still keen for DH to move to use WLDF by default. This way, DH acts as a WebLogic 'good citizen' and is also able to better cope with the increased number of MBean statistics that inevitably come with each new DH release.
• Now shows a lot more interesting Thread Pool statistics on the main page (including Throughput and QueueLength).
• Previously, for domains with many servers, it was difficult to drill into the statistics for just one specific server at a time, in the graphical web pages. Now you have the option to select which server you want to see on the web page, in isolation, by selecting the server's name from a drop down list.
• When using the WLDF based mechanism for collecting metrics, statistics for all Work Managers and all Server Channels (protocol server sockets) are now also retrieved and displayed. I have not added this capability for the JMX Polling based mechanism because I'm wary of putting too much load on each managed server during the polling process (I may revisit this decision at a later date).

Song for today: Touched by VAST

Domain Health tool Vs WLDF for monitoring

[Originally posted on my old BEA Dev2Dev blog on May 8, 2008]

Note added 11-Dec-2009: The content of this blog entry is generally superseded by a newer blog entry which addresses the differences since the introduction of the WLDF harvesting capability to DomainHealth.

DomainHealth is an open source "zero-config" monitoring tool for BEA's WebLogic Application Server. It collects important server runtime statistics over time for all servers in a domain, archives this data into CSV files and provides a simple Web Browser based interface for viewing graphs of statistics, historically.

Version 0.7 has just been released and is available to download and use on WebLogic 9.x/10.x from the project's home page at: http://sourceforge.net/projects/domainhealth

Some people have asked why I have created the DomainHealth tool when WebLogic already provides the WebLogic Diagnostic Framework (WLDF). This is a good question which I will attempt to answer below.

First of all it is worth considering that monitoring can be divided into many sub-categories, three of which are:

1. Alerting of a potential issue which is about to occur or has just occurred (eg. not much memory left) enabling the administrator to take pro-active or remedial action
2. Real-time statistics capture and viewing, so an administrator can instantly gauge the health of his/her applications and systems
3. Continuous harvesting of application and system data over time to enable administrators to retrospectively analyse trends (eg. to identify potential tunings), and to diagnose the cause of fatal problems after the problem has occurred.

...plus many others like instrumenting, profiling, heartbeating.

WLDF in combination with the "WLDF Console Extension" provides the base technology to enable administrators to configure a set of WebLogic servers and environment which caters for all 3 monitoring capabilities listed above.

The philosophy of DomainHealth is "zero-config" and as such is intended for a certain type of user base for certain types of monitoring needs (ie. mainly point (3) and to a lesser extent point (2) above). For users who require more complex customisable monitoring capabilities, the WebLogic Diagnostic Framework (WLDF) is more likely to provide the capabilities desired. In some environments, a combination of using WLDF for (1) and DomainHealth for (2) and (3) may also be an option to consider.

Here's a list of benefits which I see for using Domain Health and for using WLDF and the WLDF Console Extension....

Domain Health (DH) benefits:

• DH is easier to install (hot deploy of WAR to admin server) than WLDF which requires copying a JAR file to a specific path and then re-starting the admin server. The WLDF deployment process may be problematic when needing to retrospectively add monitoring capabilities to a production system which has already gone live.
• DH in most case requires no configuration. WLDF console has some built-in views which helps shorten the number of configuration steps likely to be required, However it is unlikely that WLDF's built-in views will be sufficient on their own. By default, the WLDF Console shows data cached on the live servers. For the WLDF Console to show historic data, the administrator must first manually configure (or more likely script the configuration) of a WLDF harvester, separately, to enable historical data to then be retrieved and shown in the console.
• For current data, WLDF Console periodically polls the cache for live data and for historic data has to contact each server's WLDF Data Accessor to retrieve data from each server's local WLDF archive files. Overall, the performance impact of this process is likely to negate some of the performance gain, from not needing to poll servers remotely to harvest their properties in the first place.
• The CSV output of DH is more friendly to generic third-party tools than WLDF is. For offline retrieval of WLDF archived data for use in such tools, scripting will be required to integrate Jython/WLST with the third-party tools.
• DH's web-based visual display is more lightweight than the WLDF console extension for administrators, requiring just a simple browser rather than also needing a Java browser plugin to be installed.

WLDF (inc WLDF Console) benefits:

• WLDF is infinitely configurable allowing administrators to track the exact server statistics that they are interested in. For DH, you are stuck with what has been prescribed as a suitable set of statistics.
• A WLDF thread runs within each managed server for property retrieval, rather than requiring remote polling from an admin server. This may lessen any potential performance impact of performing monitoring in a live environment (however this gain is at least partly lost when using the WLDF console running on the Admin Server to then view data - this requires each server to be contacted to retrieve its locally archived statistics).
• WLDF and the WLDF console are far more powerful and extensive in terms of the overall monitoring features that they provide (eg. provides visualisation for instrumentation of running code).

It is also worth noting that DomainHealth is not a replacement for a 'fully-fledged' 'off-the-shelf' Server monitoring and management tool. Such tools are usually more capable and able to capture other statistics (eg. hardware, OS and network stats) in addition, to enable a more holistic picture of the health of a server environment to be captured. However, in cases where budget does not stretch to purchase an off-the-shelf solution for production environments or where a quick and easy tool for monitoring performance in performance test environments, is required, DomainHealth may help to fill the gap.

In summary what I would like to say is that DomainHealth is not intended as a alternative to WLDF and has been developed specifically for certain use cases for a certain subset of clients which I have dealt with over the years (and is indeed being used by some of these clients today).

Soundtrack for today: Nearly Lost You by Screaming Trees

New Open Source WebLogic Monitoring tool

[Originally posted on my old BEA Dev2Dev blog on March 27, 2008]

DomainHealth is an open source server monitoring tool for BEA's WebLogic Application Server. It collects important server runtime statistics over time for all servers in a domain, archives this data into CSV files and provides a simple Web Browser based interface for viewing graphs of statistics.

DomainHealth is designed to have a low overhead on managed servers in terms of performance and is intended to be a 'zero config' tool *.

*(Currently in certain circumstances, some minimal configuration may be required - see the FAQ document on the project's host web site)

The statistics that DomainHealth collects, includes Core Server properties (eg. Open Sockets), JDBC Data Source properties (eg. Average Connection Delay) and JMS Destination properties (eg. Messages Current). The Web HTML based interface that DomainHealth provides, enables an administrator to view line-graphs for specific harvested server properties. The administrator can choose the window of time to view data for, which may go back hours, days, months and even years. Alternatively, an administrators can use his/her own tool of choice (eg. MS Excel, Open Office Spreadsheet) to analyse the contents of the generated CSV files (eg. to generate a graph view of some data from the CSV file loaded into a spreadsheet).

DomainHealth is deployed to the Admin Server of a WebLogic domain in the form of a J2EE Web-Application (WAR), and immediately collects properties from all servers in the domain, via a periodic polling mechanism, once a minute. All CSV files are generated and stored on the file-system of the Admin Server for subsequent viewing via the DomainHealth graphical web interface or for retrieval for offline analysis.

Domain Health is hosted on SourceForge at: http://sourceforge.net/projects/domainhealth

The latest version can be downloaded from the project home page. The home page also contains links to various Help Documents plus an Online Forum for users to get help and provide feedback. To install and run the DomainHealth monitor application, follow these simple steps:

1. Deploy domainhealth-nn.war to WebLogic, targetted to the Admin Server server of the domain only
2. Using a Web Browser, navigate to: http://adminhost:port/domainhealth

Supports WebLogic Server 9.x and 10.x.

Some people may ask why the need for this tool given the availability of WLDF in WebLogic. Personally I think these two tools are complimentary and I intend to blog on my reasons for this opinion in the near future.

Soundtrack for today: Auto Rock by Mogwai