Tips for Web Services Interoperability

[Originally posted on my old BEA Dev2Dev blog on August 10, 2007]

I try to follow some simple rules to maximise interoperability when developing Web Services using WebLogic and/or AquaLogic Service Bus (ALSB). Most of the rules are pretty obvious, but perhaps one or two are not?. In case some of these rules are useful to others, I thought I'd share them, so here they are:

1. Use SOAP over HTTP. I've blogged here about why SOAP over JMS shouldn't be used if interoperability is a concern.


2. Conform to the WS-I Basic Profile 1.1 by using the free WS-I Test Tool. Test the WSDL and over-the-wire SOAP requests/responses for the created Web Services, for conformity using the tool available here (look for "Interoperability Testing Tools 1.1").


3. Expose Web Services using the "Document-Literal-Wrapped" style with the 'dotNetStyle' flag to help WS-I conformity and to be especially Microsoft product friendly. I partly covered this in the blog here


4. Use the WS-* standards judiciously. WebLogic implemented standards such as WS-Addressing, WS-Security, SAML and WS-ReliableMessaging are not necessarily implemented by other Web Services products/stacks or the specification version supported by these may be different.


5. Don't necessarily dismiss the use of WebLogic 'add-value' / 'non-standard' Web Services features at face-value
   
   
   • 'Buffered' Web Services are interoperable with other client Web Services stacks at the basic SOAP-HTTP level because the service consumer is not aware that the service implementation uses a JMS queue for buffering internally.
   • 'Callbacks' may be interoperable with non-WebLogic service consumers as long as the non-WebLogic consumers include the WS-Addressing 'Reply-To' header in the request and provide a web service endpoint to be asynchronously called back on for the specified 'Reply-To' URL
   • 'Asynchronous Requests/Responses' may be interoperable with non-WebLogic service providers as long as the non-WebLogic providers honour the received WS-Addressing 'Reply-To' header of the request, by sending the Web Service response asynchronously to the specified 'Reply-To' URL.
   • However, 'Conversational' Web Services are highly unlikely to be interoperable with non-WebLogic based service providers or consumers. The specification 'WS-Conversation' which the 'Conversational' feature would probably most clearly map to, doesn't really exist as a public specification and there is no indication that it ever will (an incomplete internal draft version has been dormant for a few years now).
6. For SOAP/HTTP Proxies created in ALSB, activate the "WS-I compliance enforcement" option (for the development phase of the project at least). When ALSB is used to act as an intermediary between Web Services consumers and providers, this ALSB option will help any Web Service non-conformities to be detected, so that they can be quickly rectified.

Note: ALSB also transparently converts between SOAP version 1.1 and SOAP version 1.2 inbound and outbound messages and ALSB is specifically tested by BEA for interoperability against third-party vendor toolkits such as Microsoft .NET and Apache Axis.

Soundtrack for today: Forensic Scene by Fugazi

RPC-Encoded. Document-Literal. Does it really matter?

[Originally posted on my old BEA Dev2Dev blog on July 31, 2007]

In SOAP, there are two possible styles:

1. RPC. Implies a SOAP body structure which indicates service name, and multiple parameters and return values
2. Document. Implies a SOAP body which is a complex message document

In SOAP, there are two possible uses:

1. Encoded. Adheres to a set of rules for serialising a graph of typed objects using basic XML schema data-types, but as a whole, does not conform to a schema
2. Literal. Body content conforms to a specific XML schema

In most SOAP toolkits, the most common combinations of Style and Use are RPC-Encoded and Document-Literal. Additionally, RPC-Literal is becoming more prevalent although it is currently a lot less common. Document-Encoded doesn't really make sense and as a result I doubt you'll find it implemented in your favourite SOAP toolkit.

RPC-Encoded was the initial message format for SOAP, when SOAP was originally aimed at just the Remote Procedure Call programming model. Document-Literal was incorporated into the SOAP standard in time for SOAP 1.0. It was intended to enable XML documents (messages) to be passed as the full content of the SOAP body, usually with one input message part and one return message part.

Like most J2EE Application Servers, the core of WebLogic's Web Services support is based on the JAX-RPC 1.1 specification. JAX-RPC defines a Remote Procedure Call based programming model and API for developers who want to expose a set of Java methods remotely (JAX-RPC does not offer much in the way of support for adopting the alternative distributed computing model of "Messaging").

Given that JAX-RPC is based on the RPC programming model, then in terms of best practices, it's obvious then that we should be using RPC-Encoded (or RPC-Literal) as the preferred SOAP Style/Use for creating and exposing newly developed Web Services, isn't it?

Well, not necessarily....

The terminology of RPC versus Document SOAP Styles is very unfortunate when we start to consider Remote Procedure Call versus Document/Messaging distributed programming models. These terms imply that the RPC Style should be used for RPC programming models and that the Document style should be used for Document (Messaging) programming models. That is not the case at all. In practice, the SOAP Style has nothing to do with a programming model, it merely dictates how to translate a WSDL binding to a SOAP message. For example, WebLogic's JAX-RPC toolkit equally supports exposing the same Java methods remotely via either style. You can use either style with either programming model.

A SOAP Style/Use of Document-Literal provides two distinct advantages over RPC-Encoded:

1. WS-I Basic Profile precludes the use of "Encoded" as the SOAP Use. So, if promoting interoperability and openness is your concern then you wouldn't choose RPC-Encoded over Document-Literal* (and why else would you be using SOAP other than for interoperability?).


2. RPC-Encoded provides no real separation between the format of the SOAP body (eg. that could be defined by a Schema) and the transport protocol and invocation format of a SOAP operation (eg. defined by a WSDL). For Document-Literal, the SOAP body content conforms to one or more Schemas which can optionally be externalised from the WSDL (and then included into the WSDL via an 'import' statement). Why is this important? Well, Document-Literal can promote the re-use of the same XML Schemas across the many different Web Services you may need to expose and throughout the rest of your distributed application logic which may need to deal with the same XML data formats. As we know, re-use reduces development effort, helps avoid errors and promotes consistency.

In summary, the decision of using RPC-Encoded or Document-Literal really doesn't have a direct relation to whether one is adopting a Remote Procedure Call programming model rather than a Document/Message-passing programming model. In practice, regardless of programming model, Document-Literal offers practical advantages.

Footnote: There is a new alternative Web Services toolkit which can be used, based on JAX-WS 2.0. JAX-WS offers developers an alternative programming model based on Messaging in addition to the Remote Procedure Programming model which is also supported. JAX-WS is newly supported in WebLogic 10, with some restrictions.

* In fact, use WebLogic's 'Wrapped' option with "Document-Literal" to help further promote interoperability with both Microsoft based Web Services toolkits (which traditionally prefer the Document-Literal-Wrapped style) and Remote Procedure Call oriented client Web Services toolkits (which often expect to be able to include the 'remote operation' name within the SOAP requests they send, rather than using another mechanism such as WS-Addressing to identify the operation to invoke).

Soundtrack for today: Off To One Side by Come

WLST and SQLPLus on Linux - get UP, DOWN, LEFT, RIGHT and other keys working

[Originally posted on my old BEA Dev2Dev blog on July 24, 2007]

This has been bugging me for ages so today I spent some time trying to find a solution. I've managed to find one, so I thought I'd share it with you just in case it is useful to others....

When running command line tools in a Linux shell that read from standard input (using the 'readline' library under the covers), the UP and DOWN arrow keys can't be used to select previous history commands. Also, the LEFT, RIGHT arrow keys and the HOME and END keys can't be used to move to different parts of the current text line to easily correct mistakes in the line before submitting.

As a JavaEE/WebLogic user I find this really frustrating when I need to use tools like WLST or SQLPlus in interactive mode to mess around with WebLogic domains or change Oracle database schemas, respectively. Well the solution is a nifty little tool called rlfe which I found here.

Once installed, before running 'wlst.sh' or 'sqlplus' from a shell, first enter...

> rlfe

...and press return. Then when you run WLST or SQLPlus from the same shell, the interactive mode will work the way you always wanted it to. You'll be able to press the UP key to get to a previous command to execute it and you'll be able to press the LEFT key to move further back in the line of text to correct a typo.

The good news if you're a Ubuntu user is that this is available in the standard universe repository. To install it, just run....

> sudo apt-get install rlfe

Also, to avoid forgetting to run 'rlfe' before running your program, you can use an alias for the program to ensure that your program is always run in this 'enhanced' text entry mode. For example, add the following to ~/.bashrc :

alias sqlplus='rlfe sqlplus'

..then whenever you run sqlplus from the command line it'll work the way you want.

[In a comment on the original Dev2Dev blog 'eduardo_biagi' suggested an alternative tool called 'rlwrap' which works on Fedora. Since installing 8.04 version of Ubuntu, 'rlfe' does not work properly on Ubuntu so I now use 'rlwrap'. For example I have the following alias set up in my .bashrc: 'alias wlst='rlwrap /opt/oracle/wls1001/wlserver_10.0/common/bin/wlst.sh']

Soundtrack for today: Faded by The Afghan Whigs

The problem with using SOAP over JMS in SOA

[Originally posted on my old BEA Dev2Dev blog on March 28, 2007]

Sometimes I talk to people who seem to view the use of SOAP over JMS as the perfect combination to enable loosely coupled asynchronous shared services. However, when I dig deeper these people have invariably assumed that JMS is an 'over-the-wire' protocol, like HTTP. It is not.

Question: Why is this a problem?

Answer: Interoperability, plain and simple.

HTTP is a standard 'over-the-wire' protocol. HTTP belongs in the Application Layer of both the OSI model (layer 7) and the Internet Protocol Suite (layer 4 or 5). SOAP is a 'standard' (or W3C recommendation at least) transport agnostic protocol which uses an XML payload.

Due to the standard and technology agnostic nature of both SOAP and HTTP, many platforms and toolkits out there, written in different languages and on different operating systems, can interoperate using SOAP over HTTP by simply adhering to both of these standards (or at least the WS-I version of these standards).

However, JMS is not an 'over-the-wire' protocol. It is a Java API which requires that a client application uses a JMS provider library (JAR) provided by the vendor of the JMS Server hosting the services. This is analogous to requiring a JDBC driver for a particular vendor's database before a Java application can talk to that database. The actual 'over-the-wire' protocol used under the covers within the JMS provider library is not defined (it could be IIOP for example, or it could be some high speed non-standard vendor specific protocol).

As a result, in most cases, the only types of applications which can talk to a specific vendor's JMS Server are other Java based applications. It gets worse. If, for example, the JMS server vendor is IBM WebSphere and the service consumer is running within Oracle's Application Server, there may be problems even getting IBM's JMS client provider library working from within Oracle's Application Server in the first place, due to JMS implementation clashes. Some JMS Server vendors provide one or two non-Java based JMS libraries too (for example for C++ or .NET), but these are often limited in functionality and scope and often only support specific versions of specific platforms and operating systems.

In other words, the onus of interoperability, when using SOAP over JMS, is on the support of the vendor of the JMS server for all possible service consumer environments rather than the onus being on the service client's host environment support for standards. Vendors cannot scale to provide JMS support for all of the wide mix of programming languages, application servers and operating systems (including different versions) out there, so interoperability will take a big hit. Even for consumer applications that can use the JMS provider, one has to give the service consumer the provider library first before it can invoke services - not very loosely coupled I think.

As a result, an enterprise's design choice to use SOAP over JMS, as the default mechanism for interoperability for an enterprise's mix of heterogeneous systems, is likely to be fundamentally flawed in my opinion.

It is important to state that I am not saying that Message Oriented Middleware (MOM) does not have a place in a SOA framework. In fact, quite the opposite is true. To achieve capabilities such as asynchronous messaging, guaranteed delivery, only once delivery, and publish/subscribe mechanisms, MOMs are an essential part of the SOA fabric. That's why many vendor's ESB platforms are built on the underlying technology of Message Oriented Middleware. However, what I am saying is that JMS should not be the preferred API for exposing shared services to remote service clients. Using middleware such as an ESB for example, a service with an asynchronous interface can be exposed via a SOAP over HTTP interface, for example, where the ESB performs the switching between the consumer facing synchronous invocation protocol and the underlying internal asynchronous message passing mechanism which may or may not use JMS internally.

With the right organisation and governance in place, I believe it can be valid to decide to expose a shared service via SOAP/JMS in addition to SOAP/HTTP or another more 'open' protocol, where there are valid exceptional circumstances (eg. high performance requirements). However, it is probably best to treat these decisions on an exception by exception basis because the overhead of supporting two access methods for a service does have an additional overhead due to increased configuration, maintenance, and testing costs.

Is HTTP the perfect transport for SOAP, especially for asynchronous services? Not at all. However, if consumers can't invoke these services in the first place - that's worse.

Have I got something against JMS? Not at all. Its one of my favourite JavaEE APIs. I'm not talking about JavaEE here. I am talking about SOA.

Soundtrack for today: Happy Man by Sparklehorse