Web Service Messaging Nirvana

Web Services and specifically SOAP has got a bad rap in recent years, believed to be yet another over-hyped remote procedure call (RPC) protocol, and not cool enough in light of more current hip approaches like REST. Some of this flak is justified and the SOAP community does itself no favours when it releases a newer version of a spec (SOAP 1.2) which goes out of its way to state that SOAP is no longer an acronym. What next? TAFKAS? (The Artist Formerly Know As SOAP)

Anyway, I digress. I blogged in the past about two main areas where SOAP can be used. As you might guess, the main area that interests me is Messaging.

Over the last few years, I've been on a quest to explore how a JavaEE architect and developer can better implement messaging based web services and service consumers in a more loosely coupled and less error prone way. My motivation has been driven by a reaction to the traditional RPC based Web Services programming model that many have used and still use today and all the problems that such a model brings.

Over time I have constructed a list of 8 key values or principles that I believe are necessary for achieving "Web Services Messaging Nirvana". In most cases, these principals are based more on architectural and development approach, than the specific technology or SOAP toolkit used. However, as always, the two can't be completely divorced from each other..

Here's my 8 key values:

1. Promote Interface/Contract-First Development

• “Implementation-First” leads to tight coupling between the code in the service provider and the code in the service consumer
• "Implementation First" leads to a strong dependency on the initial SOAP toolkit used for implementation, which is then difficult to migrate off and leads to the difficulties of vendor lock-in

2. Enable a Message-Passing Programming Model

• RPC programming leads to lots of find grained 'chatty' services where brittle pairings exist between provider and consumer which are resistant to change
• RPC clients tend to assume endpoints are implementations when in reality, they may just be messaging intermediaries (eg. a service bus)

3. De-couple “Service Interface Definition” from “Message Document Definition”

• In-lining XML type definitions inside a WSDLs prevents re-use of the message structure definitions across multiple web services and other components in the system as a whole (import external Schemas instead)
• The message document structures may already be pre-defined in the organisation (eg. industry-wide standard schemas), so the ability to share these definitions across many web services is required‏

4. Promote Postel’s Law: “Be conservative in what you do; be liberal in what you accept from others”

• In other words, allow developers to create services with strongly defined interfaces, yet forgiving implementations
• Service intermediaries and end-points may only care about using part of the message content - they should not concern themselves about trying to validate other parts of a message
• Enable minor interface version additions without necessarily requiring an upgrade or change to the service provider or service consumer

5. Avoid the need for manually generated static stubs and skeletons

• These are invariably an unnecessary time-consuming addition to the development and build process
• Interface modification always requires re-generation of stubs, skeletons and Java types, forcing service providers and consumers to need to always be corrected, re-compiled and re-deployed.
• A requirement for Skeletons and Stubs is unlikely when generated Java types are not being used in the implementation

6. Separate data from the code that operates on it

• Object-oriented programming is great for service implementations but not for de-coupling disparate elements of the higher level distributed systems that are based on the principle of passing messages or events around
• Enable message data to be passed between systems without one system caring about or pre-empting how the other system may use and operate on the message’s content

7. Provide the developer with a choice of XML Tools to use

• SOAP Toolkits which force messages to be marshalled into a strongly typed Java graph of objects, results in one and only one way that the programmer can operate on the data
• If only part of a message is required or performance is key, a developer should have the option to use a SAX parser, a Streaming parser or just an XPath based Java API to query and operate on the data
• If XQuery or XSLT offers some developers a better abstraction and quicker development process then don't prevent a developer from being able plug in this option
• Retain the choice of being able to generate strongly types Java objects for situations where the developer needs or prefers it, but enable the developer to choose the technology (eg. JAXB, XmlBeans)‏

8. Integrate with the host container's Security Framework for Web Services Security

• For an application server based production environment, the server's Security Framework is already likely to be plugged into the organisation's larger security infrastructure (inc. LDAP, Kerberos, PKI and other elements) to provide full support for authentication, access control, single sign-on, identity propagation, encryption and signing. This infrastructure needs to be leveraged by the SOAP Toolkit, not ignored
• The SOAP Toolkit needs to delegate to, or at least share its security context with, the host container, especially for its support of WS-Security and SAML

In the past, with J2EE 1.2, 1.3 and 1.4 compatible application servers, these values have been very hard to achieve with the application server's built-in SOAP support (ie. JAX-RPC - oriented around the concept of RPC, skeletons, stubs, and generated Java types) or with a Third-Party SOAP Toolkit (eg Apache Axis - little integration with the container's underlying Security Framework).

As a result, on these platforms, I've struggled to be able to get anywhere near achieving my goal of hitting the 8 key values of my Web Service quest.

JavaEE 1.5 introduced the JAX-WS API which is the newer Web Services toolkit for JavaEE developers. The focus of this standard, in my opinion, is still largely based on RPC and generated Java types. However, it does offer a glimmer of hope in form of the option of using a 'dnyamic' server side API (Provider interface) and client-side API (Dispatcher interface).

To my eyes the Provider/Dispatcher facility could finally give me a chance of achieving the 8 key values in a real Web Services based solution.

Version 10.0 of WebLogic was the first version to introduce JAX-WS, but at the time lacked integration with WebLogic's Security Framework and WS-Security and SAML support, thus lessening its relevance to many solutions where securing web services was a key requirement. Many users continued to use WebLogic's JAX-RPC stack due to the need to leverage Weblogic security.

Things have now changed and the JAX-WS implementation in version 10.3 of WebLogic should address this. I hope to blog soon about my experiments with JAX-WS Providers/Dispatchers in WebLogic 10.3 and my quest to realise my 8 key values of Web Services Messaging Nirvana.



Song for today: Burn Baby by Mother Tongue

Web Services: RPC, REST and Messaging

[Originally posted on my old BEA Dev2Dev blog on February 25, 2008]

Choosing a model for interoperable communication in the enterprise

For the implementation of Web Services in the enterprise environment, I've seen many different technologies used. Recently, in my spare moments, I've reflected on this and have come to the conclusion that all these technologies tend to fit one of three models (or hybrids of these models).

I would summarise these three models as:

1. Remote Procedure Calls (RPC). A client-server based remotable pattern where a subset of an existing system's local functions is exposed pretty much 'as-is' over the wire to client programs.
2. Resource-oriented Create-Read-Update-Delete (CRUD). A client-server based resource-oriented pattern where the server-side provides a representation of a set of resources (often hierarchical) and exposes Create, Read, Update and Delete capabilities for these resources to client programs.
3. Messaging (eg. as commonly seen with Message Oriented Middleware and B2B). Messages or documents are passed asynchronously between peer systems in either, but not always both, directions.

Sometimes its hard to distinguish between these models and where the boundaries lie. In fact, I don't think there are boundaries, only grey areas and all three models lie in the same spectrum. I've attempted to visualise this spectrum in the diagram below.

Depending on where your implementation lies in the spectrum, the different highlighted characteristics should manifest themselves. 

 

In the Web Services world, we may typically implement these three models using one of the following three approaches:

1. Remote Procedure Calls: SOAP using a synchronous RPC programming approach and, typically, generated 'skeletons/stubs' and some sort of Object-to-XML marshalling technology
2. Resource-oriented Create-Read-Update-Delete: REST or 'RESTful Web Services' or ROA, re-using World-Wide-Web based approaches and standards like HTTP and URIs
3. Messaging: SOAP using an asynchronous Message/Document passing approach where invariably the documents are defined by schemas and, often, the use of message-level (rather than transport-level) security elements is required

The confusing thing is the fact that SOAP can happily and naturally satisfy two of these three models (ie. both RPC and Messaging), albeit that these two models are polar opposites which causes much confusion and is probably part of the reason why SOAP gets a bad name in some quarters.

The problem is further compounded with SOAP because the SOAP specification and accompanying collateral talks about two possible 'Styles' (ie. 'RPC' style and 'Document' style). However as I explained here, just because your SOAP Web Services are marked as 'Document' style in your WSDLs, it doesn't mean you are not doing remote procedure call based programming.

When faced with the REST zealot or the WS-* zealot, we probably need to bear this spectrum in mind. For the Web Services paradigm, there is not a 'one-size fits all' and specific requirements for a given situation should dictate which position in this spectrum best lends itself to satisfying the requirements. Also, the overlap between the models may be greater than shown in the diagram. For example, some would argue that REST can happily and more appropriately be used to fulfil what would otherwise be RPC oriented problems, in addition to solving Resource-oriented CRUD style problems.

Soundtrack for today: Purity by The God Machine