Friday, October 10, 2008

Web Service Messaging Nirvana

Web Services and specifically SOAP has got a bad rap in recent years, believed to be yet another over-hyped remote procedure call (RPC) protocol, and not cool enough in light of more current hip approaches like REST. Some of this flak is justified and the SOAP community does itself no favours when it releases a newer version of a spec (SOAP 1.2) which goes out of its way to state that SOAP is no longer an acronym. What next? TAFKAS? (The Artist Formerly Know As SOAP)

Anyway, I digress. I blogged in the past about two main areas where SOAP can be used. As you might guess, the main area that interests me is Messaging.

Over the last few years, I've been on a quest to explore how a JavaEE architect and developer can better implement messaging based web services and service consumers in a more loosely coupled and less error prone way. My motivation has been driven by a reaction to the traditional RPC based Web Services programming model that many have used and still use today and all the problems that such a model brings.

Over time I have constructed a list of 8 key values or principles that I believe are necessary for achieving "Web Services Messaging Nirvana". In most cases, these principals are based more on architectural and development approach, than the specific technology or SOAP toolkit used. However, as always, the two can't be completely divorced from each other..

Here's my 8 key values:

  1. Promote Interface/Contract-First Development
  • “Implementation-First” leads to tight coupling between the code in the service provider and the code in the service consumer
  • "Implementation First" leads to a strong dependency on the initial SOAP toolkit used for implementation, which is then difficult to migrate off and leads to the difficulties of vendor lock-in
  1. Enable a Message-Passing Programming Model
  • RPC programming leads to lots of find grained 'chatty' services where brittle pairings exist between provider and consumer which are resistant to change
  • RPC clients tend to assume endpoints are implementations when in reality, they may just be messaging intermediaries (eg. a service bus)
  1. De-couple “Service Interface Definition” from “Message Document Definition”
  • In-lining XML type definitions inside a WSDLs prevents re-use of the message structure definitions across multiple web services and other components in the system as a whole (import external Schemas instead)
  • The message document structures may already be pre-defined in the organisation (eg. industry-wide standard schemas), so the ability to share these definitions across many web services is required‏
  1. Promote Postel’s Law: “Be conservative in what you do; be liberal in what you accept from others”
  • In other words, allow developers to create services with strongly defined interfaces, yet forgiving implementations
  • Service intermediaries and end-points may only care about using part of the message content - they should not concern themselves about trying to validate other parts of a message
  • Enable minor interface version additions without necessarily requiring an upgrade or change to the service provider or service consumer
  1. Avoid the need for manually generated static stubs and skeletons
  • These are invariably an unnecessary time-consuming addition to the development and build process
  • Interface modification always requires re-generation of stubs, skeletons and Java types, forcing service providers and consumers to need to always be corrected, re-compiled and re-deployed.
  • A requirement for Skeletons and Stubs is unlikely when generated Java types are not being used in the implementation
  1. Separate data from the code that operates on it
  • Object-oriented programming is great for service implementations but not for de-coupling disparate elements of the higher level distributed systems that are based on the principle of passing messages or events around
  • Enable message data to be passed between systems without one system caring about or pre-empting how the other system may use and operate on the message’s content
  1. Provide the developer with a choice of XML Tools to use
  • SOAP Toolkits which force messages to be marshalled into a strongly typed Java graph of objects, results in one and only one way that the programmer can operate on the data
  • If only part of a message is required or performance is key, a developer should have the option to use a SAX parser, a Streaming parser or just an XPath based Java API to query and operate on the data
  • If XQuery or XSLT offers some developers a better abstraction and quicker development process then don't prevent a developer from being able plug in this option
  • Retain the choice of being able to generate strongly types Java objects for situations where the developer needs or prefers it, but enable the developer to choose the technology (eg. JAXB, XmlBeans)‏
  1. Integrate with the host container's Security Framework for Web Services Security
  • For an application server based production environment, the server's Security Framework is already likely to be plugged into the organisation's larger security infrastructure (inc. LDAP, Kerberos, PKI and other elements) to provide full support for authentication, access control, single sign-on, identity propagation, encryption and signing. This infrastructure needs to be leveraged by the SOAP Toolkit, not ignored
  • The SOAP Toolkit needs to delegate to, or at least share its security context with, the host container, especially for its support of WS-Security and SAML

In the past, with J2EE 1.2, 1.3 and 1.4 compatible application servers, these values have been very hard to achieve with the application server's built-in SOAP support (ie. JAX-RPC - oriented around the concept of RPC, skeletons, stubs, and generated Java types) or with a Third-Party SOAP Toolkit (eg Apache Axis - little integration with the container's underlying Security Framework).

As a result, on these platforms, I've struggled to be able to get anywhere near achieving my goal of hitting the 8 key values of my Web Service quest.

JavaEE 1.5 introduced the JAX-WS API which is the newer Web Services toolkit for JavaEE developers. The focus of this standard, in my opinion, is still largely based on RPC and generated Java types. However, it does offer a glimmer of hope in form of the option of using a 'dnyamic' server side API (Provider interface) and client-side API (Dispatcher interface).

To my eyes the Provider/Dispatcher facility could finally give me a chance of achieving the 8 key values in a real Web Services based solution.

Version 10.0 of WebLogic was the first version to introduce JAX-WS, but at the time lacked integration with WebLogic's Security Framework and WS-Security and SAML support, thus lessening its relevance to many solutions where securing web services was a key requirement. Many users continued to use WebLogic's JAX-RPC stack due to the need to leverage Weblogic security.

Things have now changed and the JAX-WS implementation in version 10.3 of WebLogic should address this. I hope to blog soon about my experiments with JAX-WS Providers/Dispatchers in WebLogic 10.3 and my quest to realise my 8 key values of Web Services Messaging Nirvana.

Song for today: Burn Baby by Mother Tongue


Unknown said...

Good post Paul, however one question about the Interface first approach. As you are no doubt aware, many programmers are not very familiar with creating WSDL or Schema by hand and typically adopt a code first approach. This can have issues as you noted, particularly with JAX-RPC, however I find that with JAXWS and JAXB annotations these issues can be mitigated and a pragramatic code-first but contract aware approach tends to work quite well and reasonably balances productivity against the creation of a proper contract by the service.

Have you had a look at this sort of approach and do you have any thoughts about the pros and cons of such an approach versus a purely contract first approach now that you have had a chance to use JAX-WS more?

Paul Done said...

Hi Gerald - Sorry only just saw your comment. I agree that the rich JAX-WS/JAXB integration does indeed help enable you to [eventually] get the WSDL you want if starting from Code First. However, the key thing I am hitting with many customers at the moment is that they have XML schemas already standardised for their department or business or industry and they want re-use of these schemas not just accross web services they develop but within other components of the system which need to process XML oriented business messages too. With a Code-First approach its nearly impossible to re-use existing Schemas without spending massive amounts of time trying to get all the JAXB annotatiosn just right on your Java classes to try to tease the generated WSDL/Schemas to match the already existing XML Schemas in your enterprise. Most of the article is focussed on a Web Services for *Messaging* approach. I agree that if we were just RPC enabling existing system logic (ie. making it remote-able) then a Code First approach may well be more appropriate (or a REST based approach). Cheers Paul

Unknown said...

Thanks Paul, obviously in the case where schemas already exist a contract first approach is going to be required. I was just curious what your feeling was on code-first because I use to have strong feelings about contract first when building new services from scratch but found that a pragamatic code first, contract-aware approach tended to be orders of magnitude more productive and generated very good results.

Thanks for the blog and keep up the good work.

Palace said...

Excellent checklist