March 23, 2011, 2:54 p.m.
posted by apollo
Extended Web Services Specifications
Following the broad adoption and use of the basic Web services specificationsSOAP and WSDLrequirements have grown for the addition of extended technologies such as security, transactions, and reliability that are present in existing mission-critical applications. These extended features are sometimes also called qualities of service because they help implement some of the harder IT problems in the SOA environment and make Web services better suited for use in more kinds of SOA-enabled applications.
A class of applications will find the core specifications sufficient, while other applications will be blocked or hampered by the lack of one or more of the features in the extended specifications. For example, companies may not wish to publish their Web services without adequate security or may not wish to accept purchase orders without reliable messaging guarantees.
The core specifications were defined with built-in extensibility points such as SOAP headers in anticipation of the need to add the extended features.
Web services specifications progress toward standardization through a variety of ways, including small groups of vendors and formally chartered technical committees. As a general rule of thumb, most specifications are started by a small group of vendors working together and are later submitted to a standards body for wider adoption. Specifications initially created by Microsoft and IBM, together with one or more of their collaborators (these vary by specification, but typically include BEA, Intel, SAP, Tibco, and Verisign), tend to gain the most market traction. Microsoft and IBM are the de facto leaders of the Web services specification movement and have defined or helped to define all the major specifications. Several of the WS-* specifications remain under private control at the time of writing, but we expect them to be submitted to a standards body in the near future.
Standards bodies currently active in Web services include:
Web services standardization started with the submission of the SOAP 1.1 specification to the W3C in mid-2000. After that, SOAP with Attachments, XKMS, and WSDL were submitted to W3C. At the same time, UDDI was launched in a private consortium and was later submitted to OASIS. Other major specifications submitted to OASIS include WS-Security, WS-BPEL, WS-CAF, and WS-Notification. More recently, WS-Addressing and WS-Policy were submitted to W3C, signaling a potential shift back toward W3C as the home of most of the major specifications.
Historically, OASIS is also the home of the ebXML set of specifications, which overlap to a large extent with the Web services stack. Web Services and ebXML share SOAP, but beyond that, the stacks diverge. ebXML has its own registry and its own orchestration (or choreography) language.
As mentioned previously, the SOAP and WSDL specifications are designed to be extended by other specifications. Two or more of the extended specifications can be combined within a single SOAP message header. For example:
<S:Header> <wsse:Security> ... </wsse:Security> <wsrm:Sequence> ... </wsrm:Sequence> </S:Header>
This example illustrates the use of extended headers for security and reliability. The security header typically includes information such as the security token that can be used to ensure the message is from a trusted source. The reliability header typically includes information such as a message ID and sequence number to ensure the message (or set of messages) is reliably received.
Note the separate namespaces used for the security and reliability headers, wsse: and wsrm:, respectively. The headers use different namespaces so that they can be added incrementally to a SOAP message without concern over potential name clashes. Duplicate element and attribute names are not permitted in an XML document (and a SOAP message is an XML document, after all). Namespace prefixes provide a unique qualifier for XML element and attribute names, thus protecting names from duplication. This is one way in which SOAP header extensions work composably with each other.
Adding extended features may or may not require modification to existing Web servicesthat is, the extended features can be added into the SOAP headers without changing the SOAP body in many cases. But the execution environments and mapping layers may need to change in order to handle the extensions. Certainly at least adding SOAP headers for extended features must be done within the context of knowing whether the execution environment can support them and how; otherwise, the extended headers will not work.
Web service extensions are also added to the responsibility of the SOAP processors in the execution environment. Policy declarations associated with the WSDL contracts can be used during the generation of SOAP messages to determine what should go into the headers to help the execution environment negotiate the appropriate transport protocol or to agree on features such as transaction coordination.
As illustrated in Figure, each additional extended feature added to the Web service invocation results in additional processing before sending the message or after receiving it. Later chapters provide further detail on each of these extended features. The extended features may also be related to requirements from a business process engine and may need to be supported by the registry.
Adding extended features to SOAP.
Metadata management includes the description information about a Web service necessary to construct a message body (including its data types and structures) and message headers so that a service requester can invoke a service. The provider of the service publishes the metadata so that a requester can discover it and use it to construct messages that can be successfully processed by the provider.
When invoking a service, it's important to understand not only the data types and structures to send but also the additional qualities of service provided (if any), such as security, reliability, or transactions. If one or more of these features are missing from the message, it may prevent successful message processing.
Metadata specifications include:
Service binding is different for an SOA based on Web services compared to an SOA based on J2EE or CORBA, for example. Instead of binding via reference pointers or names, Web services bind using discovery of services, which may be dynamic. If the service requester can understand the WSDL and associated policy files supplied by the provider, SOAP messages can be generated dynamically to execute the provider's service. The various metadata specifications are therefore critical to the correct operation of an SOA based on Web services.
Addressing is an important requirement of extended Web services because no directory of Web services endpoint addresses exists on the Web. SOAP messages must include the endpoint address information within the message for all but the simplest MEP. WS-Addressing replaces earlier proposals called WS-Routing and WS-Referral.
Without an addressing solution, when you send a Web service request to a provider, typically the only address the provider has is the return address to the requester, and then only for the duration of the session. If anything goes wrong on the reply, there's no good way to retry itbasically the requester's address can be lost when there's a communication failure. Also there's no good way to specify a different return address than the requester's address. And finally, there's no way to define address schemes or to identify endpoint addresses for complicated message exchange patterns or multi-transport support.
Policy is necessary for expressing any extended Web services features of a service so that the requester can provide the necessary security, transaction, or reliability context in addition to the data requirements for the messages expressed in the WSDL file.
Policy provides a machine-readable expression of assertions that a service requester must adhere to in order to invoke upon a provider. Does the service require security or support transactions? The latter can be very important when trying to figure out whether or not a long running, complex interaction can involve a transaction, or whether a transaction can span across all the Web services identified for it.
WS-Policy is necessary for achieving interoperability for the extended features because the policy declarations are the only way in which a requester can discover whether a provider requires some or all of the extended features. In the case of security, for example, different providers may support different kinds of tokens, such as X.509 or Kerberos. WS-Security is designed as a kind of open framework that can carry any token type. However, if the token type the provider expects isn't declared, the requester can only guess at what it must be.
When making the decision to invoke the provider's service, it may also be important to discover whether it supports reliability or transactions. You might want to know, for example, whether the provider's service can accept retries if the original submission fails and whether it will let you know when it has successfully received a message. Finally, you may want to know whether or not to send a transaction context to the provider to enroll the provider Web service in the transaction.
It's possible that a requester will obtain the metadata it needs using WS-MetadataExchange or another similar mechanism that queries the WSDL and associated policy files directly. WS-MetadataExchange uses a feature of WS-Addressing called "actions" to access the metadata published by a service provider. WS-MetadataExchange is designed to provide any and all information about a Web service descriptionessentially replacing UDDI for most applications.
Developers may or may not use UDDI, despite its existence. It's fair to say that the public UDDI does not provide the metadata management facilities required to support interoperability requirements at the extended specification level and that WS-MetadataExchange may be needed for requesters to ensure they have the information they need to achieve interoperability with providers using extended features.
Security concerns apply at every level of the Web services specification stack and require a variety of mechanisms to guard against the numerous challenges and threats that are a part of distributed computing. The mechanisms may have to be used in combination to guard against a particular threat or combination of threats. In the Web services and SOA world, it's particularly important to evaluate the need for protection at the network layer, the message layer, and for the data in the message.
Basic security protection mechanisms are built around encryption, authentication, and authorization mechanisms and typically include comprehensive logging for problem tracking. The industry has achieved consensus around a single specification framework, WS-Security, although ongoing work is necessary to complete the profiles and additional related specifications.
WS-Security was developed to provide message-level security on an end-to-end basis for Web services messages. Typical HTTP-based security mechanisms, such as SSL, provide only transport-level point-to-point protection. Sometimes additional security may be provided through the use of an alternative transport mapping, such as CORBA's IIOP or WebSphere MQ, but as with the rest of the extended features, the security specifications are written for HTTP as a kind of default or lowest common denominator transport and therefore can be applied to any transport.
WS-Security headers include the ability to carry strong authentication formats such as Kerberos tickets and X.509 certificates and can use XML Encryption and XML Signature technologies for further protecting the message contents. Although a WS-Security authorization profile for the Security Assertion Markup Language (SAML) is being developed, SAML can also be used on its own for exchanging authorization information.
Because Web services are XML applications, and because XML has security challenges of its own (it is basically human-readable text sent over the Internet), XML-based security technologies are also often important for protecting the XML data before and after it's included in a SOAP message. These technologies include:
Reliability and Messaging
Messaging includes SOAP and its various message exchange patterns (MEP). The industry has not achieved consensus on a single, unified set of specifications for advanced messaging. However, competing specifications in the categories of reliability and notification work essentially the same way, and so an amalgam of the two is used here for the sake of introduction.
In general, reliable messaging is the mechanism that guarantees that one or more messages were received the appropriate number of times. Reliable messaging specifications include:
Reliable messaging is designed to ensure reliable delivery of SOAP messages over potentially unreliable networks such as the HTTP-based Internet.
Reliable messaging is a protocol for exchanging SOAP messages with guaranteed delivery, no duplicates, and guaranteed message ordering. Reliable messaging works by grouping messages with the same ID, assembling messages into groups based on message number, and ordering them based on sequence number.
Reliable messaging automates recovery from certain transport-level error conditions that the application would otherwise have to deal with on its own. Reliable messaging also supports the concept of bridging two proprietary messaging protocols over the Internet.
Also in the general messaging area are specifications for extended MEPs such as event notification and publish/subscribe, which basically extend the asynchronous messaging capability of Web services. Specifications in this area include:
Notification delivers messages through an intermediary often called a message broker or event broker. Subscribers identify the channels or topics for which they wish to receive messages. Publishers send messages to the channels or topics on which subscribers are listening. Notification is a messaging mechanism that can be used to set up broadcast and publish/subscribe messaging.
Transactions allow multiple operations, usually on persistent data, to succeed or fail as a unit, such as processing an order or transferring funds. One of the most important aspects of transaction processing technologies is their ability to recover an application to a known state following an operating system or hardware failure. For example, if any failure occurs before a funds transfer operation is completed (that is, both the debit and credit operations), transactions ensure the bank account balances are what they were before the funds transfer operation started.
Many Web services applications may require only the transaction processing capabilities inherent in the underlying execution environment, such as those provided by application servers and databases. Others may require multiple Web service invocations to be grouped into a larger transactional unit, including a transactional context within SOAP headers so that the transaction can be coordinated across multiple execution environments.
Web services transaction specifications extend the concept of the transaction coordinator, adapt the familiar two-phase commit protocol for Web services, and define new extended transaction protocols for more loosely coupled Web services and orchestration flows. Transaction coordinators currently exist in most execution environments, including J2EE, the .NET Framework, and CORBA. Web services specifications define extensions for these (and other) coordinators for compensation-based protocols and long-running coordinator-coordinator protocols that bridge software domains.
Coordination is a general mechanism for determining a consistent, predefined outcome for composite Web service applications. The coordinator model includes two major phases: the acquisition phase in which Web services that are participating in the composite are enrolled with the coordinator for a specific protocol (such as two-phase commit, compensation, or business process) and a second phase in which the coordinator drives the agreed-upon protocol to completion following the end of the execution of the set of services in the composite. When a failure occurs, the coordinator is responsible for driving the recovery protocol (if any).
Both sets of specifications are centered on an extended coordinator with pluggable transaction protocols. Both sets of specifications define atomic transactions (i.e., two-phase commit) and compensation-based transactions. WS-CAF breaks context management into a separate specification and adds a third transaction protocol specifically designed for business process management.
Web services can and eventually will be published for most software systems and applications within a given IT environment, and in fact across multiple organizations' IT environments. Rather than have Web services invoke each other using one or more of the message exchange patterns supported by SOAP and WSDL, an orchestration engine can be used to create more complex interaction patterns in long-running business process flows with exception handling, branching, and parallel execution. To accomplish this, the orchestration engine has to preserve context and provide correlation mechanisms across multiple services.
A Web service orchestration may also be published as a Web service, providing an interface that encapsulates a sequence of other Web services. Using the combination of MEPs and orchestration mechanisms, entire application suites can be built up out of Web services at multiple levels of encapsulation, from those that encapsulate a single software module to those that encapsulate a complex flow of other Web services.
The industry has reached a consensus around a single orchestration specification: the OASIS Web Services Business Process Execution Language (WS-BPEL). WS-BPEL assumes that Web services are defined using WSDL and policy assertions that identify any extended features.
Typically, a flow is initiated by the arrival of an XML document, and so the document-oriented Web services style tends to be used for modeling the entry point to a flow. Parts of the document are typically extracted and operated upon by the individual tasks in the flow, such as checking on the inventory availability for each line item from a different supplier, meaning the steps in the flow may be implemented using a combination of request/response and document-oriented Web services.
The WS-BPEL specification differs from other extended specifications in that it defines an executable language compatible with various software systems that drive business process automation. Whereas most other Web services specifications are XML representations of existing distributed computing features and capabilities that extend SOAP headers, orchestration represents the requirement for composing Web services in a declarative manner.
The W3C's Web Services Choreography Definition Language (WS-CDL) is another specification in the general area of orchestration. Choreography is defined as establishing the formal relationship between two or more external trading partners. One difference between WS-CDL and WS-BPEL is that WS-CDL does not require Web services infrastructure at all of the endpoints being integrated.