4.8. Queuing Products and StandardsA variety of queue manager products are available. One of the original implementations was in IBM’s IMS TP monitor, where queued TP was the default behavior. Queuing is integrated with many other transactional middleware products, such as Oracle’s WebLogic and JBoss Messaging. It is also integrated in Oracle Database, called Oracle Streams AQ, and in Windows, called Microsoft Message Queue (MSMQ). Some vendors offer queuing in independent products, such as TIBCO’s Enterprise Message Service, Progress’ SonicMQ, Apache ActiveMQ, and IBM’s Websphere MQ (MQ = Message Queuing). A consortium sponsored by JP Morgan Chase has proposed a messaging standard, called Advanced Message Queuing Protocol. We briefly describe WebSphere MQ and Oracle Streams AQ here as two examples of such products. IBM’s Websphere MQIBM promotes WebSphere MQ[1] as an integration solution among its various operating system and TP environments and those of other vendors. It has a proprietary API, called Message Queuing Interface (MQI), a Java Messaging Service (JMS) API, and a non-Java equivalent of JMS. It can be used by applications running under IBM’s transactional middleware such as WebSphere Application Server and CICS Transaction Server, and on any operating system supported by WebSphere MQ, including IBM AIX, i5/OS, OS/400, and z/OS, as well as HP-UX, Linux, Sun Solaris, and Microsoft Windows.
The WebSphere MQ queue manager accepts input from an application via the JMS API or MQI verbs. The main verbs are MQPUT to enqueue a message and MQGET to dequeue a message. A named queue can support multiple concurrent enqueuers and dequeuers. To process an MQPUT, the queue manager starts a transaction if the application is not already executing one and places the message in the queue. The operation is committed along with the rest of the transaction (which can be the normal exit from the application) or can optionally run in its own transaction as described in the subsection on nontransactional queuing in this section. The enqueued message consists of application data and the message context, including a variety of parameters, such as a system-generated message ID, a flag indicating whether the message is persistent, a message priority, the name of the destination queue when forwarding, the name of the reply queue (if any), message type (datagram, request, reply, report), correlation id (to link a reply to a request), priority, expiry time, application-defined format type, code page identifiers (for language localization), context information (to identify the user and application that generated the message), and report options—whether the recipient should confirm on arrival (when it’s enqueued), on delivery (when it’s dequeued), on expiration (if the expiry time is exceeded), on positive action (the application successfully serviced it), on negative action (the application was unable to service it), or on exception. A message that is oversized for the queue manager or application can be decomposed into smaller segments. Moreover, several messages can be assigned to a group, which allows the application to correlate independent messages, such as those that arrive from different sources but must be processed by the same application. An application can request that MQI operations participate in a transaction. Otherwise, by default, each individual MQPUT or MQGET executes outside a transaction, meaning that the operation completes immediately whether or not the application is executing a transaction. WebSphere MQ offers several transaction management options for applications that are running within a transaction. If the only transactional operations are MQI operations, then the transaction can be managed as a local transaction by MQ. If the transaction needs to access other transactional resources, then MQ can play the role of a resource manager under an external transaction manager, such as the Java Transaction API in Java EE. If no external transaction manager is present, then on non-mainframe platforms MQ’s XA-capable transaction manager can coordinate the transactions across MQ and databases. Like many queuing products, WebSphere MQ offers the ability to enqueue persistent and nonpersistent messages in the same queue. Nonpersistent messages are more efficient but less reliable. They do not incur logging overhead and normally are handled in main memory, without being written to disk. Both types of messages obey transaction semantics. However, a persistent message is delivered exactly once, whereas a nonpersistent message is delivered at most once; that is, once (in the absence of failures) or not at all (if there is a failure). Queue forwarding is handled by another component, which is much like an ordinary client that does MQGET from one queue manager and MQPUT to another, though it does have special access to the log for its sequence number management. So if MQPUT has a destination queue name that maps to a remote queue, this component forwards the message asynchronously and transactionally, using an intermediate node if necessary, to the system on which the remote queue exists (see Figure 4.12). The queue forwarding component uses a transaction that’s internal to MQ to coordinate updates to the source and target queues. Figure 4.12. WebSphere MQ Architecture. Messages can be forwarded transparently between queue managers running on different platforms.
An application issues an MQGET to dequeue a message. The queue manager starts a transaction upon receipt of an MQGET verb, dequeues the message from the message queue, and upon the commit from the application, physically removes the message from the queue. If the transaction aborts, the queue manager returns the message to the queue. MQGET supports a blocking option, which blocks the caller if the queue is empty and awakens it when a message is available or a timeout expires. It also supports a signaling option, where the caller can continue executing and is notified when the desired message arrives. Messages can be retrieved in order or by searching for a given ID or key. The queue can also be browsed, to examine the messages of the queue without deleting them. WebSphere MQ also includes a dispatcher that triggers the execution of an application when the first message arrives on a queue, whenever a new message arrives, or when the queue length reaches a predefined threshold. WebSphere MQ supports multiple named queues per queue manager. Each queue manager has the following components: a connection manager for managing the connections between the application and the queues; a message manager for remote communications; a data manager to manage the physical storage of the linked lists comprising the queue; a lock manager for locking queues and messages; a buffer manager for caching data, ordering writes, and flushing data to disk; a recovery manager for keeping track of active transactions in the event of a failure/restart; and a log manager for handling the recovery log. The component names differ slightly in different products. Features of most message queuing systems that support JMS are similar to WebSphereMQ. Oracle Streams AQUnlike most queue management products, which are independent middleware components, Oracle Streams AQ is a queuing facility that is built into Oracle Database. [2] It is built on top of Oracle Streams, which enables the propagation and management of information in data streams, either within a database or from one database to another. AQ can be accessed from most popular programming languages via APIs for PL/SQL, Oracle Call Interface, Oracle Objects for OLE, and extended versions of JDBC and JMS that provide access to Oracle-specific features such as those in AQ. It also offers web-based access via SOAP through the AQ XML Servlet.
In Oracle Streams AQ queues are mapped to a table that can be accessed using the standard types of queuing operations, such as enqueue and dequeue. Since queued messages are stored in a table, they can also be accessed by SQL queries. Oracle Streams AQ is a complete queuing system offering most of the capabilities described in Section 4.5. The enqueue operation takes a queue name, payload, message properties, and enqueue options as input and returns a message ID. Message properties and enqueue options control the behavior of the enqueue operation. For example, using message properties, the sender can control the earliest time when a message is consumed, whether a message is volatile or persistent, the retry threshold after which a poison message is added to an exception queue, whether the operation is transactional, and priority ordering. The entire history of information about a message is maintained along with the message itself. This serves as proof of the sending and receipt of messages and can be used for nonrepudiation of the sender and receiver. The history includes the name of the agent and database that performed the enqueue or dequeue operation and the time and transaction ID of the operation. After the message is propagated to the destination queue, it still includes the message ID of the source message so that the source and destination messages and their histories can be correlated. Stronger nonrepudiation can be achieved by storing the digital signature of the sender and receiver. The following are some additional features worth highlighting:
|
|
|
