Enterprise messaging is the technology backbone of communications for applications and systems within and between organizations. Both its importance and its complexity are growing as organizations increasingly have to provide real-time responses to business customers and consumers as well as their own business professionals who support them and their internal supply chains. The variety of use cases for enterprise messaging also is growing rapidly, expanding to the Internet of Things (IoT) market of sensors and devices including wearable technology; to new generations of applications and services for consumers and customers; to cloud computing and the shift to platform or infrastructure as a service (PaaS or IaaS); and to real-time big data and analytics. All of these innovations will enable the these types of transformation to digital business that is impacting organizations around the world.
Enterprise messaging is closely related to message-oriented middleware (MoM) technology that consumes and publishes messages as part of applications and services as well as to other middleware and integration technologies. Through acquisitions and partnerships many middleware and some integration technology providers have blended their interfaces with enterprise messaging to 
ensure they are part of real-time networks across IT and business. As for their customers, our previous research into operational intelligence found that almost half (48%) of organizations evaluated alternatives in their messaging middleware throughout 2015. Our new research into The Internet of Things and Operational Intelligence, currently under way, is assessing the role of enterprise messaging in the changing technology landscape.
Today’s enterprise messaging is advancing beyond the design of message-oriented middleware; those queuing or brokering approaches are struggling to keep pace with millisecond and even faster transmission of data across internal and external fiber networks that in some cases needs to support guaranteed delivery like that found in financial trades and other commerce. The decades-old technology in existing enterprise messaging systems and MoM spans multiple generations of brokering and queuing approaches, a range of protocols and standards, and dozens of vendors. This technology enables exchange and transport of messages between applications and systems. The messages can be processed asynchronously or synchronously, in the publish and subscribe method and in secure encrypted formats. The variance across approaches ranges in level of latency from low to very low, which performs in subsecond times between points across a network.
In recent years the necessity of processing messages fast has placed extreme pressure on message queue and broker approaches that were not designed to meet such low latency demands or efficiently process the huge volumes of data found in the emerging generation of enterprise and consumer-focused applications and services. The diversity of these new systems challenges the most experienced enterprise architects, who have to rationalize complex legacy environments and determine where to simplify them to become more cost-effective and in some cases more secure. These challenges push many organizations to reassess their architectures for messaging and examine alternative approaches.
The middleware technology approach to messaging is challenged further by the externalization of enterprise systems from on-premises to private and public cloud computing. As middleware-related markets have consolidated, the transition to platform or infrastructure as a service has necessitated new middleware for enterprise architectures as messaging and APIs are becoming more virtual, in what is called microservices. Simultaneously, businesses demand more real-time functionality as they discover that their underlying transaction and information architectures are ineffective for rapid communications and processing of data to meet new requirements.
Reliability, performance and scalability of the messaging technology and the infrastructure and resources required to support it are focus points of re-evaluation for organizations. Part of that review involves addressing the requirement that messaging must interface to the middleware or PaaS that is being used to develop new applications. For many organizations the messaging API they use depends on the middleware they’re using for applications; it may be provided by IBM, Microsoft, Oracle, Red Hat or another vendor that has a stake in binding the organization’s infrastructure to its technology. At minimum these providers influence developers to look first at the messaging that is part of their middleware or PaaS. Further complicating the issue, messaging between applications and systems is not controlled by one vendor, and depending on the history, biases or preferences of individuals and the use case, evaluations may not make it to the RFP or RFI stage. That could be risky for organizations that are not keeping up with the technological and architectural changes that have occurred.
In addition, standards often play a role in selecting technologies. One recent standard is Advanced Message Queuing Protocol (AMQP), which evolved from financial markets and operates across the wire on TCP to facilitate a robust approach to messaging. Another, Message Queue Telemetry Transport (MQTT), is being used for IoT and connecting machines to messaging on the Internet. Other approaches such as Java Message Service (JMS) have gained traction through enterprise familiarity with Java and middleware such as Red Hat’s. Even the cloud computing offerings from Amazon, IBM, Microsoft and Oracle have added integrated messaging to their environment.
Architecturally, organizations are also examining microservices, which embed independent services that bind into applications, typically through an API that separates logic and communications from messaging. This technology approach provides a pattern for development and does not preclude the interface to APIs and messages that communicate with enterprise messaging.
The new focus on enterprise messaging has organizations examining their legacy approaches for messaging middleware. Users of IBM and Tibco, for example, have had to increase spending on hardware and resources to scale out and support the reliability required for their growing messaging volumes. It is no surprise that our research has found that messaging middleware is insufficient in almost one-quarter (23%) of organizations that want to use it for other applications and tools in the enterprise. This lack of overall reliability places pressure on the management and monitoring of servers to ensure that they scale adequately; many struggle to meet the requirements for very low latency and guaranteed messaging. Many organizations feel forced to re-evaluate their architecture and approach to enterprise messaging to find one that is more cost-effective, more efficient and more reliable. Some organizations are working with commercialized open source messaging approaches such as Apache ActiveMQ, RabbitMQ and StormMQ.
The next generation of enterprise messaging now in the market includes virtualized messaging across cloud-based platforms and the internet and the use of appliances and related tools for networking. Enterprise messaging appliances are attractive because they are able to handle extremely large volumes of messaging but can be managed by software are already in operation at data centers and network operations centers. These appliances can be placed into the data center or hosted on the internet in a distributed computing approach. One such enterprise messaging appliance provider is Solace Systems, which has been operating its appliances for years in large global deployments. More recently IBM and Tibco introduced appliances into the market to address the shortcomings in their software-based approaches, which, as I have already mentioned, are challenging and costly for companies to maintain.
The advances in intelligent communications across devices and machines demand reliable throughput, which enterprise messaging can provide. Messaging appliances and virtualized messaging are part of the emerging future in which digital technologies operate in real time and support how consumers and business operate. I will write more about these tools in 2016. If you have not examined your organization’s messaging and infrastructure, look into enterprise messaging to better understand what you will need to be successful in the new digital business that is interconnected and happens in real time.
