Asynchronous Messaging Architecure as an Integration Solution

Fundamental challenges of integration solutions include most of the fallacies of distributed application
1. The network is reliable
2. Latency is zero
3. Bandwidth is infinite
4. The network is secure
5. Topology doesn't change
6. There is one administrator
7. Transport cost is zero
8. The network is homogeneous

We can summarize the major challenges of integration solutions into to the following

• Networks are unreliable
• Networks are slow
• Any two applications are different
• Change is inevitable
  

When we are trying to integrate two Apps A and B ,there are 4 main approaches to do this and overcome the above mentioned challenges

1. File Transfer : One application writes a file and another application reads from the file. The applications will need to agree on filename,location and format
2. Shared Database: Multiple applications share the same database schema, the integration happens through the database.
3. Remote Procedure Invocation/ Web services: One application exposes one sort of functionality that can be accessed remotely by other application as remote procedure. This could as well be a web service. Communication occurs in real time and synchronously.
4. Messaging: One application publishes a message to a common message channel. Apps need to agree upon message channel and format. Communication is asynchronous.

The issues with approaches in 1,2,3 include

• Apps are tightly coupled.
• Apps A and B need to scaled together
• If A needs high availability then even B needs to made highly available.
• The model of communication involves the applications to be polling for the availability of the data from the other application.

Messaging is more immediate than file transfer, better encapsulated than a shared database and more reliable than a web service/RPI.Advantages of going with the messaging approach include:

• Apps involved in integration (A,B) are loosely coupled .
• Apps can be scaled independently of each other
• Apps can functionally be available independent of each other. App A can be up and functional even when B is down. App A will queue up the data that is required by B , When B is up it start consuming the events from A.
  
• Apps do not have to be build to support peak load as the asynchronous messaging allows the processing to be spread out over time. Asynchronous dampens the load.
• Throttling Ability.
  
• Reliable Communication - reliable and guaranteed delivery
• Platform/Language agnostic integration

Asynchronous Messaging Architectures have in turn have its own challenges.

1. It is a complex programming model. Asynchronous messaging requires developers to work with an event driven programming model.
2. Message sequence issues - Message channels guarantee message delivery but they do not guarantee when message will be delivered. Might require some logic to re-sequence the events at the consuming end if IN ORDER Delivery is a required SLA.
3. Synchronous Scenarios- There might be scenarios where synchronous is required for better user experience. Intelligent workaround schemes might be required to support it.
4. Performance-- Messaging is best suited to keep the system in sync after the initial data replication. Messaging is not suited from the performance aspect to be used as mechanism to replicate data.
5. Limited platform support. This methodology is not yet mainstream and the support and best practices are in limited availability.
6. Vendor Lock-in: Messaging brokers used might result in a vendor-lockin problem. But as along as we stick to to the standards like JMS and stay away from using the broker specific features most of the vendor lock problems can be mitigated.