User:Mwgabriel/EnterpriseIntegrationPatterns

Enterprise Integrations Patterns (EIP) are a set of open source programming design patterns that help businesses share data between what would otherwise be large sets of very separate data. These design patterns provide generalized solutions to common problems that occur when trying to integrate disparate pieces of software.

Background

Many corporates and organizations require a multitude of software solutions to run their day to day operations. Businesses operate more efficiently when actions taken in one system are synchronized throughout the entire enterprise. This data integration effort ensures that every part of a business has the most accurate data possible. This is generally achieved through some form of middleware to connect disparate applications into a complete solution. Middleware provides interconnectivity within a system through services such as data transport, data transformation, and routing.

Building integration solutions is often complex and greatly depends on the workings of existing software components. As a result, there is typically many possible right solutions and integration can be more of an art than a hard process. Integration Patterns emerged as a way to capture experts' knowledge of the most effective ways in enterprise integration.

There are four general solutions to share information across a set of integrated systems:

• File transfer: systems write to and read from a file. This solution typically works well when extreme asynchronicity is used and many distributed systems are accessing this information. Systems should read from the file every several hours.
• Shared database: systems that share information share a database. The main advantage of this solution is speed, so systems that are physically close get the most benefit of this system.
• Remote Procedure Invocation: applications call other application’s APIs directly. This is most useful when information must be shared in real time, but it requires a high level of synchronicity.
• Messaging: applications publish information to a shared messaging channel. This is in many ways a lightweight version of the file transfer method that can handle a higher frequency of access and even more interfacing systems.

Problems for Enterprises

Applications do not exist in isolation. Frequently, they must interface with other applications, which may be written in different languages, for different platforms, or provide functionality that doesn’t work well with integrated programs.

In fact, enterprises frequently see the same reoccurring problems:

• Information Portals: visually display data from different systems on the same page.
• Data Replication: different systems need to use the same data (customer's email address)
• Shared Business Functions: different systems need to use the same methods (change_customer_email)
• Service-Oriented Architectures: managing a large set of shared business functions.
• Distributed Business Processes: manage business processes split across multiple systems.
• Business-to-Business Integration: communicating between multiple businesses.

Example

Consider a point-of-sale (POS) machine. Its main functions are to scan and interpret barcodes, reference those barcodes with a table of prices, and accept payment. There are 10 of these in a particular store, and they run on iPads.

This setup introduces problems for the manager because he has to submit requests for more inventory midmorning once per week. He can’t close out each POS machine when there are customers in the store to check how much has been sold that morning, and placing his order without that information could result in a week’s worth of shortages.

He has an inventory database, but it runs on an MS-DOS machine and it only ever gets updated during quarterly inventory checks.

A good solution to this problem would be to integrate the POS system and the inventory database. But how should this be done?

First, let the systems speak in the same language. Add an application to the POS machines and the database machine that translates their proprietary message formats and APIs to a common XML message format. The result of this is that when the POS machines process payment for an order, an XML message is created that shows which items were purchased, as well as how many were purchased.

On the database side, when an XML message is received it is translated into a command that decrements the database entry for each item.

Next, the messages must be able to be sent from the POS machines to the database. Since there are only 10 POS machines, it’s actually completely feasible to maintain 10 open TCP socket connections, so we’ll do that. The XML messages are encapsulated into TCP/IP packets, which guarantee delivery.

This solution demonstrates the following Enterprise Integration Patterns:

• Messaging Systems: setting up a channel that allows all endpoints to publish to.
• Messaging Channels: making sure all 10 POS machines can receive messages from the database correctly.
• Message Construction: allowing messages to actually make changes in the database.

The EIP Solution

The example above is extremely simple, but these integration issues increase in complexity very quickly, and trying to develop a completely custom solution in each case is time-consuming and error-prone. For a store that supports online shopping, one customer's information could be stored in a customer database and used by an authentication system, mailing system, shipping system, and an ordering system. How does a change in a customer's information make it to every single one of these systems?

Enterprise Integration Patterns describe a set of solutions to make an integrator’s life easier. An integrator can choose a standard solution that fits scaling, synchronicity, and infrastructure requirements and then develop scripts that translate information from proprietary systems to the standard. Of the four ways to share information, Enterprise Integration Patterns uses messaging as a base, and then shows many different ways to implement it to solve a specific problem. Specifically, it lists a set of 65 patterns to describe different ways to handle:

• Messaging Systems: setting up a basic channel with basic functionality.
• Messaging Channels: handling more complex channels like multipoint and guaranteeing delivery.
• Message Construction: how messages can be used to invoke methods or perform other functionality.
• Message Routing: making sure messages get to the right system, at the right time, in the right order.
• Message Transformation: adding more information to messages as they pass through.
• Messaging Endpoints: how endpoints interact with the messaging system.
• System Management: debugging and testing the integration.

EIP Applications

The messaging pattern is implemented in various products such as:

• Oracle Service Bus - An Enterprise Service Bus (ESB) implementation by Oracle.
• Spring Integration - A lightweight framework that's built on top of the core Spring Framework. It enables the integration of event-driven architectures.
• Apache Kafka - An open-source message broker project developed by the Apache Software Foundation. Originally developed by LinkedIn.

• Amazon Simple Queue Service (SQS) - A SaaS distributed queue messaging service developed by Amazon. It focuses on web application communication over the internet.
• Windows Communication Foundation (WCF) - A component of the .NET Framework that provides application level communication in service-oriented architectures (SOA)

These applications, with the exception of Apache Kafka, implement proprietary messaging systems developed and optimized for specific enterprise use cases. Apache Kafka is an implementation of a distributed messaging system with focus on high throughput.

There are also many companies that approach enterprise integration without using these patterns.