Using different IT components for different tasks is a common practice. But as business functions expand, companies may become overwhelmed by lots of disjointed tools that can’t share data and work together. That’s when system integration comes to the rescue.
In this article, we'll examine existing methods and technologies to meld separate pieces of software and hardware into a single ecosystem, touching on key integration steps and the role of a system integrator.
What is system integration and when do you need it?
System integration is the process of joining software and hardware modules into one cohesive infrastructure, enabling all pieces to work as a whole. Often referred to as IT integration or software integration, it results in the following advantages.
Increased productivity. Integrated systems allow for centralized control over the daily processes which adds to the efficiency of the entire workflow. A company gets more work done in less time due to the fact that employees can use all apps and data they need from one entry point.
More accurate and trustworthy data. Data is updated across all components of the system simultaneously, keeping all departments on the same page.
Faster decision-making. Data is no longer scattered across siloed storages. So, to perform analytics, you don’t need to manually download and export it to the centralized repository. Instead, with a holistic view of all information, you can extract useful business insights to make good decisions more rapidly.
Cost-effectiveness. More often than not, system integration comes at a lower cost than replacing all disjointed parts with a new single system. Not to mention the tricky process of implementing new computer infrastructures.
Below, we’ll list the most common types of system integration that meet different business needs.
Legacy system integration
Goal: integration of modern applications into existing outdated systems
Many organizations use outdated software to perform their core business functions. It cannot be removed and replaced with more modern technology as it is critical to a company's day-to-day workflow. Instead, legacy systems can be modernized by establishing a communication channel with newer information systems and technology solutions.
Goal: unification of different subsystems inside one business environment
While growing, companies incorporate more and more enterprise applications to streamline their front- and back-office processes. These applications often share no points of convergence and accumulate huge volumes of data separately. Enterprise application integration (EAI) brings all functions into one business chain and automates real-time data exchange between different applications.
Example: creating one ecosystem for accounting, human resources information, inventory management, enterprise resource planning (ERP), and CRM systems of a company.
Third-party system integration
Goal: expanding functionality of the existing system
Integration of third-party tools is a great option when your business needs new functionality but can’t afford custom software development or just has no time to wait for features to be built from scratch.
Example: integrating an existing application with online payment systems (PayPal, WebMoney), social media (Facebook, LinkedIn), online video streaming services (YouTube), etc.
Business-to-business integration
Goal: connecting systems of two or more organizations
Business-to-business or B2B integration automates transactions and document exchange across companies. It leads to more efficient cooperation and trade with suppliers, customers, and partners.
Example: connecting a retailer's purchasing system to a supplier's ERP system.
Whatever the situation, the main objective of system integration is always the same — to put the fragmented and divided pieces together by means of building a coherent network. Let's look at existing technologies and architecture models that make the integration magic happen.
Ways to connect systems
There are different ways to achieve connectivity between separate systems. We'll review the most common “connectors” in a nutshell.
Application programming interfaces (APIs) provide the most common and straightforward way to connect two systems. Sitting between applications and web services, they enable the transmission of data and functionality in a standardized format. Most online service providers — from social media to travel platforms — build external APIs so that clients can easily link to their products.
Middleware is the hidden software layer that glues together distributed systems, applications, services, and devices. It handles different tasks such as data management, messaging, API management, or authentication. Cloud middleware can be accessed via APIs. In turn, an API gateway can be considered a type of middleware between a collection of services and systems using them.
Webhooks, also known as HTTP callbacks are real-time messages, sent by one system to another when a certain event happens. Say, accounting software may receive webhook notifications about transactions from payment gateways or online banking systems.
EDI — the abbreviation for electronic data interchange — is the exchange of business information in a standard electronic format, that replaces paper documents. EDI generally happens in two ways: via a value-added network (VAN), in which a third-party network is in charge of data transmission, or direct connections through the Internet.
All these connectors can be mixed and leveraged when building complex system integrations. If companies have unique needs and requirements for system integration, it's better to opt for custom built solutions whether they are APIs, webhooks, or middleware.
How to approach system integration
System integration is multifaceted and can be approached through different architectural models, depending on the number and nature of components that need to be connected.
Point-to-point model
Point-to-point integration (P2P) is the architectural pattern in which every system is directly connected to all other systems and apps it needs to work in tandem and share information with. This model can be realized via APIs, webhooks, or custom code.
With a point-to-point connection, data is extracted from one system, modified or formatted, and then sent to another system. Each application implements all the logic for data translation, transformation, and routing, taking into account the protocols and supported data models of other integrated components.
The point-to-point (star/spaghetti) integration architecture.
Pros and cons: Among the main advantages of point-to-point integration is the ability of an IT team to build a small-scale integrated system quite quickly. On the flip side, the model is hard to scale and the management of all the integrations gets very demanding when the number of applications grows. Say, to interconnect six modules you need to perform 15 integrations. This results in the so-called star/spaghetti integration.
When to use it: This approach suits companies that don't have complex business logic and run their operations on just a few software modules. It is also a perfect option for businesses aiming at connecting to SaaS applications.
Hub-and-spoke model
The hub-and-spoke model is a more advanced type of integration architecture that addresses the issues of point-to-point and helps to avoid the star/spaghetti mess. The connections between all subsystems are handled by a central hub (message broker), so they don’t communicate with each other directly.
The hub serves as a message-oriented middleware with a centralized integration engine to translate operations into a single canonical language and route messages to the right destinations. The spokes (adapters) connecting the hub to the subsystems are managed individually.
The hub-and-spoke integration architecture.
Pros and cons: As opposed to P2P, the model brings quite a few benefits to the table including higher scalability. Since every system has only one connection to the central hub, things get better in terms of security and architecture simplicity. However, the centralization of the hub can be a weakness in such a model. The whole infrastructure is dependent on the single integration engine which can become the key bottleneck as the workload increases.
When to use it: The hub-and-spoke model is widely-used in e-commerce, financial operations, and payment processing. Besides, it’s a preferable architecture for highly regulated industries that face significant security risks.
Enterprise Service Bus (ESB) model
The ESB architecture involves the creation of a separate specialized subsystem — an enterprise service bus — that serves as a common user interface layer connecting other subsystems.
The ESB can be described as a set of middleware services that glue multiple systems, serving as a messaging backbone. In contrast to hub-and-spoke with a single centralized integration engine, in ESB, each system is supplied with a separate integration engine and an adapter that translates a message into the canonical format and back into the destination supported format. Initially designed to bridge complex internal systems of large enterprises, ESBs can work with cloud services too.
The enterprise service bus integration architecture.
Pros and cons: One of the best things about ESBs is that each subsystem is decoupled by a “messaging bus,” so it can be replaced or changed without affecting the functionality of other subsystems. This plays in favor of high scalability. Also, such projects are reliable and quite easy to design. As far as the cons, maintenance and troubleshooting get more complex with the spreading of integration tasks across the systems.
When to use it: An ESB model is an optimal way to implement large projects such as enterprise application integration (EAI), allowing them to scale when needed. It’s a good fit if a company needs to bring it together on-premises.
Deployment options for integrated systems
Though we've described the three most common architectures, the reality is more complicated than that. A single integration approach may not be sufficient anymore — especially when speaking of enterprises leveraging a wide range of technologies. Often, companies have to combine all three patterns within one ecosystem, using various types of middleware and API layers between IT components. Luckily, the growing number of cloud platforms offer their services to handle complex integrations. Below are two popular deployment options to consider.
Integration platform as a Service (iPaaS)
Integration Platform as a Service is a set of cloud-based integration solutions mostly used for building and deploying integrations in the cloud.
As an all-inclusive service, iPaaS couples systems, processes, and data, making them accessible via a single user interface. It presents a library of pre-built connectors that enable disjointed applications to communicate with one another despite where they are hosted. iPaaS handles data transformation and shipping from and to applications.
Simplified illustration of possible iPaaS connections.
Pros and cons: iPaaS is advantageous in tons of ways. It’s flexible, multi-functional, and scalable. With iPaaS, integration activities are automated, facilitating the connection of systems and databases deployed in any environment and enabling faster project delivery. As far as the shortcomings, there might be security concerns as with any public cloud.
When to use it: iPaaS is great for real-time applications and it enables a variety of integration scenarios including enterprise application integration (EAI), data integration, cloud integration, B2B integration, API management, Internet of Things integration, and many more.
Hybrid integration platform (HIP)
A hybrid integration platform or HIP is a more universal version of what iPaaS offers. It is an array of integration software providing built-in capabilities to make on-premises and cloud-based solutions work as a single unit.
Integration platforms act as the middleware between legacy systems that run on physical hardware, applications and databases in a private cloud and systems running in a public cloud. Such platforms require minimum configuration. They interface and integrate with any systems by using two main components — protocol connectors to handle communication protocols such as HTTP, TCP, JMS, etc., and message formatter to handle various data formats such as JSON, XML, etc.
Hybrid integration platform diagram.
Pros and cons: HIPs provide different perks from managed APIs and cloud offerings to reusable integration templates for common use cases. With this model, companies can count on high security and the reduction of integration costs and time as well as maintenance efforts. At the same time, integration platforms haven’t reached the maturity phase, which is why it may be difficult to pick a suitable out-of-the-box solution.
When to use it: The main focus of HIPs is placed on the digital transformation of legacy systems. This is a great framework for organizations needing to enable connectivity between on-premise and cloud solutions.
Key steps of system integration
Companies can automate and gain full transparency of their business operations by joining enterprise data and systems together. If you want to conduct effective integration and return your investments in no time, there are a few crucial steps to take.
Steps to take to implement system integration.
Planning and feasibility analysis
Every integration process starts with the assessment of systems to be integrated and mapping out a realistic strategy. Draw an accurate picture of your current software and its tech specifications and define all integration requirements. Also, determine the scope of your integration project, its schedule, and cost. It's a good idea to include all possible risks in your plan and ways to handle them.
Architecture modeling
This step involves choosing one of the common models we mentioned above or designing a custom architecture to meet your specific needs. You also need detailed blueprints of how systems will communicate with other comprehensive systems. The most labor-intensive, this phase is of great importance as it outlines the integration model, methods, and process in general. Preliminary and physical designs are created at this stage.
Implementation
The new integrated system is thoroughly tested to make sure that all modules seamlessly interact with one another without losing any data during transmission. After that, it can be implemented and presented to users. It is recommended to keep the implementation stage short to avoid challenges relating to possible changes in the integration process. Agile project management can be applied during and after this phase to help a company adjust to the changing landscape of integration systems.
Maintenance
You shouldn't neglect routine maintenance on the system. Scheduling performance diagnostics to make sure that all modules function flawlessly and no errors occur is advised.
What is the role of system integrators?
You may have the best integration technologies, but, without human expertise, it won't bring you much value. Instead of designing and implementing the integration project on your own, you can opt for the services of system integrators that have all required resources and experience.
A system integrator (SI) is an individual or company that helps clients bridge scattered computer subsystems from different vendors and ensures those subsystems function in accordance with one another. System integrators perform various tasks such as planning, regulating, testing, and often maintaining computer operations.
The services of system integrators can be your best bet in case you want to save time and effort. Instead of searching for and communicating with vendors yourself, you hand the project over to specialists who already have all the needed connections and know how to best approach system integration in your case. The most beneficial thing of having a system integrator is the expertise and resources they offer that customers often lack internally.
