Integration Architect – Design Integration Solutions

Event handling is the receipt of an identifiable occurrence at a designated receiver (“handler”). The key processes involved in event handling include:

• Identifying where an event should be forwarded.
• Executing that forwarding action.
• Receiving a forwarded event.
• Taking some kind of appropriate action in response, such as writing to a log, sending an error or recovery process, or sending an extra message.

Keep in mind that the event handler might ultimately forward the event to an event consumer.

Common uses of this feature with middleware can be extended to include the popular “publish/subscribe” or “pub/sub” capability. In a publish/subscribe scenario, the middleware routes requests or messages to active data-event subscribers from active data-event publishers. These consumers with active listeners can then retrieve the events as they are published.

In Salesforce integrations using middleware, the control of event handling is assumed by the middleware layer; it collects all relevant events (synchronous or asynchronous) and manages distribution to all endpoints, including Salesforce.

Alternatively, this capability can also be achieved with the Salesforce enterprise messaging platform by using the event bus with platform events.

See http://searchsoa.techtarget.com/definition/event-handler.

Protocol conversion “is typically a software application that converts the standard or proprietary protocol of one device to the protocol suitable for another device to achieve interoperability.

In the context of middleware, connectivity to a particular target system may be constrained by protocol. In such cases, the message format needs to be converted to or encapsulated within the format of the target system, where the payload can be extracted. This is also known as tunneling.” ¹

Salesforce doesn’t support native protocol conversion, so it’s assumed that any such requirements are met by either the middleware layer or the endpoint.

See https://www.techopedia.com/definition/30653/protocol-conversion.

Transformation is the ability to map one data format to another to ensure interoperability between the various systems being integrated. Typically, this entails reformatting messages en route to match the requirements of the sender or recipient. In more complex cases, one application can send a message in its own native format, and two or more other applications might each receive a copy of the message in their own native format.

Middleware translation and transformation tools often include the ability to create service facades for legacy or other non-standard endpoints; this allows those endpoints to appear to be service-addressable.

With Salesforce integrations, it’s assumed that any such requirements are met by either the middleware layer or the endpoint.  Transformation of data can be coded in Apex, but we don’t recommend it due to maintenance and performance considerations.

See http://en.wikipedia.org/wiki/Message-oriented_middleware.

Queuing and buffering generally rely on asynchronous message passing, as opposed to a request-response architecture. In asynchronous systems, message queues provide temporary storage when the destination program is busy or connectivity is compromised. In addition, most asynchronous middleware systems provide persistent storage to back up the message queue.

The key benefit of an asynchronous message process is that if the receiver application fails for any reason, the senders can continue unaffected; the sent messages simply accumulate in the message queue for later processing when the receiver restarts.

Salesforce provides only explicit queuing capability in the form of workflow-based outbound messaging. To provide true message queueing for other integration scenarios (including orchestration, process choreography, quality of service, and so on) a middleware solution is required.

See http://en.wikipedia.org/wiki/Message-oriented_middleware.

Mediation routing is the specification of a complex “flow” of messages from component to component. For example, many middleware-based solutions depend on a message queue system. While some implementations permit routing logic to be provided by the messaging layer itself, others depend on client applications to provide routing information or allow for a mix of both paradigms. In such complex cases, mediation (on the part of middleware) simplifies development, integration, and validation).

“Specifically, Mediators coordinate a group of objects so that they [do not] need to know how to coordinate with each other….Then, each consumer could focus on processing a particular kind of message, and the coordinator [Mediator] could make sure that the right message gets to the right consumer.”⁴

Process choreography and service orchestration are each forms of  “service composition” where any number of endpoints and capabilities are being coordinated.

In addition, “an orchestration shows the complete behavior of each service whereas the choreography combines the interface behavior descriptions of each service.”⁷

Portions of business process choreographies can be built in Salesforce workflows or using Apex. We recommend that all complex orchestrations be implemented in the middleware layer because of Salesforce timeout values and governor limits (especially in solutions requiring true transaction handling).

Transactionality can be defined as the ability to support global transactions that encompass all necessary operations against each required resource. Transactionality implies the support of all four ACID (atomicity, consistency, isolation, durability) properties, where atomicity guarantees all-or-nothing outcomes for the unit of work (transaction).

While Salesforce is transactional within itself, it’s not able to participate in distributed transactions or transactions initiated outside of Salesforce. Therefore, it’s assumed that for solutions requiring complex, multi-system transactions, transactionality (and associated roll-back/compensation mechanisms) be implemented at the middleware layer.

See http://en.wikipedia.org/wiki/Distributed_transaction.

Routing can be defined as specifying the complex flow of messages from component to component. In modern services-based solutions, such message flows can be based on a number of criteria, including header, content type, rule, and priority.

With Salesforce integrations, it’s assumed that any such requirements are met by either the middleware layer or the endpoint. Message routing can be coded in Apex, but we don’t recommend it due to maintenance and performance considerations.

Extract, transform, and load (ETL) refers to a process that involves:

• Extracting data from the source systems. This typically involves data from several source systems, and both relational and non-relational structures.
• Transforming the data to fit operational needs, which can include data quality levels. The transform stage usually applies a series of rules or functions to the extracted data from the source to derive the data for loading into the end target(s).
• Loading the data into the target system. The target system can vary widely from database, operational data store, data mart, data warehouse, or other operational systems.

While not strictly necessary, most mature ETL tools provide a change data capture capability. This capability is where the tool identifies records in the source system that have changed since the last extract, which reduces the amount of record processing.

Salesforce now also supports Change Data Capture which is the publishing of change events which represent changes to Salesforce records. With Change Data Capture, the client or external system receives near-real-time changes of Salesforce records. This allows the client or external system to synchronize corresponding records in an external data store.

See http://en.wikipedia.org/wiki/Extract,_transform,_load and Change Data Capture Developer Guide.

Long polling, also called Comet programming, emulates an information push from a server to a client. Similar to a normal poll, the client connects and requests information from the server. However, instead of sending an empty response if information isn’t available, the server holds the request and waits until information is available (an event occurs). The server then sends a complete response to the client. The client then immediately re-requests information. The client continually maintains a connection to the server, so it’s always waiting to receive a response. If the server times out, the client connects again and starts over.