Integration Architect – Design Integration Solutions
Integration Patterns and Best Practices
Each integration pattern follows the same pattern and the pieces can be identified into the parts below.
- Name – The pattern identifier that also indicates the type of integration contained in the pattern
- Context – Overall scenario the pattern addresses
- Problem – The problem or scenario expressed as a question
- Forces – The constraints and circumstances that make the scenario difficult to solve
- Solution – Recommended way to solve the integration
- Sketch – A UML sequence diagram that shows you how the solution addresses
- Results – Explains the details of how to apply the solution to your scenario
- Sidebars – Additional sections related to the pattern that contain technical issues, variations of the pattern, pattern-specific concerns
- Example – An end-to-end scenario that describes how the design pattern would be used
List of Patterns
|Remote Process Invocation—Request and Reply||Salesforce invokes a process on a remote system, waits for completion of that process, and then tracks state based on the response from the remote system.|
|Remote Process Invocation—Fire and Forget||Salesforce invokes a process in a remote system but doesn’t wait for completion of the process. Instead, the remote process receives and acknowledges the request and then hands off control back to Salesforce.|
|Batch Data Synchronization||Data stored in Lightning Platform is created or refreshed to reflect updates from an external system, and when changes from Lightning Platform are sent to an external system. Updates in either direction are done in a batch manner.|
|Remote Call-In||Data stored in Lightning Platform is created, retrieved, updated, or deleted by a remote system.|
|UI Update Based on Data Changes||The Salesforce user interface must be automatically updated as a result of changes to Salesforce data.|
|Data Virtualization||Salesforce accesses external data in real time. This removes the need to persist data in Salesforce and then reconcile the data between Salesforce and the external system.|
|Data Integration||Process Integration||Virtual Integration|
Addresses the requirement to synchronize data that resides in two or more systems. Requires proper information managements.
Includes master data management (MDM), data governance, mastering, de-duplication, data flow design and others.
Addresses the need for a business process to leverage two or more applications to complete the given task. The triggering application has to call across process boundaries to other applications. These require both orchestration (where one is the central controller) and choreography (where applications are multi-participants and there is no central controller)
These require complex design, testing, and exception handling requirements. These are more demanding on the underlying systems because they support long running transactions and the ability to report on and/or manage process state.
The category addresses the need for a user to search, view and modify data that’s stored in an external system.
The triggering application has to call out to the other applications and interact with their data in real time.
This removes the need for data replication
Type – specifies the style of integration
Timing – Specifies the blocking or non blocking nature of the integration
- Synchronous – blocking or near real
- Asynchronous – nonblocking, queue or message-based requests are invoked by a one-way operation
Integrating Salesforce with Another System
|Type||Timing||Key Pattern to Consider|
|Process Integration||Synchronous||Remote Process Invocation—Request and Reply|
|Process Integration||Asynchronous||Remote Process Invocation—Fire and Forget|
|Data Integration||Synchronous||Remote Process Invocation—Request and Reply|
|Data Integration||Asynchronous||UI Update Based on Data Changes|
|Virtual Integration||Synchronous||Data Virtualization|
Integrating Another System with Salesforce
|Type||Timing||Key Pattern to Consider|
|Process Integration||Synchronous||Remote Call-In|
|Process Integration||Asynchronous||Remote Call-In|
|Data Integration||Synchronous||Remote Call-In|
|Data Integration||Asynchronous||Batch Data Synchronization|
Middleware Terms and Definitions
Event handling is the receipt of an identifiable occurrence at a designated receiver (“handler”). The key processes involved in event handling include:
Keep in mind that the event handler might ultimately forward the event to an event consumer.
Common uses of this feature with middleware can be extended to include the popular “publish/subscribe” or “pub/sub” capability. In a publish/subscribe scenario, the middleware routes requests or messages to active data-event subscribers from active data-event publishers. These consumers with active listeners can then retrieve the events as they are published.
In Salesforce integrations using middleware, the control of event handling is assumed by the middleware layer; it collects all relevant events (synchronous or asynchronous) and manages distribution to all endpoints, including Salesforce.
Alternatively, this capability can also be achieved with the Salesforce enterprise messaging platform by using the event bus with platform events.
Protocol conversion “is typically a software application that converts the standard or proprietary protocol of one device to the protocol suitable for another device to achieve interoperability.
In the context of middleware, connectivity to a particular target system may be constrained by protocol. In such cases, the message format needs to be converted to or encapsulated within the format of the target system, where the payload can be extracted. This is also known as tunneling.” 1
Salesforce doesn’t support native protocol conversion, so it’s assumed that any such requirements are met by either the middleware layer or the endpoint.
|Translation and transformation||
Transformation is the ability to map one data format to another to ensure interoperability between the various systems being integrated. Typically, this entails reformatting messages en route to match the requirements of the sender or recipient. In more complex cases, one application can send a message in its own native format, and two or more other applications might each receive a copy of the message in their own native format.
Middleware translation and transformation tools often include the ability to create service facades for legacy or other non-standard endpoints; this allows those endpoints to appear to be service-addressable.
With Salesforce integrations, it’s assumed that any such requirements are met by either the middleware layer or the endpoint. Transformation of data can be coded in Apex, but we don’t recommend it due to maintenance and performance considerations.
|Queuing and buffering||
Queuing and buffering generally rely on asynchronous message passing, as opposed to a request-response architecture. In asynchronous systems, message queues provide temporary storage when the destination program is busy or connectivity is compromised. In addition, most asynchronous middleware systems provide persistent storage to back up the message queue.
The key benefit of an asynchronous message process is that if the receiver application fails for any reason, the senders can continue unaffected; the sent messages simply accumulate in the message queue for later processing when the receiver restarts.
Salesforce provides only explicit queuing capability in the form of workflow-based outbound messaging. To provide true message queueing for other integration scenarios (including orchestration, process choreography, quality of service, and so on) a middleware solution is required.
|Synchronous transport protocols||Synchronous transport protocols refer to protocols that support activities wherein a “single thread in the caller sends the request message, blocks to wait for the reply message, and then processes the reply….The request thread awaiting the response implies that there is only one outstanding request or that the reply channel for this request is private for this thread.”2|
|Asynchronous transport protocols||Asynchronous transport protocols refer to protocols supporting activities wherein “one thread in the caller sends the request message and sets up a callback for the reply. A separate thread listens for reply messages. When a reply message arrives, the reply thread invokes the appropriate callback, which reestablishes the caller’s context and processes the reply. This approach enables multiple outstanding requests to share a single reply thread.”3|
Mediation routing is the specification of a complex “flow” of messages from component to component. For example, many middleware-based solutions depend on a message queue system. While some implementations permit routing logic to be provided by the messaging layer itself, others depend on client applications to provide routing information or allow for a mix of both paradigms. In such complex cases, mediation (on the part of middleware) simplifies development, integration, and validation).
“Specifically, Mediators coordinate a group of objects so that they [do not] need to know how to coordinate with each other….Then, each consumer could focus on processing a particular kind of message, and the coordinator [Mediator] could make sure that the right message gets to the right consumer.”4
|Process choreography and service orchestration||
Process choreography and service orchestration are each forms of “service composition” where any number of endpoints and capabilities are being coordinated.
The difference between choreography and service orchestration is:
In addition, “an orchestration shows the complete behavior of each service whereas the choreography combines the interface behavior descriptions of each service.”7
Portions of business process choreographies can be built in Salesforce workflows or using Apex. We recommend that all complex orchestrations be implemented in the middleware layer because of Salesforce timeout values and governor limits (especially in solutions requiring true transaction handling).
|Transactionality (encryption, signing, reliable delivery, transaction management)||
Transactionality can be defined as the ability to support global transactions that encompass all necessary operations against each required resource. Transactionality implies the support of all four ACID (atomicity, consistency, isolation, durability) properties, where atomicity guarantees all-or-nothing outcomes for the unit of work (transaction).
While Salesforce is transactional within itself, it’s not able to participate in distributed transactions or transactions initiated outside of Salesforce. Therefore, it’s assumed that for solutions requiring complex, multi-system transactions, transactionality (and associated roll-back/compensation mechanisms) be implemented at the middleware layer.
Routing can be defined as specifying the complex flow of messages from component to component. In modern services-based solutions, such message flows can be based on a number of criteria, including header, content type, rule, and priority.
With Salesforce integrations, it’s assumed that any such requirements are met by either the middleware layer or the endpoint. Message routing can be coded in Apex, but we don’t recommend it due to maintenance and performance considerations.
|Extract, transform, and load||
Extract, transform, and load (ETL) refers to a process that involves:
While not strictly necessary, most mature ETL tools provide a change data capture capability. This capability is where the tool identifies records in the source system that have changed since the last extract, which reduces the amount of record processing.
Salesforce now also supports Change Data Capture which is the publishing of change events which represent changes to Salesforce records. With Change Data Capture, the client or external system receives near-real-time changes of Salesforce records. This allows the client or external system to synchronize corresponding records in an external data store.
Long polling, also called Comet programming, emulates an information push from a server to a client. Similar to a normal poll, the client connects and requests information from the server. However, instead of sending an empty response if information isn’t available, the server holds the request and waits until information is available (an event occurs). The server then sends a complete response to the client. The client then immediately re-requests information. The client continually maintains a connection to the server, so it’s always waiting to receive a response. If the server times out, the client connects again and starts over.
The Salesforce Streaming API uses the Bayeux protocol and CometD for long polling.
Outbound messages use notifications() to send SOAP messages over HTTP(S) to a designated endpoint when triggered by workflow or flow.
- A single SOAP message can include up to 100 notifications
- Each notification contains the object ID and a reference to the associated Sobject data
- If object changes occur after the message is sent only the most recent is sent
- If the endpoint is unavailable messages stay in queue until they are successful or 24 hours elapses (messages older than 24 hours are dropped)
- Messages are retried independent of their order in the queue so if a message fails it may be delivered later out of order
- Messages can be delivered sometimes more than once
- Check the notification ID in the notifications delivered to your listener before processing
The metadate needed for outbound messaging is in a separate WSDL that is can be accessed from the setup menu.
- Lock down the client application to only accept requests from Salesforce IP ranges
- Use SSL/TLS
- Send Session ID – only HTTPS is supported for the endpoint URL to secure transmission
- The SessionID included is only for API requests
- If your application endpoints server’s SSL/TLS allows validate using the Salesforce client certificate
- The organization is included in each message
|OrganizationId||ID||ID of the organization sending the message.|
|ActionId||string||The workflow rule (action) that triggers the message.|
|SessionId||string||Optional, a session ID to be used by endpoint URL client that is responding to the outbound message. It’s used by the receiving code to make calls back to Salesforce.|
|EnterpriseURL||string||URL to use to make API calls back to Salesforce using the enterprise WSDL.|
|PartnerURL||string||URL to use to make API calls back to Salesforce using the partner WSDL.|
|Notification||Notification||Defined in the next section, contains the object datatype and its Id, for example OpportunityNotification or ContactNotification.|
Canvas enables you to integrate a third-party application in Salesforce. This means new or existing applications can be made available as part of their Salesforce experience.
Can be used for:
|Authentication||If your application requires authorization, you can implement it by using a signed request or OAuth 2.0.|
|Context||API support that enables you to retrieve context information about the environment in which the canvas app is running.|
|Resizing||Methods that support the ability to resize your canvas app.|
|Canvas Apps in Aura||An Aura component that lets you expose your canvas app in a custom Aura component.|
|Canvas Apps in Visualforce||A Visualforce component that lets you expose your canvas app on a Visualforce page.|
|Canvas Apps in the Publisher||Lets you add a canvas app as a custom action and expand the publisher to include a canvas app.|
|Canvas Apps in the Chatter Feed||Lets you expose your canvas apps as feed items.|
|Canvas in the Salesforce Mobile App||Makes your canvas apps available in the Salesforce mobile app.|
Other options may include web tabs and HTML iframes in a custom component
- Application Integration – you’re a partner, systems integrator or customer with cloud apps you’d like to integrate
- Application rationalization/enterprise desktop – you’re a large organization that has many existing apps you would like to integrate
|Chatter tab||The default dimensions are 800 pixels (wide) by 900 pixels (high). The maximum dimensions are 1,000 pixels (wide) by 2,000 pixels (high).|
|Chatter feed||The default dimensions are 420 pixels (wide) by 100 pixels (high). The maximum dimensions are 420 pixels (wide) by 400 pixels (high).|
|Open CTI||The default and maximum dimensions are determined by the way you set up the custom console component.|
|Publisher||The way you set up the canvas publisher action determines the default height. The default width is 522 pixels. The maximum dimensions are 522 pixels (wide) by 500 pixels (high).|
|Salesforce Console||The default and maximum dimensions are determined by the way you set up the custom console component.|
|Visualforce page||The default dimensions are 800 pixels (wide) by 900 pixels (high). A developer can change these dimensions by modifying the attributes on the apex:canvasApp Component.|