The requirement for transactional FIFO (first in, first out) guaranteed-ordered transactional receipt of messages between integrated systems continues to arise. In fact, it has long been a staple scenario within the systems integration space. DMS come across it most often in certain integrations like ERP or industry specific scenarios like EDI/HL7. There are many examples out there but a simplistic and obvious one most of us can relate to, is the example of a downstream healthcare system being unable to move a patient from bed to bed, until the patient has first been checked in. Obviously in a message integration situation the patient check-in message must get processed before the bed-move message.
You will know if you have a scenario requiring FIFO as the dependency of order for message processing should be very apparent. What can be harder to determine is what integration options to utilize. When designing a system integration with this requirement you’ll be pondering options and which protocols/transports you can use. While it is possible to roll your own (e.g. a 2-phase commit over REST) we think you should work to exhaust options before you go there. Adding more custom code that you could avoid means additional unnecessary project risk and lifetime support costs. Below we share a quick table to help review options:
Protocol / Transport | Platforms Supported | Notes/Comments |
WS-* SOAP protocols Specifically WS-ReliableMessaging /WS-AtomicTransaction | Open Spec | WS-* popularity has declined significantly with advent of REST-based options. While it is undoubtedly more complex, unless you want to implement custom 2P commit over REST it may be a good choice for some HTTPS based integrations. Vendor frameworks like Microsoft’s WCF can reduce some of the programming effort. |
IBM MQ Series | Windows Linux AIX (UNIX) IBM i (midrange) IBM z/OS (mainframe) | The granddaddy of queuing. If you are integrating an IBM supported/licensed system, then this is a good choice. |
Microsoft MSMQ | Windows Only | MSMQ has been around since 1997 and maintains good support and robust messaging protocols, including options for clustering its service to enable high-availability. The downside is that it is a Windows only option both for queue hosting and respective programming frameworks. |
RabbitMQ | Linux/Windows/MacOS | RabbitMQ open sourcing and wide list of options for programming frameworks have made this popular however if you’re dealing with a scenario of multiple send/receives then you’ll need to implement additional logic. |
Azure ServiceBus ~OR~ Amazon SQS | Cloud hosted Queues with programming frameworks for Windows/Linux/MacOS | Unless you have an aversion to public cloud services, we’d highly recommend either of these two. Both provide price competitive queue storage services and wide array of programming framework options. At DMS we spend a lot of time working with Azure ServiceBus and have found it to be foundational to many of our clients’ integration solutions. |
Did we miss any options you’ve had success with? Remember our focus here is on discrete asynchronous message level integration going from application to application and with relatively modest payloads (< MB). We did leave out streaming options like Kafka and Azure EventBus which maybe more appropriate for solutions handling big data scenarios.
At DMS Group we do a lot of integration utilizing Microsoft tools like Power Automate, Azure and BizTalk. Most of the options above have adapters/connectors that can be configured and, where they don’t, custom code can enable you at a preference to heading off-road with any home-grown FIFO protocols.
Lastly, do not forget to review your preferred option against requirements for Performance/Availability/Scalability as well as Security. Does it hold up? FIFO has its own overheads and challenges especially around requirements like scaled throughput. It is always a good idea when you start out on these use cases to begin with validating the need for FIFO. Often, it’s obvious but you will kick yourself if you did not have to invest cycles on this additional complexity. Once you know that your message integration scenario needs FIFO, all that remains is to navigate options that provide your optimum solution within its constraints. Remember if you need help navigating or implementing these kinds of scenarios you can always reach out to DMS Group. It’s what we do, and we’d be happy to assist. Happy integrating!
Comments will load here
Be the first to comment