Sitecore commerce provides almost all the commerce features out of the box. For example, we get Catalog/product management, pricing management, delivery fee management etc. All the commerce applications we create using Sitecore Commerce get the option to use these features and go to market. But at times, we use other tools rather just Sitecore Commerce to manage our business needs. I will explain this in details.
Recently working on a Sitecore commerce project, we came across a situation where Product Management is done outside the Sitecore Commerce Environment. The client is using a legacy system which manages catalogs and products.
In order to sync those products from the legacy system into Sitecore Commerce Catalogs, we considered certain options
- Exposing an API in SC Commerce which accepts a list of products. When new products are created or there are changes to existing products, the legacy system would pass them to our API
- Exposing an API in the legacy system and poll it every 2 hours let say to get the products.
- Using an enterprise service bus Pattern like Azure Service Bus etc.
Considerations:
Certain obvious concerns for this synchronization process
- Scalability
- Reliability
- Data Integrity
- Time constraint
Commerce API:
It could be scaled and made robust with horizontally scaling the API but it might throttle Commerce Database in scenarios where thousands of products are updated or created at once. And this is very much possible because the Legacy system supports uploading excel sheets with as much new/updated products as possible.
Also, it’s hard to achieve reliability e.g. what happens when a product synchronization fails or what if the system is down for maintenance and products are being updated?
One can say let’s retry but building a retry mechanism is another feature in its own which might be hard to maintain.
Polling the Legacy System for Updates:
It seems to be concerning because polling is not considered a better option and also you have to wait for 2 hours to get the products. Reliability is hard to achieve as well.
Enterprise Service Bus Pattern:
The third option stood out to be the best one in given circumstances. I will explain the technical details of implementation but let me explain first why we wanted to go with an enterprise service bus like Azure Service Bus(ASB).
If you haven’t worked with any enterprise bus service, it’s recommended to read the details of an enterprise service bus. Following diagram explains the gist of it.
Wiki defines this pattern as “An enterprise service bus (ESB) implements a communication system between mutually interacting software applications” ESB works on the notion of publisher/subscriber pattern. As a publisher, applications can push messages to service bus while as subscribers, the application can listen to a message in ESB and act accordingly.
So that’s what we are trying to achieve here; we want our commerce system to interact with an external product management system. For implementation purposes we used Azure Service Bus becuase all our infrastructure including Sitecore Commerce servers live in Azure cloud.
Why:
Because Azure Service Bus pattern would give us
- Reliability: We can really easily implement retry mechanisms in case of failures.
- Scalability: It’s very much scalable solution. How we will see soon
- Controlled Throttling: We can control the frequency of database hits even in case of thousands of products.
- Downtimes: If Sitecore management servers are down for maintenance at any point, the queues/topics would hold the messages until servers are available again for processing the messages.
How:
Get yourself familiarize with the following if you are not across. I will briefly explain them here
- Azure Service Bus: An enterprise bus which has queues and topics. You can listen to or write a message to either a topic or queue
- Service Bus Message: A message can be thought of a json string which can be serialized from a POCO model or deserialized to a POCO model. We can define our custom models for messages. For our example, we created a product model and pushed it to the service bus as a message.
- Queues and Topics: Queue or topic is a data structure which can hold a message and pass it on to its subscriber. As said, it works on the basis of the publisher/subscriber pattern. For instance, in our example here, the legacy system publishes product updates to a topic in enterprise bus while Commerce System would need to have a subscriber to that message. As soon a product update arrives in the service bus, the subscriber would try to update it in commerce database.
- Azure Functions: Azure function is a serverless application that can be triggered by different events. In the Azure portal, we can create an Azure Function App which can host multiple Azure functions. A function is nothing but a console app for instance. We can create a console app and host it inside the Azure Function App. Then we can use this Azure Function app to subscribe to any topic of queue inside a service bus. When a new message arrives
A very nice explanation of all these can be found on msdn. This is how our finalized architecture looks like
