In a previous article called SharePoint Workflows Aren’t Business Processes, I talked about why it is important to separate the idea of a SharePoint workflow from the business process that workflow supports. It seems to have resonated, but again, my students come to the rescue to point out where I need to elaborate further.
The question one of my students asked was how they could design their business entities or content types to support complete disconnection from the supporting workflow, and I had to think for a bit to come up with a good answer.
Note: I’ve changed the scenario to avoid disclosing details from the student scenario, but the idea and outline remains the same.
In our scenario, we’re looking at an order process with three stages. Order reception, order processing, and order completion. We want to design a system that can maintain this process without tying the process into the orders or the workflow into the process.
Our order entities can be very simple, and to simplify even further, I’ll even drop some of the necessary details, such as the customer relationship.
We need to maintain the following data:
- Order Number (arbitrary identifier)
- Order Date (Date/Time)
- Order Sum (Currency)
- Order Status (Received, Processing, Complete)
- Received By (Person receiving the order)
- Packaged By (Person packaging the order)
- Sent By (Person shipping the order)
Again, this is simplified from a more complex business requirement, but should be sufficient to illustrate our approach.
Designing the Entities
Our first step is to normalize our data and come up with the business entities required to support this process.
Note: Data normalization is the process of structuring our data in a way that removes duplicate data and preserves data integrity.
You may be tempted in this scenario to use a workflow to represent the order process. By doing so, you can use the workflow status to represent the order status. SharePoint will even throw in a workflow status column in your default view when you start a new workflow.
No, not really.
You see, by doing that, we hard link the workflow we build into the order entity, and that order entity and its future now depends on that workflow operation like we think it will operate.
“Fine,” you say, “but I can just create a new workflow if I need to change the process radically”
If you do that, then you’ll break fundamental principles of virtually any development discipline. Your order will have two columns, one representing the old workflow and one representing the new. At the very least, this will confuse your users, but at worst, you create conflicting information. One of the workflows may say “Complete” while the other says “Processing”. Which is correct? You’ll have to tell your users, or hide the wrong workflow from sight.
“I’ll live with that,” you say, “we’re just a small company so telling the five people handling orders how to do this correctly is no issue”.
Well, you still have the issue of data history. If you remove a workflow that holds business data, you are essentially killing the history of those orders. Any orders process from that point in time will have a status but any orders processed prior to your new workflow will not.
Instead, we need to disconnect the workflow completely from the data.
First Stage of Separation
The first approach many take is to simply allow the workflow to update the Order Status. In a SharePoint Designer workflow, this is as simple as setting the value of a field in the current item.
The benefit of this approach is that you get rid of the direct relationship between the workflow status and the order status. If the workflow changes or goes away, you still maintain historical data. If you add a new workflow later, that new workflow can simply update the Order Status field and we’ve disassociated our workflow from our data.
This approach, while certainly better than trying the workflow directly to the Order Status field, is still not enough, though. Our goal isn’t just to separate the workflow from the data, we also want to separate out workflow from the process.
At this stage of separation, there is no distinction between the business process and the workflow. We still suffer from the same problems if our business process changes; we need to completely redesign our workflow to add a third intermediate order stage, for example.
What’s worse, if we cannot support in SharePoint workflows the requirements of the business process, we’re stuck. We either have to change the business process or add manual steps, both of which are very sub-optimal from the business perspective.
Finally, even though we do separate the workflow from the business data, we may still end up with business process information mixed in with the business data.
Let’s say that the order process requires an approval of some sort, maybe even a few approval stages (order approved, shipping verified, etc). In order to store such approval states outside the workflow, we need to add a column to the business data to store the state.
However, by adding one or more approval state columns, we have essentially polluted our business data. Whether an order is approved is irrelevant to the order itself; the approval is part of the process, not the order.
If we later decide we no longer need the shipping verification, for example, we now have a ton of orders containing irrelevant shipping verification data.
If we didn’t think that we needed a shipping verification in the beginning but later decided to introduce it, well, all orders prior to that decision will not have the correct shipping verification values and our reports will not be correct.
This certainly isn’t convenient in a flexible process where we want to ensure that our workflows adapt to changes in the business process.
Second Stage of Separation
The answer is to introduce a new stage of separation.
We’ve determined that we need to separate the business process from the workflow, and the business data from the process. Neither of these links are beneficial from the perspective of ever-changing business processes.
To accomplish this, we need to create an entirely new entity; the order process entity.
This entity will represent the order process, regardless of how the business data looks and regardless of what our workflow does.
In fact, this layer of separation allows us to have multiple and independent workflows, even from different automation technologies. We can combine SharePoint Designer workflows with event receivers with SharePoint 2013 type workflows with manual processing.
The business process entity will be responsible for working with the business data by updating information as required. Our workflows will never touch that business data at all, except to read information relevant to the workflow.
In our relatively simple order process, our order entity may look something like this:
- Order ID (Lookup or reference to an order identifier)
- Order Process Stage (Received, Processing, Complete)
- Order Approval State (Approved, Declined, Pending)
The order process entity will be the only gateway to update or modify our orders. Our workflow will update information in the order process, not in the orders.
This stage of separation requires a bit more work, though, because we need to get the business process to update the business data. However, this is a one-time operation that allows us much more flexibility in how we later work with our business process, so the tradeoff is definitely worth it.
By isolating our access through such an entity, we accomplish several goals.
- We avoid storing superfluous data in the order (such as ever-changing states)
- We have a single interface with which we can modify order data, accessible by any method or technology
- We ensure that changes in the order data does not break our workflows
This approach is very common in regular software development, where developers are accustomed to building separate business layers and data layers, and access the data through the business layer only. Generally, it is considered a much safer approach as it prevents error in our interfaces from ruining our business data.
Note: Of course, this statement is only true if our business logic layer does not break the data 🙂
Further, this approach allows us much more freedom in terms of controlling our process. Let’s say we have one order that falls somehow outside the regular process for example if a customer picks up the shipment directly. In this case, an employee needs to manually change the process for this order only.
If we maintain a hard workflow-to-data model, in which the state of the order is stored in the order, we need to modify the order and possibly break the workflow currently running on that order.
With the two-stage separation model, however, the user can update the process entity without fear of breaking the underlying data or other processes that run.
Of course, this doesn’t apply to manual updates only. Perhaps we want to change our workflow engine to utilize the new SharePoint 2013 engine or we’ve purchased an external engine like Nintex Workflow.
Well, guess what, as long as our new workflow operates on the business process entity only, that’s perfectly fine. We can swap out parts of the business automation system without killing data or ruining reports.
Finally, this separation also allows us to change the business process without ruining data. This happens because there aren’t any links between the business data and the business process either; the process is simply making changes to the data but has no other ties to that data.
If we change or even remove completely the business process, that’s fine, our underlying data remains unchanged.
I hate writing these, but I’d like to summarize a few points.
Separation of data, business logic, and interface is a key principle in modern software development. This model explains how to accomplish this in SharePoint workflows as well, by creating a complete disconnection between the software we use to manipulate data (the workflow), the business logic or process, and the underlying data.
For years, I’ve been trying to get novice and professional SharePoint developers to understand that software development principles apply to SharePoint as well. Equally, bad software development introduces the same risks in SharePoint as it does on any other platform.
The multi-tier model described here will be well-known to experienced software developers, for example those familiar with MVC patterns. However, for many SharePoint developers, who have little or no other software development background, this may seem like a lot of extra hassle without any tangible benefits.
If you think this, however, I would like to take this opportunity to call you an idiot.
Found this article valuable? Want to show your appreciation? Here are some options:
a) Click on the banners anywhere on the site to visit my blog's sponsors. They are all hand-picked and are selected based on providing great products and services to the SharePoint community.
b) Donate Bitcoins! I love Bitcoins, and you can donate if you'd like by clicking the button below.
c) Spread the word! Below, you should find links to sharing this article on your favorite social media sites. I'm an attention junkie, so sharing is caring in my book!