Large-Scale Federated Processes

A presentation that I gave at the Stevens BPM day covered the subject of Large Scale Federated Processes. What is a federated process?

It is a distributed process that spans many servers. Distributed process support might be designed and implemented in a very centralized way: for example a single process application with parts of the application deployed to different machines. This allows the process to be much larger that it might be if limited to a single server, but that really is not the point of federation.

A federated process is a distributed process where the different parts of the process are controlled by different people. Not only is the running of the process decentralized, but the control of the design of the process is decentralized as well. Why does this matter?

• One of the biggest challenges of automating processes is “process discovery” where you determine what, for a given organization, is the right process to use. It is common for people in different parts of the organization to disagree. You spend a lot of time coming to agreement with a single process that everyone can agree on. But federated processes avoid this: it allows each organizational unit to have the process that is right for them. You only need to agree on a common interface for invoking the process.
• Another challenge is privacy. Not every organizational unit want to expose how they accomplish processes. Actually they usually won’t admit this, and instead they will make up a process that they want everyone to believe they follow. A federated process does not have this problem since each organizational unit can control their own part of the process.
• Another challenge is keeping a large process up to date when so many different organization units are involved and have to sign off on every change. Federated processes can be more agile, especially in large organizations. With a federated process, when one part of the organization need to make an adjustment to their part of the process, they can do so without having to get all the other parts to review and approve the change.
• Another challenge is appropriate, fine grained, access control. AS the size of an organization gets larger, there is an exponential increase in the effort needed to accurately map access control to the right resources to the right people. By federating the process, the access control to the individual process fragments can be controlled by people within that organizational unit, and much closer to the problem. It is even possible that each process fragment is accessible to completely disjoint sets of people.
• By decentralizing the design of the process, an organization can easily delegate the responsibility for the process far better than if the process had to be designed in a completely centralized, unified representation. It is not surprising that delegation is best for supporting large organizations; it is only surprising that so many technical approaches attempt to centralize the automation (based in my opinion on the naive assumption that everything would be best if everyone did things exactly the same way.)
• Federated processes can be built dynamically, and the really interesting part of this is that individuals can decide at run time which remote subprocess to bind in, according to the individual case. IT is also possible for them to use business rules to determine this. The result is an adaptive federated process which might be far more efficient than otherwise possible.

Federated processes are discussed in the literature under many names. For example, the BPMN specification talks about “collaboration processes” which described the interaction between two processes. B2B interactions can be seen as interactions between fragments within a single federated processes.

I have heard analysts recently use the term “Process Networks” which is the same concept of a collection of individual process fragments connected by information exchange.

In the extreme, we have what I call the “Ephemeral Network”: a collection of systems which are brought together to solve a single problem, connected by a single federated process. An Ephemeral Network can be seen a kind of logical connection which is built as the process executes, lasts as long as the process instance does, and is essentially thrown away when the process is complete. While connected, the people involved can communicate to help achieve the goal of the process.

Federated processes work in a non-homogenous infrastructure. A federated process can employ the latest software for one part of the process, while other parts run on 20-year old mainframes. Keep in mind as well that there is no reason your smart phone can not be hosting part of the federated process as well.

To allow federated processes to work in a non-homogenous infrastructure, a standard protocol is needed, and that is Wf-XML. More on this in future posts, but in the mean time, check out the SlideCast.