Go Flow

In my last post, I introduced the concept that there are two predominant views of BPM. One view is that of the Automators, who are creating business processes which replace humans by doing the same things that had traditionally been done manually. The other view is that of the Facilitators, who are creating BPM processes to involve actual people in processes can not and probably never will be fully automated. Both groups see themselves as making “human processes”, both groups create BPMN diagrams filled activities and gateways.

To illustrate the differences, consider a very simple process that represents a writer writing an article for a reputable publication, and an editor which must reviewing that article to make a decision on whether the article is suitable for publication or not. This is not a pretend process, this is a real process which when used in a real organization provides real value. A Facilitator would draw it like this:

That is all you need at a human activity level. In order to explain this process to a person who was to participate in this, the two nodes would be enough: Betty will write the document, and when completed, George with review the document. Both Betty and George understand their role in the process, and can go about their work.

An Automator is likely to look at this process and say it is ridiculously simple. People are not just sitting around like servers handling requests like “Write Article”. Automators are looking specifically at what the server must do, to enable the process. For example, the server does not write the document, but people do. What the server does is

• determine who should be the writer this time
• put the task on a task list that the user can browse
• some users also want an email message prompting them
• receive the email message back with the document in it
• remove from task list
• determine who should be the editor this kind of article
• put the task on a task list that the editor can browse
• some users also want an email message prompting them
• receive the email message back with the decision in it
• remove from task list

It gets more complex when you include features which are commonly included in a human BPM system. First of all, it is not always possible to know (with the data available to the system) who the write editor is. It might on the surface be an article about vacation travel, slated for the Travel section, but it actually is a news story about smugglers pretending to be on vacation. Many good human BPM systems give you the ability to reassign the activity to another person. So you need to add that there are two possible emails that might come back: one with the results, and the other saying “Joe should review this document”.

Second area of flexibility is the initial assignment of the writer. A good human BPM system will have a way to offer a task to a group of people, and people can accept the task, and bring it to them, informing the others that the task has been taken. This now makes the task list more like:

• send an email offering the chance to write an article
• receive email from first person responding that they will take the task
• send email to all the original group informing them that the task has been taken
• put the task on a task list that the user can browse
• send an email (some users)prompting for the actual writing
• receive the email message back with the document in it
• remove from task list
• determine who should be the editor this kind of article
• put the task on a task list that the editor can browse
• some users also want an email message prompting them
  • receive the email message back with the decision in it
  • OR receive an email sayting to reallocate to a different person, loop back a couple steps.

  remove from task list

Most good human workflow systems have a way to remind the users that they are approaching a deadline. A typical two step reminder is that the first reminder is to the person performing the task, and if that does not work, then an “escalation” reminder which goes not only to the performer, but also to a group of others who will look into the problem and see why it is no progressing. This adds a few more steps:

• send an email offering the chance to write an article
• receive email from first person responding that they will take the task (presumably you ignore any later email message to this effect, and drop them on the floor)
• send email to all the original group informing them that the task has been taken
• put the task on a task list that the user can browse
• send an email (some users)prompting for the actual writing
  • send reminder email to performer
  • send escalation email to wider group of people
• receive the email message back with the document in it
• remove from task list
• remove from task list
• determine who should be the editor this kind of article
• put the task on a task list that the editor can browse
• some users also want an email message prompting them
  • send reminder email to performer
  • send escalation email to wider group of people
• receive the email message back with the decision in it
  • OR receive an email sayting to reallocate to a different person, loop back a couple steps.
• remove from task list

Such a process might be diagrammed a bit like this:

I am sure there are MANY problems with the above diagram. If you would like to draw this process correctly, please do so and attach it to a comment (or email it and I will attach). I am not sure that a complex condition node can “loop back” to a previous task (that is what I want to do but it might not be allowed by BPMN guidelines) I am not sure how to have one timer go, and then a later time, and still allow that if the response comes before one or the other the process continues. I think I need to use a subprocess box to get that to work correctly. in any case, the exact way that one would draw this depends upon details of the system you are implementing on. For example, how did you recognize that the email was a completion email, or a request for reassignment. And what is missing from this is the requests that the user makes to the system to retrieve the current worklist. But this diagram is good enough for me to make a point.

The Automator is happy, because this diagram represent a more “truthful” description of what the system has to do. It makes explicit the various email messages that must be sent and received. What we have done here is to move away from the representation of what people do, and moved instead to a representation of what bits and bytes need to be sent and received by the system. The activities describe what the system does, not what the people do. That is great, if you are a programmer.

The detailed diagram has lost the representation of the fact that there are actually only two human activities being performed. You can’t really see what the people are doing, without getting lost in the details of what the system has to do to communicate to the people. The diagram could be simplified by hiding a lot of the detail in subprocesses, so you have only the two nodes at the top level and drill down to see the detail.

The point is not that “Automator” processes are more complex. My point is that with a human BPM system, the “Facilitator” gets a lot of capability built in “for free”. Simply draw an activity, and assign it to a person, and the human BPM system at that point does multiple things to inform that person that they have something to do. There can be built in mechanisms to negotiating who does that activity. There can be built in mechanisms for reassigning the activity. There can be built in mechanisms for reminder messages. And finally there is a built in mechanism to recognize when the person declares that they are done. All of these capabilities are common across any task you wish to give to a human. Why should you have to re-invent how to notify the user for every process? The Facilitator wants to focus on the human tasks, not the bits and bytes that the system sends and receives. The Facilitator learns that all these capabilities are built into an activity node.

But what to do? BPMN does not have the concept of assigning a person to an activity. BPMN was designed primarily (but not exclusively) by people with the Automator viewpoint. Much time was spent on how to translate to BPEL, which is exclusively Automator viewpoint. The BPMN group kept their options open, to allow for a Facilitator view implementation, but there are contradictions if you follow this path. This causes people to look at an implementation of BPMN, and way “that is not standard”. Indeed it is not: a Facilitator diagram does not look like an Automator diagram.

WfMC experts are again presenting the standards tutorials at two venues in Europe.

1. Poznań Poland.  Due to a big upsurge in BPM use in Poland in the past couple of years, we were invited to present a day and a half on Oct 8 and 9.  Here are links for overview and registration.

2. Paris La Défense, France.  We have long had membership in the coalition in France, it is nice to finally hold an event there on Oct 10.  Here is a links for  registration (in French).

WfMC members take note:  will be holding the fall meeting of the WfMC in Paris, hosted by TIBCO, on Oct 11 and 12.

I spoke at the e-gov Enterprise Architecture Conference in Washington in September and was asked an intriguing question by a visitor.  We were talking about the relevance of BPMN, as well as quality of support for BPMN.  What distinguishes human work from what can be automated?  As a reference, I used a very simple two step process which I often use in my presentations – Account Application – which is essentially a two step process once you remove the automated activities.  The second step is a “Decide whether to Create Account”.  His response:  “Would that be represented by a decision node?”

I was stopped in my tracks.  In this case it is NOT a decision node, but why not?  Sometime a conditional branch gateway is called a “decision node” since the server “decides” whether to go one way or another.  In reality, those nodes don’t actually make decisions.  The real decisions are made long before you get to this gateway.  Why then do we call them decision nodes?

Determining whether to create an account is a real decision.  If it is important to select the people who hold accounts, if there is a reason that some people would not be suitable to hold an account, then you must have some sort of process to clear applicants, and someone must decide whether to give out the account or not.  The decision, in the extreme case is easy: a well known wealthy local resident is an easy decision; a criminal convicted multiple times for monetary fraud is another easy decision.  The easy decisions might be coded into rules, but the rules will not cover 100% of all cases.  So therefore ultimately there must be a person to fall back on to make the tough borderline decisions.

The node he was speaking of is also called a branch point gateway.  The process branches to one of a number of possible directions at that point based on information available.  The branch might depend upon rules, but is any real decision being “made” at this point?  Wasn’t the “decision” made long ago?  Consider an example like: “An applicant with credit ratings below a certain value will not be eligible for loans over a certain magnitude.”  Without getting deeply philosophical about it, wouldn’t you say the “decision” is made at the time that this rule is drawn up?  After that time, the execution of the rule simply branches the process based on that former parameterized decision in a completely mechanized way.  This calls into question the real meaning of “decide”.

Do we call them decision nodes because we anthropomorphize the computer system that is executing the process?   You can imagine the server playing a role in the process, and doing things on our behalf.  After all, when we get to that step that says to email all the participant, WHO is it that is doing the emailing?  It is the server doing the emailing, right?  Therefore the server is playing a role in the process as an actor, as if it were a stand-in for a human.  I would agree that it is carrying out tasks that it is instructed to do, but is it really making decisions for us?

I one heard someone make the point that military personnel are trained to understand their responsibility in making the decision to launch weapons.  The radar may say clearly that there is an enemy plane flying a dangerous route, but it is the commander who decides to strike.  There may be rules that identification of a particular threat dictates a particular response, and sensory equipment may indicate the presence of  such a threat, it still it is still the responsibility of a person to interpret that sensory data, and take action.  Business decisions, while never quite as final as some military combat decisions, are and should be the same way.

A real decision is the kind that is not easy to make.  A collection of information will be presented to a person who is responsible for the success of the business, and a decision is made based on many factors that are difficult to quantize.  In some cases, there may be a gut feel.  Malcom Gladwell wrote a great book titled “Blink” which talks about the “native intelligence” which can tap into so much additional information in parallel, and which our conscious stream of thought does not have time to be aware of.  An experienced account manager will be able to sum up a case relatively quickly, and decide whether it is worth risking extending an account to this person.  (Interestingly, this same person may then come up with an explanation for why they made the decision, but that explanation might or might not actually be the real reason, but I digress to far….)

Let’s wave the magic wand for a moment, and imagine what it would be like if we could reduce all decision making to rules.  Then we might have the “one button office”.  This is the idea that you go to work every day, and in the middle of your desk is a single button, and you press it, and the system does all your work for you.  This was the goal of the office automation movement in the late 70’s and early 80’s.  The problem is that as soon as you isolate all the rules, there is someone who figures out how to play those rules, and you have to go and adapt the rules for this change in the environment.  While rules are very important in relieving us from the tedium of routine case assessments, there will never at time in the future be a point where we can stop adjusting and modifying the rules, and there will always be edge cases for which it will be quicker and more efficient to have a person simply “decide”.

A great example of a task that will never be automated is that of deciding which advertisement to run in the next promotion series.  More examples of non-automatable tasks include: the decision to merge two different companies together; the choice of which campaign slogan to use; the decision to hire or fire a team member; whether to run an article in a newspaper.  Let’s call these for a “Business Decisions” because they are the kind of real decisions that must be gotten right in order for the business or an organization to thrive.  Often business decisions depend upon a complex web of current events, legal and ethical constraints, as well as imperfectly known evidence.

Business Decisions are made at nodes which are activities which are assigned to humans.  A person need to perform the task of making this decision.  Those are the true decision nodes, not the branch gateways.

Let me take this a bit further: we know that there are at least two distinct “camps” in the BPM field: those that want to support human work (”Facilitators”), and those that want to automate work that was previously done by humans (”Automators”).  The Automators are always looking for example that can be completely automated.  For example, 30 years ago if I wanted to know my bank account balance, I would have gone to visit the bank and asked an attendant.  Now, of course, I access a web site, and it is 100% automated.  In that situation, all of the “decisions” that had been done by the attendant can be reduced to branch gateways, and you can think of gateway taking the place of the decision that the person used to do.  Facilitators, on the other hand, are looking for example processes which include tasks that can never be fully automated.  For example, a hiring process which includes deciding whether a particular person makes a good fit for the team.  The hiring decision is made by a person at an activity node, and this controls the flow later in the process.

The BPMN specification does in fact call branch gateways “decisions” and this term is in widespread use.   This shows that the BPMN is strongly grounded in the ideas of the Automators.  Facilitators will also use the term “decision” when talking about a conditional branch gateway, but a Facilitator will know that real Business Decisions are not actually made there.  At best, the conditional branch gateway is directing the flow of the process in response to a Business Decision made by a person in a previous step.  Perhaps there is clarity if the Facilitators would talk about a “Business Decision Activity” where a person is given the task of making a decision.  This also reflects on of the fundamental difference: Automators like to draw processes which describe what the computer will be doing, and Facilitators like to draw processes that describe what the people will be doing.  Both can be drawn with BPMN, but many of the current disagreements come from these different interpretations of the same symbols.

I will leave you today with a great quote from a presentation at the Gartner BPM conference by the Futurist Watts Walker:

Good decisions come from experience … and experience comes from bad decisions.

There has been a flurry of discussion on the Business Modeling and Integration Domain Task Force (BMI-DTF) at the OMG over the direction of development of the new versions of their specifications. Whether BPMN should have choreography capability or not. When BPMN should be linked to BPDM the meta-model and file format behind the notation standard. BPDM has essentially no adoption at this point, but it is still very new so this by itself is not indicative of anything. Yet some of the committee believe that putting BPDM into the BPMN spec will force adoption of this file format. Other feel that the combination will be too complex and possibly will inhibit adoption of the notation part. There have been several comparison between BPDM and XPDL. XPDL is in widespread use, and it appears the commitee members would like BPDM to do everything that XPDL does and more, in a much cleaner way.

The conversation often drifts to “marketing” and “adoption by end-user and vendor communities”. They talk about strategies and air play of their standards. It is as if marketing is some sort of genie that will step in and magically make their work adopted across the marketplace by fiat. To this, I have some suggestions.

Before diving into suggestions, I want to remind everyone that the standards process is driven forward by the huge efforts of a small number of very dedicated people. Most of these people truely want to build something that will be of lasting value to the high tech world. There are always, in any standards committee, a few who attend purely for self interest of promoting their own particular approach to become the recognized standards and then to capitalize on that. Such sheer self interest is ultimately detremental to the success of a standard, and so most successful standards committees tend to weed those people out when discovered. However, let’s assume for this discussion that the vast majority of those in the BMI task force are honestly motivated to develop something that solves a real problem, and such a solution would present a real value to the marketplace. In this case, I would recommend:

• Remember that no amount of “force” will cause a file format to become a standard. A standard must work and must provide a value. If it does not, it will not be used, regardless of how superior the design. Simply linking the file format with the notation in order to force its use will not fool anyone. Combining make sense only if BPDM can stand on its own merit.
• Remain focused on immediate real concerns. One of the more alarming comments was: “BPDM is designed to meet a business need that may not yet be fully appreciated by vendors or customers (who are distracted by proprietary marketing interests).” I would recommend focusing on needs that are appreciated today, or you may not have a chance to tackle the needs of tomorrow.
• Prove that BPDM works in many domains. There is a large body of real processes which are used for real business uses. The standards seem repleat with “toy” processes, such as the example of process between the patient and the doctor’s receptionist. While these are useful to as teaching tools to explain basic concepts, they fall short of actually proving that a meta-model is complete enough to represent the full range of semantics that are required for a real process. To do so, you need a broad coverage of a wide range of different engine types.
• Prove that BPDM is a superset of XPDL. Since XPDL is an XML based language, it is trivial to write a parser, and should not be difficult to write a converter. The exercise of writing this converter will assure everyone that all semantics contained in XPDL have a clear and unambiguous representation in BPDM. You should be able to demonstrate round-trip: from XPDL to BPDM and back, and the resulting file is not degraded from the original. This is in fact a goal for some committee members, but apparently not enough, because not even the specification for such a converter is available today.

My primary concern, without really knowing, is that BPDM is not complete (yet). The internals of XPDL are messy, and that is because real world processes are complex. I can’t count the number of times I have seen committees of people “reject” a defacto standard, in order to provide a far more elegant one, only to find themselves in the end proposing something just as messy as the original. These “new” approaches are often useful in some special case scenario, but are not necessarily able to handle a wide variety of cases. The BMI task force routinely claims that XPDL is a poor approach, but usually without any supporting evidence. It turns out that support for human processes is a very complex and subtle area. There are many different opinions on how to approach this, and those opinions come from different business cultures and customs. It is conceivable that the committee that designed BPDM are aware of only a subset of these approaches. This would be natural and will be resolved only through actual trial in the real world. Showing that one engine can use BPDM proves nothing, except that that particular engine is at least as limited at the standard. We won’t know that the language is complete until it can be demonstrated in use with a wide variety of existing engines.

All concerns would be eliminated with simply providing a BPDM/XPDL round trip converter. A small history lession: JPG raster image format became popular to a large extent because the JPG standard group provided a free tool for displaying and converting images to and from JPG format. Providing such a converter would make BPDM immediately accessible to more than 60 products on the market today. Allowing that converter to be freely embedded into vendor products will allow many products to claim BPDM support very quickly. This is, after all, the goal of the task force: to provide something of value that is adopted by the user and vendor communities.

To those of us wondering whether or not to bet on BPDM: we will know if the committee is serious as soon as they have made available a program that can convert an XPDL file into BPDM, and back again without loss. On the other hand failing to provide such a converter will give every indication to the market of being satisfied to remain an isolated and possibly esoteric file format. My humble suggestion to the BMI DTF is that they make this a top priority because this might be their best “marketing strategy”.