Go Flow

I spoke at the e-gov Enterprise Architecture Conference in Washington in September and was asked an intriguing question by a visitor.  We were talking about the relevance of BPMN, as well as quality of support for BPMN.  What distinguishes human work from what can be automated?  As a reference, I used a very simple two step process which I often use in my presentations – Account Application – which is essentially a two step process once you remove the automated activities.  The second step is a “Decide whether to Create Account”.  His response:  “Would that be represented by a decision node?”

I was stopped in my tracks.  In this case it is NOT a decision node, but why not?  Sometime a conditional branch gateway is called a “decision node” since the server “decides” whether to go one way or another.  In reality, those nodes don’t actually make decisions.  The real decisions are made long before you get to this gateway.  Why then do we call them decision nodes?

Determining whether to create an account is a real decision.  If it is important to select the people who hold accounts, if there is a reason that some people would not be suitable to hold an account, then you must have some sort of process to clear applicants, and someone must decide whether to give out the account or not.  The decision, in the extreme case is easy: a well known wealthy local resident is an easy decision; a criminal convicted multiple times for monetary fraud is another easy decision.  The easy decisions might be coded into rules, but the rules will not cover 100% of all cases.  So therefore ultimately there must be a person to fall back on to make the tough borderline decisions.

The node he was speaking of is also called a branch point gateway.  The process branches to one of a number of possible directions at that point based on information available.  The branch might depend upon rules, but is any real decision being “made” at this point?  Wasn’t the “decision” made long ago?  Consider an example like: “An applicant with credit ratings below a certain value will not be eligible for loans over a certain magnitude.”  Without getting deeply philosophical about it, wouldn’t you say the “decision” is made at the time that this rule is drawn up?  After that time, the execution of the rule simply branches the process based on that former parameterized decision in a completely mechanized way.  This calls into question the real meaning of “decide”.

Do we call them decision nodes because we anthropomorphize the computer system that is executing the process?   You can imagine the server playing a role in the process, and doing things on our behalf.  After all, when we get to that step that says to email all the participant, WHO is it that is doing the emailing?  It is the server doing the emailing, right?  Therefore the server is playing a role in the process as an actor, as if it were a stand-in for a human.  I would agree that it is carrying out tasks that it is instructed to do, but is it really making decisions for us?

I one heard someone make the point that military personnel are trained to understand their responsibility in making the decision to launch weapons.  The radar may say clearly that there is an enemy plane flying a dangerous route, but it is the commander who decides to strike.  There may be rules that identification of a particular threat dictates a particular response, and sensory equipment may indicate the presence of  such a threat, it still it is still the responsibility of a person to interpret that sensory data, and take action.  Business decisions, while never quite as final as some military combat decisions, are and should be the same way.

A real decision is the kind that is not easy to make.  A collection of information will be presented to a person who is responsible for the success of the business, and a decision is made based on many factors that are difficult to quantize.  In some cases, there may be a gut feel.  Malcom Gladwell wrote a great book titled “Blink” which talks about the “native intelligence” which can tap into so much additional information in parallel, and which our conscious stream of thought does not have time to be aware of.  An experienced account manager will be able to sum up a case relatively quickly, and decide whether it is worth risking extending an account to this person.  (Interestingly, this same person may then come up with an explanation for why they made the decision, but that explanation might or might not actually be the real reason, but I digress to far….)

Let’s wave the magic wand for a moment, and imagine what it would be like if we could reduce all decision making to rules.  Then we might have the “one button office”.  This is the idea that you go to work every day, and in the middle of your desk is a single button, and you press it, and the system does all your work for you.  This was the goal of the office automation movement in the late 70’s and early 80’s.  The problem is that as soon as you isolate all the rules, there is someone who figures out how to play those rules, and you have to go and adapt the rules for this change in the environment.  While rules are very important in relieving us from the tedium of routine case assessments, there will never at time in the future be a point where we can stop adjusting and modifying the rules, and there will always be edge cases for which it will be quicker and more efficient to have a person simply “decide”.

A great example of a task that will never be automated is that of deciding which advertisement to run in the next promotion series.  More examples of non-automatable tasks include: the decision to merge two different companies together; the choice of which campaign slogan to use; the decision to hire or fire a team member; whether to run an article in a newspaper.  Let’s call these for a “Business Decisions” because they are the kind of real decisions that must be gotten right in order for the business or an organization to thrive.  Often business decisions depend upon a complex web of current events, legal and ethical constraints, as well as imperfectly known evidence.

Business Decisions are made at nodes which are activities which are assigned to humans.  A person need to perform the task of making this decision.  Those are the true decision nodes, not the branch gateways.

Let me take this a bit further: we know that there are at least two distinct “camps” in the BPM field: those that want to support human work (”Facilitators”), and those that want to automate work that was previously done by humans (”Automators”).  The Automators are always looking for example that can be completely automated.  For example, 30 years ago if I wanted to know my bank account balance, I would have gone to visit the bank and asked an attendant.  Now, of course, I access a web site, and it is 100% automated.  In that situation, all of the “decisions” that had been done by the attendant can be reduced to branch gateways, and you can think of gateway taking the place of the decision that the person used to do.  Facilitators, on the other hand, are looking for example processes which include tasks that can never be fully automated.  For example, a hiring process which includes deciding whether a particular person makes a good fit for the team.  The hiring decision is made by a person at an activity node, and this controls the flow later in the process.

The BPMN specification does in fact call branch gateways “decisions” and this term is in widespread use.   This shows that the BPMN is strongly grounded in the ideas of the Automators.  Facilitators will also use the term “decision” when talking about a conditional branch gateway, but a Facilitator will know that real Business Decisions are not actually made there.  At best, the conditional branch gateway is directing the flow of the process in response to a Business Decision made by a person in a previous step.  Perhaps there is clarity if the Facilitators would talk about a “Business Decision Activity” where a person is given the task of making a decision.  This also reflects on of the fundamental difference: Automators like to draw processes which describe what the computer will be doing, and Facilitators like to draw processes that describe what the people will be doing.  Both can be drawn with BPMN, but many of the current disagreements come from these different interpretations of the same symbols.

I will leave you today with a great quote from a presentation at the Gartner BPM conference by the Futurist Watts Walker:

Good decisions come from experience … and experience comes from bad decisions.

There has been a flurry of discussion on the Business Modeling and Integration Domain Task Force (BMI-DTF) at the OMG over the direction of development of the new versions of their specifications. Whether BPMN should have choreography capability or not. When BPMN should be linked to BPDM the meta-model and file format behind the notation standard. BPDM has essentially no adoption at this point, but it is still very new so this by itself is not indicative of anything. Yet some of the committee believe that putting BPDM into the BPMN spec will force adoption of this file format. Other feel that the combination will be too complex and possibly will inhibit adoption of the notation part. There have been several comparison between BPDM and XPDL. XPDL is in widespread use, and it appears the commitee members would like BPDM to do everything that XPDL does and more, in a much cleaner way.

The conversation often drifts to “marketing” and “adoption by end-user and vendor communities”. They talk about strategies and air play of their standards. It is as if marketing is some sort of genie that will step in and magically make their work adopted across the marketplace by fiat. To this, I have some suggestions.

Before diving into suggestions, I want to remind everyone that the standards process is driven forward by the huge efforts of a small number of very dedicated people. Most of these people truely want to build something that will be of lasting value to the high tech world. There are always, in any standards committee, a few who attend purely for self interest of promoting their own particular approach to become the recognized standards and then to capitalize on that. Such sheer self interest is ultimately detremental to the success of a standard, and so most successful standards committees tend to weed those people out when discovered. However, let’s assume for this discussion that the vast majority of those in the BMI task force are honestly motivated to develop something that solves a real problem, and such a solution would present a real value to the marketplace. In this case, I would recommend:

• Remember that no amount of “force” will cause a file format to become a standard. A standard must work and must provide a value. If it does not, it will not be used, regardless of how superior the design. Simply linking the file format with the notation in order to force its use will not fool anyone. Combining make sense only if BPDM can stand on its own merit.
• Remain focused on immediate real concerns. One of the more alarming comments was: “BPDM is designed to meet a business need that may not yet be fully appreciated by vendors or customers (who are distracted by proprietary marketing interests).” I would recommend focusing on needs that are appreciated today, or you may not have a chance to tackle the needs of tomorrow.
• Prove that BPDM works in many domains. There is a large body of real processes which are used for real business uses. The standards seem repleat with “toy” processes, such as the example of process between the patient and the doctor’s receptionist. While these are useful to as teaching tools to explain basic concepts, they fall short of actually proving that a meta-model is complete enough to represent the full range of semantics that are required for a real process. To do so, you need a broad coverage of a wide range of different engine types.
• Prove that BPDM is a superset of XPDL. Since XPDL is an XML based language, it is trivial to write a parser, and should not be difficult to write a converter. The exercise of writing this converter will assure everyone that all semantics contained in XPDL have a clear and unambiguous representation in BPDM. You should be able to demonstrate round-trip: from XPDL to BPDM and back, and the resulting file is not degraded from the original. This is in fact a goal for some committee members, but apparently not enough, because not even the specification for such a converter is available today.

My primary concern, without really knowing, is that BPDM is not complete (yet). The internals of XPDL are messy, and that is because real world processes are complex. I can’t count the number of times I have seen committees of people “reject” a defacto standard, in order to provide a far more elegant one, only to find themselves in the end proposing something just as messy as the original. These “new” approaches are often useful in some special case scenario, but are not necessarily able to handle a wide variety of cases. The BMI task force routinely claims that XPDL is a poor approach, but usually without any supporting evidence. It turns out that support for human processes is a very complex and subtle area. There are many different opinions on how to approach this, and those opinions come from different business cultures and customs. It is conceivable that the committee that designed BPDM are aware of only a subset of these approaches. This would be natural and will be resolved only through actual trial in the real world. Showing that one engine can use BPDM proves nothing, except that that particular engine is at least as limited at the standard. We won’t know that the language is complete until it can be demonstrated in use with a wide variety of existing engines.

All concerns would be eliminated with simply providing a BPDM/XPDL round trip converter. A small history lession: JPG raster image format became popular to a large extent because the JPG standard group provided a free tool for displaying and converting images to and from JPG format. Providing such a converter would make BPDM immediately accessible to more than 60 products on the market today. Allowing that converter to be freely embedded into vendor products will allow many products to claim BPDM support very quickly. This is, after all, the goal of the task force: to provide something of value that is adopted by the user and vendor communities.

To those of us wondering whether or not to bet on BPDM: we will know if the committee is serious as soon as they have made available a program that can convert an XPDL file into BPDM, and back again without loss. On the other hand failing to provide such a converter will give every indication to the market of being satisfied to remain an isolated and possibly esoteric file format. My humble suggestion to the BMI DTF is that they make this a top priority because this might be their best “marketing strategy”.

I have not written much about BPDM, the new metamodel specification from the OMG. It is a long spec, and it appears that lots of very good work has gone into it. Making a general metamodel to allow for interchange of all of the various types of process definitions that exist is both very important and also a very big challenge. So this effort deserves a lot of support.

One design choice though surprised me enough that I think it is worth a comment here. The BPDM 1.0 specification is designed to store process definitions in an XML file, but the layout of the process definition is not included at the current time. The “model” is stored (the nodes and relationships) but the positions of the nodes and the geometry of the lines connecting them are not stored at this time.

This is important. I have mentioned in the past that one of the goals of XPDL is to represent the diagram of the process so that these process diagrams can be exchanged between tools that people use in what we call the process design ecosystem. I have an example that may help illustrate the importance of preserving the layout.

Imagine that you want to travel on a subway train from station A to station B. In order to find your way, you need a subway map, which is a model of the subway system. At the very least, the model would need to include a representation of each station, and the trains that connect it to other stations. It is true, that this is all you need in order to calculate how to get from station A to station B. But imagine that you, as a person, look at the map, and every time you do, the stations are in displayed in different locations on the page! Such a map would be almost unusable because every time you would need to read the entire map. A lot of effort goes to making a subway map pleasing and easy to remember. As humans, we need things to appear in the same place every time. The layout of the map is a very important aspect of such a map.

I addressed a similar issue in a past post called “Thow Out The Diagram?” where I was surprised by discussions with people who felt it was right and proper to remove the layout information. Several people from the OMG working group assured me that this was not the case with them, and that layout information is simply delayed until the BPMN 2.0 spec (where BPMN and BPDM will be combined into a single spec) which is anticipated at the end of 2008. The reason is that there is already a proposal for including layout into an XMI format file, and that proposal was not mature enough to include at this time, but it is anticipated that OMG will be able to offer storage of layout in the near future. Nevertheless, and I can’t help feeling that this is a major limitation. If I use BPDM to convert my XPDL file to another format, it will lose a very important aspect of the process diagram.

So in summary, XPDL is still the choice for sharing process diagrams within a process ecosystem with over 60 process tools supporting the standard today.

Why is there all this concern about a single process standard? I was reminded of this while reading a recent article in “Viticulture Obscurant” about a similar situation in Elbonia with standardization of the broad range of wines, which I reproduce in full below:

——-

Elbonia - July 16, 2007 - Citizens and government officials of the republic of Elbonia have been up at arms, almost literally, about the bewilderingly large number of varieties of wine that are now available on the market. There is a growing movement to limit the number of varietals that can be sold in order to simplify the act of ordering dinner at the local restaurants. The diners’ complaint is that it is tremendously difficult to match the right wine with the right dinner because there is simply so many choices.

“The wine list at my favorite place has Cabernet Sauvignon, Merlot, Pinot Noir, Cabernet Franc, Grenache, Malbec, Montepulciano, Shiraz, Zinfandel, Petit Syrah, Sangiovese, Lambrusco, Dornfelder, Bovale Sardo, Schwarzriesling, Ruby Cabernet, Tinta Barroca, and Preto de Mortágua and that is just on the red list.” says the town Mayor Harrison. “It is difficult enough to pick from all these in the first place, but it gets worse when you consider vintages. You have to keep track of which years are good for which regions. What we need is a single wine that everyone can have for every occasion.”

This sentiment is reflected in a new standardization efforts on multiple fronts. One such group produced the Border Practice Multi-Index to define the way that a wine should look, smell, and taste, even though this group does not actually produce any wine. A representative commented: “We did not want to show favoritism to any particular variety so we took the best qualities of all wines, and came up with standardized descriptions. So when someone says that the wine has a ‘musty nose’ or that the ‘bouquet includes blackcurrants, eucalyptus, chocolate, tobacco’ we will all know exactly what they mean.”

The index however has been criticized because it still allows multiple varieties of wine to exist. One restaurant critic complained, “I thought the index was supposed to define the qualities of the perfect wine. A vineyard would then simply create a wine with all the correct qualities and you would have a wine for all occasions. If a vineyard produced a wine with some of the qualities, and not others, consumers could simply boycott that wine until it got the required components. That way all wine would be perfect for any occasion.” There is disagreement about whether this was the goal of the index in the first place.

A different group represented by the two largest grape growing regions got together to form the Basque Puncheon Evaluation League and decided that “Chardonnay” would be the single official wine in the industry. “I have had many many good meals with Chardonnay, and they all have turned out great! I really have never had any reason to drink any other wine. Besides, all those other flavors really confuse me and I find it easier to stick with a wine I know.”

Since that time, an entire sub industry has grown up offering “Chardonnay-only” wines, and producing recipe books that feature the varietal in cooking and serving of all kinds of cuisines. Wine masters have written well-received books saying that ‘the perfect meal starts with a layer of Chardonnay as the foundation.’ It seems almost as if nothing can go wrong as long as it has Chardonnay in it. When local diners ask their sommelier why Chardonnay was the chosen varietal, the answer given, after a brief look of vague hopelessness, is “because the two largest grape growing regions chose it, so we know it is going to be successful.”

While clearly simplifying the wine lists, this move is not without its detractors. One buvionist, speaking under conditions of anonymity said “The problem is that even though all these wines say they are Chardonnay, the don’t taste the same! Each vintner makes the wine in a different way, putting their own special enhancements to it, and the result is that each Chardonnay has a distinct taste. The original point was to have a single wine for all occasions. How is this achieved if they are all different?”

Sceptical of Chardonnay’s ability to meet the needs of every meal, an irrepressable band of vintners continue to produce red wines in the traditional ways. In spite of the desire for a single wine for all occasions, people have continued to enjoy red wines as well as other varieties of white wines. This has not gone completely unnoticed, and in a surprising move a number of Chardonnay vendors have acquired significant holdings in red-grape vinyards, clearly with the intent to enhance their line to include both white and red wines.

Even more suprising is the announcement from a group known as Winesap Blue for People who have announced a new varietal called “Chardonnay Dark” which is based on Chardonnay (and therefore the perfect ingredient for any meal) but includes many of the features of a fine red wine. The acolytes praise this move to finally provide a single wine that is good for all occasions. A prominent Elbonian wine enthusiast had this to say: “I tried the new Chardonnay Dark, and I really liked it. In the bouquet I detected hints of blackcurrants, eucalyptus, chocolate, and, yes, even tobacco! I expect to be drinking this with every meal in the future.” Unfortunately the wine will not be available for a few more years.

At the same time, and probably unrelated, is a new development called the “Omni-grape” from the Borg Province Dept of Metamagic. The omni-grape is a variety of grape that contains all flavors of all possible grapes. “With a vinyard full of these grape vines, you can decide at harvest time what kind of wine you want.” The technical literature about the grape only includes instructions for making Chardonnay, but we have been assured that all other wines can be readily produced, as soon as representative of those other varietals get around to defining the mappings. While this is clearly an important development, it has been criticized for not actually simplifying the final wine list.

So the situation is far from resolved. Every evening, Elbonians continue to face a daunting task to determine their libation. While some restaurants now offer the simplified Chardonnay-only list, it does not seem to have the overall effect that was promised. However there is hope that new standardization efforts are continuing so that one day, in the not too distant future, diners will be able to sit down to a satisfying meal without ever looking at a wine list.

—————–

It struck me that there are some surprising parallels between this situation and the one we find ourselves in the area of Business Process Management. By drawing this parallel, I do not mean to say that there should be as many process languages as there are varieties of wine. But I do mean to draw attention to how questionable it is to be hunting for a single process language for all situations. Different process languages are fit for different purposes. Work in an office space is complex. There are no simple solutions for that complexity, but still people yearn for a simple solution. We should instead focus how well a particular language worked in a particular situation.

Going to be at BPM Think Tank? I will. Maybe we can discuss this over a glass of something-that-is-not-Chardonnay.

There are some tips in the field of BPM that you don’t want to find out by trial and error. If you have done a business process improvement initiative, you already know that the first step is to model the process. In order to model the process, you must uncover what the process is, and this step is called process discovery. How to you discover the process? You ask the people who work there, of course.

The naive approach will be to identify the process participants, set up an appointment to sit with them and as “what do you normally do for your part of this process?” Seems like a pretty good approach on the surface, but there is a hidden pitfall. To the extent that the person knows what they do, this will yeild the “sunny day” scenario. When you complete this exercise you have a process that represent what happens if everything goes right.

The “sunny day” process diagram is fairly clean and straightforward, and most people agree it is correct, but when you put it into use something very surprising happens. You find that what people actually do is very different. Those differences are called “exceptions” to the rule, and therefor don’t fall into the category of things that people “normally” do. At this point you try to retrofit the exceptions into the process, which can take longer now that it is happening late in the development cycle. Many organizations actually spend a majority of time and effort handling such exceptions and so their consideration in the process is very important. These are not failures, but simply cases that do not fit the normal rules.

There is a better way. I attended a talk by Dr. Michael zur Muehlen today at an impressive “BPM Day” at the Stevens Institute of Technology in Hoboken New Jersey. He says that the right question to start with for process discover is one like: “tell me about the most difficult case that you handled”. He says it is then easier to combine a number of these into a common a “rainy day” scenario process that includes the important exceptions from the beginning. Not obvious, but it sounds like good advice to me. Leave it to a German to point out the benefit of a “rainy day” scenario.

BPM Day was a seminar to cover tips and techniques in Business Process Modelling presented by:

• Michael zur Muehlen, the Director for the Center of Excellence on Business Process Management at the Howe School of Technology Management. BPM Day highlights their new graduate program on Business Process Management and Service Innovation.
• Jan Recker from the Queensland University of Technology. Jan covered the BPMN modeling standard, and reflects the advanced work on process technology going on in Brisbane. In September 2007, QUT will be hosting the “Fifth International Conference on Business Process Management” which is likely to be an excellent forum for the latest in BPM.
• In association with the New York chapter of the Association of Business Process Management Professionals (ABPMP). Look to these people to provide more excellent seminars and discussion forums in the New York area in the future.

Queensland University of Technology is doing a survey on BPMN Usage. Anyone that uses BPMN to create process models for whatever purpose is welcome and encouraged to participate. These guy in general are doing a lot of good work for BPM, so I assume this is another worthwhile endeavor. Here is the info:

BPMN is gaining momentum in practitioner communities, up to a point that even those vendors who were initially reluctant to adopt it, can no longer completely ignore it. But what exactly are the factors that drive this acceptance? How satisfied are end users of BPMN with the notation? Do user experiences on BPMN match those by BPA tool vendors?

Jan Recker from the BPM Research Group at Queensland University of Technology is undertaking a worldwide survey on the use of BPMN by process modellers to shed light into this question. You can help Jan by completing the survey available here:

http://www.bpm.fit.qut.edu.au/projects/acceptance/survey/BPMN/

The best way to contact Jan is via email: j.recker@qut.edu.au

The next BPM event will be happening May 21 through 24 in the Washington DC area at the Transformations and Innovation conference.

It starts with one day workshop on May 21 on “BPM in Practice: Understanding and Implementing Workflow and Business Process Management” given by key members of WfMC. To be covered are:

• BPM 101
• BPM and Enterprise Architecture
• Human BPM compared to EAI Processes
• Business Process Analytics
• BPMN overview
• XPDL overview
• BPEL overview
• Wf-XML overview
• Summary and open questions

It is designed to be a comprehensive overview of the BPM space for those new to the subject and wanting to get their bearings, and also for those familiar with the subject who want to big a bit deeper through direct interactions with the people who have been working on the standards. See the brochure.

Also BPM Focus will be there with a session on Enterprise Architecture.

I just got my copy of the 2007 issue of the Workflow Handbook, and this looks to be the best issue yet.

I have been discussing HDR photography with friends and colleagues for the past few weeks, but it seems what everyone needs is a really good example. I have published a bunch of HDR photos on my Flickr page. But the real question is where do you really need this. Since discovering this technique, I find my self looking for shots where the dynamic range is beyond that which my camera can handle. I set up these shots just to see how it will look. This past weekend I succeeded in finding an example where HDR really makes a difference.

Below is a picture of a room and a window. Most photographers know that this shot is going to be difficult at best. The exposure either needs to be set for the room, or for the outside, but you never will get both. So I set up the camera, and took a normal shot. Here is how it came out:

Please click on the photo to see the full sized result. What you clearly see is that the scene outside the window is considerably over-exposed. Note how the colors outside are almost completely lost as the brightness nears the maximum that the camera can record. At the same time, the room is dark, and again in the dark the colors the colors are similarly indistinguishable. This is using the camera’s on-board color conversion capability. Normally, I would use a flash with this kind of picture in order to bring the level of light to be nearer that of the outside. But a flash brings its own quality to a picture, drowning out the shadows, and making things look flat.

The camera actually can record more dynamic range than the JPG format can carry. JPG is limited to 8 bits per channel, and my camera records 10 bits per channel, which should be a factor of 4 more range. So I took a single RAW format picture of the scene, and converted it using the same post-processing that I did with the rest of these, only it is a single photograph. It certainly carries more dyniamic range, but the result is decidedly unsatisfying:

Then I took the picture again, with three different exposures, -2, 0, and +2ev. Here was my surprise. This is not enough for this scene. It is clearly better, but still the outside is a bit over exposed, and the interior colors of the couch are noisy. Three pictures was simply not enough. Here is what I got:

OK. I am going to have to go set up again. This time I take 7 different exposures ranging from -6 to +6 in 2EV increments. Very dark to capture the colors outside, to very light to get the colors inside. The result of the combined picture looks like this:

Finally, that is a great shot. Please click on the photo to see the full scale photo. Of course there is a dramatic difference: you can see the colors of the garden outside the window, as well as the details within the room at the same time. This photo has tremendous dynamic range that could not be captured in a single photo. Go back and look at the original.

Isn’t this a dramatic difference? Anyone want to guess how long until camaras exist that will do this automatically?

Finally something useful: a BPMN poster:

Claims to have everything you need to know. It certainly contains a lot of useful info. It is from the University of Maribor, Faculty of Electrical Engineering and Computer Science, Institute of Informatics, in Slovenia. The authors invite comments, and I imagine a comment on this post will get them.

Bruce Silver put together a summary of The Real Issues with XPDL, BPEL, and BPMN where he explained better than I could that the aspect of portability that is more valuable depends on what you’re trying to do. He correctly points out that “XPDL captures the diagram, while BPEL captures the process semantics.”

Bruce brings BPMN into the discussion as potentially the standard that is possibly the most important. There have been a number of discussions recently of the relation of these three standards, including Jason Stamper in Computer Business Review and Jon Pyke in EBizQ.

Before going any further, there is one thing that must be straightened out. XPDL 2.0 captures every aspect of any BPMN diagram. It is a complete representation of all element and all attributes of BPMN. A couple of the people were involved in both BPMN and XPDL specifications, and single guiding principle for the 2.0 release is that there should be a well defined way to store every property mentioned by the BPMN specification. My descriptions of XPDL often focus on the X & Y coordinates, and the shape of the lines, because this is an obvious differentiator from BPEL, but it does not stop there. XPDL also represents the attributes that specify a web service call, or the event definitions, and every other attribute mentioned in the BPMN specification. XPDL also extends this with additional ability to express process semantics that was developed over the years in WPDL and XPDL 1.0 by the collection of workflow/BPM vendors attempting to define exactly the important semantics to exchange.

XPDL is a strict superset of BPMN. This means that everything and anything that can be expressed in BPMN, can be captured by XPDL. The complete process diagram, including any and all process semantics expressed by that diagram.

Bruce has criticized XPDL for inability to take an executable process from one vendor product, and bring it to another vendor product, and guarantee is it understood. He is right. There is no guarantee that a process drawn in one product, saved in XPDL, will be immediately executable in another product. This is not because XPDL fails to capture the semantics, but instead a failure to (1) be able to unambiguously capture those semantics in standard BPMN, as well as (2) a failure of the receiving tool to understand the same semantics that the sending tool transmitted.

To the extent that BPMN can represent the complete process design, XPDL can transfer that complete process definition, including all the semantics of that process. To the extent that BPMN is ambiguous and misunderstood by another tool, XPDL is similarly likly to fail. This is because the XPDL is essentially a one-to-one representation of the BPMN.

Let me use an analogy again. Another standard that is very important is the Unicode standard of characters and character encoding. I can write English all day long and be completely assured that every expression is 100% captured and stored in a Unicode file (let’s ignore markup for the moment). I would say that Unicode is a great success at allowing people to communicate, and it carries the complete semantics of every expression. Others would point out the obvious flaw in my thinking: only 1 person in 10 speaks English, and thus 90% of the world can not read my posts (well, there are probably another 30 to 40% that could muddle by enough to make sense of it, but you get what I mean…). A French person can use Unicode as well, and be completely assured that their expression will be preserved with Unicode, including all the semantics. Zut alors! While Unicode guarantees that the expression is completely preserved, it does not guarantee that all readers will be able to understand the expression.

BPMN gives us a vocabulary to describe business processes, in the same way that a dictionary gives us words. But please note: the underlying meaning is not in the words, but in how those words are put together. There are a number of different dialects of BPMN out there. Two tools that use the same dialect of BPMN, can easily exchange processes via XPDL.

Let me give you an example. Lets say there was a product called “Vulcan Mind-Meld” which used BPMN to express the diagrams that have meaning to this product. BPMN defines what each of the symbols mean, but the real meaning, the real semantics comes from the way that the symbols are composed together. Mind-Meld would guide you as you draw this diagram, making sure that you do not put anything together in a way that is nonsensical. The author of this diagram is making an expression that has meaning in Mind-Meld. Here is a possible diagram which is consistent with the BPMN specification:

What does this mean? I frankly don’t know. I tried hard to make an ambiguous diagram that still obeyed all the BPMN rules. This is not to imply that there is any flaw in BPMN, but simply that with any language you can follow the rules, and make something that is meaningless to some readers. But stick with me for the moment.

We can argue about whether Mind-Meld should allow this expression, the main point I want to make here is that Mind-Meld can save this diagram to XPDL and read it back in again with complete fidelity. All the attributes of the elements are preserved. None of the semantics would be lost. Another product called “Klingon Kommand & Kontrol” would be able to read that XPDL file. It would have all the nodes, and all the attributes of those nodes. It could reproduce the diagram. Its ability to understand the meaning of the diagram depends upon it interpreting the combinations of BPMN symbols in the same way that Mind-Meld did.

This is the essence of Bruce’s concern: he would like Mind-Meld product to interpret the BPMN expression into a “universal process meaning”, and then communicate that meaning to the Komand & Kontrol product in such a way that it can not be misunderstood. That is kind of the Holy Grail of process modeling, there is only one problem: this is very very very difficult.

We could wish for the same thing in the Unicode analogy: that the English expression could be somehow digested, understood, and represented in a language independent way, such that any reader would be guaranteed to receive the meaning. Thus I would write an expression in English, it would be digested and decoded, and then would be displayed in French to the French reader, German to the German readers, and surfer-lingo in parts of Santa Cruz. This, of course, is very very difficult.

But if you think about it, why would you save and exchange anything other than the diagram that the author composed? The original diagram is the expression of the author. How that expression be converted into anything that carries more meaning than the original diagram? If the meaning in BPMN is unambiguous, then every tool that reads it will interpret it in the same way.

If I was to write in English, why would I store and exchange anything other than the English that I wrote? If a French reader wants a translation, it can be translated from English to French at that time. If there was an intermediate structure that could represent the meaning of the English, it is not clear that it would be better than the English that was written originally.

It works the same with BPMN/XPDL. If you want to transfer a process definition from one product to another product, it makes the most sense to transfer the original diagram that the designer drew. This contains the best representation of what the designer intended. Then, if a translation is needed from one dialect to another, it can be done at that time.

This was a conscious choice in the design of XPDL. XPDL represents the process diagram in a one-to-one manner, more or less directly what the process designer drew, and not an abstraction of it. This does not mean that the semantics are dropped on the floor — they are there in the diagram just as the designer put them there. Each product, however, must interpret the BPMN to get the meaning from it.

Back to the Vulcan Mind-Meld example: can this diagram be represented using BPEL and read back in again? Most likely not. Again, this is not a flaw of BPEL. The strength of BPEL is that all of its elements are rigorously defined. This increases the liklihood that it is understood by all readers. But it also decreases the flexibility. I might be wrong: if someone would like to post the BPEL representation of the Mind-Meld process, I think we all would be very much interested in seeing it. The XPDL, on the otherhand is easy to make.

BPDM, the future metamodel from OMG, promises to be the “universal process meaning” for process products that can represent any process, including Vulcan Mind-Meld. Every product will interpret the BPMN diagram, and express the meaning of that diagram in BPDM. This would be a huge benefit to us all. We don’t know how well this works since it is not available yet. But in any case, it is not clear to me that this representation would be more faithful than the original BPMN diagram. It seems that the OMG folks expect different products to interpret the BPMN in different ways! With this assumption, then how are we humans supposed to know what a BPMN diagram means by looking at it? The whole point of BPMN is that it should be an unambiguous representation of the process, so that when people see it, they interpret the meaning, and when products read it, they get the same meaning from it. I am with Bruce on this: BPMN is the most important standard for unification. That is the goal of BPMN, we are not there today, but we have a path that may lead there.

Until the advent of XPDL 2.0, there has not been a way to transfer BPMN diagrams from product to product. Each product was isolated and developed dialects of BPMN the way that isolated people developed different spoken languages. Now that we have a way to transfer these diagrams, it is very likely that groups of products that work similarly will tend to use BPMN in the same way, and that shared dialects will emerge.

This trend is enabled because XPDL can represent any BPMN diagram.

Why can’t all products interpret the BPMN in the same way? This is another of Bruce’s concerns. Why can’t there be a single dialect of BPMN? Please stay tuned for my next post.