21 Questions to Ask a BPM Vendor

With all the work I do in getting BPM system to interoperate with each other, I have come to recognize a set of potential problem areas.  Naturally, vendors are not always forthcoming with these little glitches.  So I have put together a list of questions that someone who is currently evaluating product might want to ask the vendor, and gauge the response.  Pick and choose, but I hope this list is helpful in getting some probing questions:

• Does your product support a standard external interchange format for process definitions? – some products support only an internal proprietary file format. That file format may be XML based, but that in itself does not guarantee interoperatibility with other tools because XML defines the syntax of the file, the semantics of the file may be entirely proprietary.

• Does your product support XPDL? – Most full featured workflow and BPM products today support XPDL at some level. It is currently your best option for transferring a process definition between products. But there do exist other standard formats. A good advice is to consider a collection of different products that people in your organization might want to use, and see what common formats exist between these tools. This will avoid the situation that you are stuck with a lot of valuable work in a proprietary format.

• Can the product import that standard format? – Some products claim support, but only export the format.

• Can the product export that standard format? – Some products claim support, but only import the format. In some cases by design the product consumes a process definition but never modifies it, so there is no need to export the process definition.

• When the product exports a process, and then imports it in again, does it look that same? – Some workflow products can both export and import a particular exchange format, but when doing so, some information is lost (or added) such that reading the same file in again produces a result that is not the same at the original. In some cases there may be a good reason that this is not possible, and you should understand that reason before deciding to use the product.

• How many other products read that export format? – In some cases vendors claim support for a “standard” which they invented and only their product supports. Current web technology is full of such claims for formats that are “standards” simply because one group claimed it is a standard. Many such “standards” are not finished or fully agreed upon yet, and there are a few cases of “standards” for which no draft specification exists! Even when a bonafide standards group ratifies a specification, you want to know how many products actually have implemented it to be sure that the standards is a success.

• What other vendor tools have you demonstrated the ability to exchange process definitions? – Public demonstrations of the actual interchange of process definitions are relatively easy and represent a commitment from that vendor to coexist with other vendors. Lack of such a demonstration might indicate that their implementation of the interchange format is not mature.

• Can you add or modify a activity in a process instance while it is running? What other modifications can be made to the process definition? – Some workflow products support on-the-fly modification, and others do not. Your need for this relates to how long your processes are going to run. If you are writing processes that completes within a few minutes, then there is little need to change the process while it is running. But if you processes is expected to last weeks, months, or years, you will find that laws will change, the market will change, customers will change, and the processes that coordinate between these must also change during the lifetime of a given process.

• Can one process be invoked as a subprocess of another? How many levels deep can this go? – Subprocess invocation is an important way to package up parts of a process in a unit that can be called as a whole, and may even be called by many different parent processes. Surprisingly some products do not support this directly. Be careful about answers that clarify that you can design a process that can make calls to invoke another process. Any process can be invoked somehow externally, and most process engines can make external calls, but this is different than actually representing the relationship between the two processes as a parent and child relationship. The parent-subprocess relationship exists in the running instances; they need to in some sense know about each other, and often there is a way to navigate the UI from one to the other.

• What kind of historical data is available? Can you see past values of process variables? – Virtually all process engines maintain some form of history, since one of the key benefits of process technology is to give you insight on how things are running. It is important to know what kinds of information are held in history. Many products allow you to track the values of process variables over time, but others do not.

• Is there a way to send log events to an external analytics engine (Business Activity Monitoring tool)? – WfMC defined interface 5 in recognition of how important it will be to externalize this monitoring data, and to be able to consolidate this information from multiple runtime engines into a single analytics engine that can give you an overall view of your processes. Many products today can do this, but others can not. WfMC has started the process of standardizing “Business Process Analytics Format” ( BPAF) to allow products to interchange this information, but we should not expect implementation in products until after the end of 2009.

• Does your product support Wf-XML? – This lesser known standard specifies a set of web standard services which can be used to start, stop, update, and monitor the status of a running process instance. At this point this is not widely implemented, but if the product does, then it would allow you to easily construct a process on one engine that invokes a process on another engine. If you are thinking about using this new engine as a strategic way forward, but are not yet ready to re-write all your existing processes, this standard allows you to leave the old processes in place, while writing new processes on the new engine, and invoking the old processes when needed.

• Can you extend the system with new capabilities, and call those capabilities from within a process? How is this accomplished? – Most workflow and BPM products today offer a way to “extend” the engine in order to add capabilities to it. Usually there is some way to plug in a module of some sort, and use that module within the process.

• Are process definitions exposed as web services (without the need of an additional component)? – Some process engines offer strong capabilities to call a web service, but then offer now way for their processes to be called as a web service. You should remember that while the project today might be about being able to invoke those services as part of a process, the project tomorrow will be how to invoke that process as a component. It makes sense that every process definition should automatically appear as a web service.

• Can every data type supported by the product be mapped to an XML Schema data type? – every organization concerned about SOA will be expecting to transfer data as XML packages in the future. For this to be successful, there must be complete fidelity when expressing internal process data as XML, and vice versa. This can be done many ways, but XML Schema is the standard way to define XML structures.

• Can a non-programmer create a workflow? – Some workflow and BPM products are aimed primarily at developers, and some are aimed at business users. You should know which one this is.

• What platforms are supported. Are process definitions platform independent? Can a process developed on one platform (eg. Macintosh) be run on another platform (Unix)? – You should know what flexibility you will be able to have in the future.

• Can an activity contain a timer that sends a reminder to the person doing the activity (without canceling the activity)? – This is one of the big differentiators between human support and system-only support of processes. If an activity is assigned to a human, there should be an easy way to send a reminder that the due date is about to passed, without having to leave the activity. Some systems support timers, but those timers “complete” the activity and move on to another activity.

• Can multiple reminders be sent on a single activity? – You might have multiple reminder, or multiple ways to remind the person. You should not be limited to one.

• If an XPDL is read with foreign Extended Attributes are they preserved and re-written when exporting? – This is a detail about implementation of XPDL in the proper way that allows a process design ecosystem to work. While reading the XPDL file, each product is supposed to preserve any extended attribute structure that it does not understand, and simply include that extended attribute when it writes the file back out. Products that do not do this will still exchange information with product on a point to point basis, but are less useful when a chain of three or more products are used.

• Can activities and other element be named with Kanji and other non-European characters? – Some products claim Unicode support, but restrict that support to only data fields. An activity name describes what it to be done. If it is important to you to describe activities with non-ascii character you should be sure to find out what character sets are supported on process definition elements.

I maintain this list at this wiki page.