Saturday, September 24, 2011

If you are trying to solve LOB Integration problems using BizTalk 2010


“Microsoft BizTalk 2010: Line of Business Systems Integration “ 


I am sure that most BizTalk Architects, Developers, and Administration will agree that BizTalk Server 2010 is a collection of Tools and Services that can be used to solve many Business Problems. The most common problem is Integrating Systems that have no direct means of communication. These are usually the core business systems.

There are several good BizTalk reference books available, but only one of them addresses BizTalk's Integration capabilities. That book is Microsoft BizTalk Server 2010: Line of Business Systems Integration. This book is more than just another reference book. It starts where the published WCF LOB Documentation leaves off. Each chapter is like a training session.

The chapters in this book are very diverse. They range for developing custom WCF LOB Adapters using the WCF LOB Adapter SDK to integrating with Each chapter provides a through description of the LOB Adapter and step-by-step instructions in its use. There are several examples of how the LOB Adapter can be used to solve real business problems. There is source code available for each of the examples.

BizTalk is a complex product that can be overwhelming to those just starting to work with it. I have been working with BizTalk for over nine years and still am still overwhelmed with its capabilities. You can't be an expert on all that BizTalk offers. Microsoft BizTalk Server 2010: Line of Business Systems Integration has provided me with answers to questions that help me solve my client's LOB Integration problems.

The authors are highly recognized as the "LOB Integration Experts". Although I don't know them personally, I have communicated with them and avidly follow their blog postings.

If you are trying to solve LOB Integration problems using BizTalk 2010, Microsoft BizTalk Server 2010: Line of Business Systems Integration is a must buy.

Monday, September 12, 2011

Business Rules and Business Process Exceptions

Often a business process team will create a working prototype of a business process. These prototypes might arise from 'use-cases' or business cases.

This idealized process develops a deep understanding of the final version, yet much of the important work remains. You should even consider executing this development artifact in a test bed. Yet to complete a business process, the technical team should detail the process diagram with the necessary scheduling, exceptions and compensations. The purpose of the details is to erect a more complete process model. The business processes should act on all the messages, including the system failures. The process should handle every anticipated exception.

There are 3 types of exceptions may affect process execution:

  • Technical exceptions
  • Transient exceptions
  • Business exceptions

As we will see, business rules can create, named business exceptions. If a BizTalk Orchestration is transactional, the process should compensate for transactions that fail to complete. In compensation, the process cleans up or backs out records from ERP’s, operational datastores or data warehouses.

Often processes should be complete within a time period, so there could be time out actions noted on the diagram. Business processes usually receive a message and translate that message into yet another message for consumption by the next part of the process. BPMN is powerful for developing time-outs, exceptions and compensations. As seen in the BPMN fragment below, you can attach time-outs, exceptions and compensations directly to a subprocess task list. When time-outs and exceptions occur within the two process tasks, the BizTalk Orchestration traps these at the point of the subprocess.



Figure 1 BPMN fragment

In this scenario, a grant authorization system evaluates grants and records the data. The time-out BPMN (the clock) will raise a time-out exception and wait. The diagram reports any exception that occurs (the zigzag) to the exception task. The compensation takes care of a record in the grant database.

You can model Business Processes with advanced abilities that suspend processing, or communicate with disconnected, mobile processes. These powers yield opportunities for the enterprise to fine-tune their most complicated practices. Yet to develop these capabilities you should understand how to handle the three types of exceptions and what you must plan for when developing the production version of the process diagram.

A business process diagram should use exception handlers to redirect the process flow in case a transient or business exception happens. However, no process can manage every exception. There are technical exceptions that will cause the entire process instance to suspend.


Technical Exceptions

Technical exceptions occur when the process causes the process instance to fail. An internal server exception should be abnormal; however there are operational circumstances that cause these. Technical exceptions include:

The common .Net Null exceptions

  • Exceptions caused by Illegal Argument exceptions, (suppose an external operational system decides to change a field from 10 to 20 characters)
  • Server database exceptions, out of tablespace, violation of database constraints

These are outside and therefore invisible to the calling subprocess. Processes cannot handle them because the execution state of the process has stopped. When this technical exception happens, the current transaction is always marked for rollback. The process throws the exception to the scheduler or message router.


Transient exceptions

A transient exception is usually caused by code external to the server like database connections or connection pooling exceptions. Within the scope of the current transaction, a transient exception is visible and the process can catch it with an exception handler. If the exception is not handled in the scope of the current transaction, the subprocess marks the transaction for rollback and throws the exception to the scheduler or message router. The process is suspended.

The suspended process can be restarted and resume processing after the database connection or some other condition is restored.


Business Exception

A business exception is one that can be expected by the process. Examples of these exceptions include:

  • ·Exception raised by an activity, a specific, named exception is raised, for instance a business rule could raise a named exception.
  • Exception caused by receiving an error message (like an SOAP fault).

If the exception is not handled in the scope of the current activity, the activity fails. When the activity defined a coordinated transaction, the transaction is committed. When no exception handling takes place Exception processing begins and leads to compensation and the failure of the process.


Business Rules Decisions and Business Exceptions

Many organizations use the BizTalk 2010 Business Rules Engine (BRE) to validate and transform incoming transactions. Validations and transformation are important types of 'decisions' that a business process makes with the data within process flows.

This is a common decision pattern: evaluate a set or transactions, say from an incoming data set. Each record that fails the validation should be stored in an area for latter cleanup. You can either handle the validation directly, with values in the process flows or you can raise a business exception, create a reusable process for handling the exception and move on.

Another common decision pattern validates user form input, in a workflow. Users might correctly complete a form and send it to a process flow. If there are business rules that violate complex data and business conditions, the process should collect data from the user at a later date.

Figure 2 is an example of what I am describing. A decision service validates incoming data. If the data does not pass validation, the subprocess raises a named exception, otherwise the record is processed.


Figure 2 Decision Services

The basic idea is to design your exceptions, time-outs and compensations to enlarge the flexibility of the process and allow most process to continue unattended. Handle the process and move on.


Diagram complexity

The complete BPMN diagram has many important exception features that must be accommodated.

Unfortunately, these additional boxes and notations make it difficult for business users to understand. This is where you use a BPMN Structuring Phase diagram. This diagram does not display exceptions, compensations and timers attached to subprocesses. The tasks following these should also not be displayed. This would clarify the core nature of the process and provide your business users an easy to understand document.


Windows Live Tags: Rules,Exceptions,Often,team,prototype,prototypes,version,development,artifact,diagram,compensations,purpose,messages,system,failures,exception,execution,Technical,Transient,BizTalk,Orchestration,transactions,compensation,data,warehouses,message,consumption,BPMN,task,tasks,Figure,scenario,authorization,clock,database,Processes,abilities,opportunities,enterprise,capabilities,production,handlers,instance,server,Null,Illegal,Argument,violation,constraints,transaction,connection,Within,scope,handler,Examples,error,SOAP,failure,Decisions,Many,Engine,Validations,transformation,decision,validation,area,cleanup,Another,user,Users,example,Services,Handle,notations,Phase,timers,Tags,outs,rollback,scheduler,router

Friday, September 9, 2011

Using BPMN to design and document BizTalk 2010 Projects


The following are the recommended steps to design and document a BizTalk 2010 Project using Visio 2010 Premium – BPMN

1. The Project Manager and Business Analyst will create the Functional Requirements documentation.

2. The BizTalk Architect or Lead BizTalk Developer will create a new Visio 2010 BPMN document for the project.

3. The BizTalk Architect or Lead BizTalk Developer will create the “Black Box” BPMN diagram.

4. The Project Manager and Business Analyst will review the diagram to validate that it meets the project requirements.

5. The BizTalk Architect or Lead BizTalk Developer will then add a new tabs for the BPMN Structuring Phases (1 -3). The Structuring Phase diagrams will be constructed.

6. The Business Analyst will use the Structuring Phase diagrams to determine what supporting documents need to be created. These documents include; Transformation (Maps), Business Rules, etc...

7. The BizTalk Architect or Lead BizTalk Developer will validate the Structuring Phase 3 BPMN Diagram for any errors.

8. The BizTalk Architect or Lead BizTalk Developer will add a new tab for the Reconstruction Phase Diagram.

9. The BizTalk Architect or Lead BizTalk Developer will create the Reconstruction Phase Diagram.

10. The Business Analyst will validate the Reconstruction Phase Diagram.

11. The Business Analyst will add a new tab for the Instrumentation Phase Diagram.

12. The Business Analyst will attach the documents to the Shapes within the Instrumentation Phase Diagram.

13. The Business Analyst will enter the attributes values for each shape in the Instrumentation Phase Diagram.

14. The BizTalk Architect or Lead BizTalk Developer will validate the Instrumentation Phase Diagram.

Tuesday, September 6, 2011

BPMN Diagramming Basics

Microsoft has recently published a Training Course, "Learn the basics of Business Process Modeling Notation (BPMN) and how create a BPMN flowchart diagram in Visio 2010 Premium" on the MS Office Support site.

I highly recommend this for anyone involved in designing BizTalk Solutions.  The BPMN shapes and attributes relate to BizTalk Orchestrations.