BizTalk Message

A BizTalk message is any binary stream of data. It can be an XML Document, flat-file, .NET serialized class, or any other binary stream of data such as zip or pdf.

A BizTalk Message is represented by two different objects, depending on which part of BizTalk its being processed.

• In the Messaging Engine, all messages are represented by type IBaseMessage. Similarly, message parts are represented by IBaseMessagePart.
  • So if you want to access a message and its parts within a pipeline, you will use the IBaseMessage and IBaseMessagePart types.
• Once inside an Orchestration, a message is represented by XLANGMessage.
  • Also, once any message inside an Ox, its implicitly treated to be of type System.Xml.XmlDocument.
  • This means that inside an Orchestration, you can cast any message to type XmlDocument,
  • and you can represent any message whose type is unknown at design time, as XmlDocument. I will return to this when I discuss Typed and UnTyped messages.

Binary Message Processing

BizTalk has the ability to process binary messages, both in the messaging engine and the orchestration engine.

In the messaging engine, IBaseMessage, treats a message as a byte stream; therefore, it has the ability to process binary data.

• In order to accept binary messages, you need to use the receive passthru pipeline.
• However, using passthru means that the MessageType property is not promoted. So if you want to use this property for a binary message, you have to create a custom pipeline component that does promotion.
• In case you’re wondering, property promotion does not mandate that the message be XML. Promotion works also for binary messages.

Now Inside an OX, if you set the message type to XmlDocument, you can process a binary message.

As strange as this might sound, within an Ox, a message of type XmlDocument does not have to be XML.

• The reason for this, it that messages inside an Ox, are not loaded into the Document Object Model, so XmlDocument message type will handle any binary type.
• The exception to this, is if the message’s context contain distinguished fields, which are Xpath expressions to message elements, in which case the message has to be XML. Therefore distinguished fields do not apply for binary messages. I will talk about distinguished fields in the next sections.

Of course, you’ll mostly use XML documents in your projects; especially if the project includes orchestrations. BizTalk power is most evident if your using XML messages. However, there might be cases when you need to use binary messages. Now what are the scenarios that might force you to build a business process around a binary message depend on each case.

Immutable Messages

A final important concept you need to know about BizTalk messages, is the concept of Immutable messages. Since a message can have multiple subscribers, then if a certain subscriber gets a message, changes it and publishes it back, then other subscribers might receive messages that they are not expecting.

As such, BizTalk messages are said to be Immutable; meaning that once a message is published into the MessageBox, it cannot be changed. If a subscriber wants to change a message, first it must create a new copy of it where it can perform the required updates. This leaves the original message intact. This is applied in Orchestrations, where you are not allowed to edit a message.

But, wait: then how come that during pipeline processing, messages get updated – for example have encryption applied – and even you can edit them in custom pipeline components?

Well, the answer is that simply, in receive pipelines, a message is not yet published into the MessageBox. Similarly, during a send pipeline, I can change a message, however, I am doing so without publishing it again into the MessageBox. So the published message inside the MessageBox remains unchanged.

Message Context

BizTalk Messages has a message context, which contains a set of name/value pairs of properties. These properties

• Are either extracted from the message itself, for example, order id or shipment number
• Or added by pipelines and adapters at the receive end, for example, transport name or receive port name

There are mainly two benefits of this context:

• The first is to provide the various components of BizTalk, easy access to these properties, without having to parse the message
• The second, is to support content based routing

The context is represented by the IBaseMessageContext interface, which you will use when you want to access the context form within a pipeline component.

Now its very important to know that Context properties, can be either written or promoted to the context.

• Promoted properties are the ones that can be used for message routing. These are the promoted properties that get promoted in the Disassemble stage of a pipeline as well as by the adapters; and these are the ones that are used for content based routing. They can also be used for tracking. These can be user defined properties or system defined properties.
• Written properties, cannot be used for routing or tracking, and are typically only used inside Orchestrations. These properties are written inside pipelines in the disassemble stage. These can only be user defined properties; adapters and pipelines do not write any system-level properties

You need to know that in BizTalk terminology, promoted properties are called Property Fields, while written properties, are called Distinguished Fields.

How to Populate the Context?

There are different ways to populate a message context, depending if you are writing or promoting properties.

Now lets examine the first method, which is using BizTalk Editor. Using the BizTalk Editor, you can create both Property and Distinguished fields. Distinguished fields are straightforward; you just right click the schema node and select show promotions:

Now select the field you want, and add it to the Distinguished Fields tab:

Property Fields, require first that you create a Property Schema. Recall from the discussion around schemas, that a Property Schema is a simplified version of an XML Schema. In this schema, you add the fields that you want to get promoted. In the next steps, you will link this field with an Xpath expression of the related field from the XML schema:

Now again, right click the Schema node of the XML schema of your message, and click show promotions. Only this time select the Property Fields tab, import the Property Schema you have just created, and then add the required field from the XML schema to the Property Field list, and link it up to the property you defined in the Property Schema:

These property and distinguished fields, are then populated into the context, in the disassemble stage of a receive pipeline; for example by the Xml and Flat-File disassemblers.

The second method to populate the context, is by System-Level property fields. System-Level property fields are those promoted by adapters and pipelines. Keep in mind that there are no system-level Distinguished Fields. Being system-level properties, you do not have to define a Property Schema using the BizTalk Editor. Instead the adapters and pipelines use out of the box system-level property schemas available in the assembly Microsoft.BizTalk.GlobalPropertySchemas.dll

You can see what type of system property fields are promoted, from within the administration console. From the BizTalk.System application, select the schemas node, and from there you see all available system level property schemas in the GlobalPropertySchemas assembly:

Double click on any of these schemas, and select the Tracking tab, to see all promoted Property Fields:

The other way to populate the context, is to use an API inside a custom pipeline component. Using this method you can promote both Property Fields and Distinguished Fields. From within a pipeline component, you access the context of the message, which is represented by the IBaseMessageContext interface. This interface contains 2 methods – Write and Promote:

Write is used to write Distinguished fields. While Promote is used to promote Property fields

As you can see both of these methods, take the same set of properties:

• The name of the property to write or promote
• The namespace of the property schema
• And the value to give for this property.

Now lets see what values to supply for these parameters, in each of the following cases:

• When promoting custom Property Fields, the name and namespace, are those of the field you defined in the custom property schema.
• When promoting system properties, the namespace is that of the system property schema which is set by biztalk (http://schemas.microsoft.com/BizTalk/2003/system-properties)

For example, here is the code you would use to promote the MessageType for a binary message:

opInMsg.Context.Promote(“MessageType”, “http://schemas.microsoft.com/BizTalk/2003/system-properties”, “http://MyBinaryMessage/#Request”);

Distinguished fields do not use property schemas, however, still there is namespace set by BizTalk (http://schemas.microsoft.com/BizTalk/2003/btsDistinguishedFields)

For example, to write a custom field, you would use the following code:

pInMsg.Context.Write(“PONum”,“http://schemas.microsoft.com/BizTalk/2003/btsDistinguishedFields”,“123”);

Accessing the Context

After having the context populated, you can access it from inside pipeline components and orchestrations.

From a pipeline component, you again use the IBaseMessageContext only this time to read instead of writing or promoting properties. The syntax will not matter on whether you’re reading a property or distinhuihsed field. You just supply in the name and the namespace:

From within the Ox, you use an expression shape; and the syntax vary. You read a Distinguished field, using the dot syntax; while you read a property filed using the bracket syntax, as shown below:

Performance Considerations

There are important performance considerations to keep in mind, when deciding on the usage of Property and Distinguished Fields.

Consideration #1:

• Context properties – both Property and Distinguished fields – are stored in the message box alongside the message.
• Also, if message tracking is enabled, Property Fields are also stored in the Tracking Database.

Tip #1:

• As such, do not blindly populate the message context, as it will increase the database size and causes an additional overhead when storing and retrieving the message into and from the message box.

Consideration #2:

• On the other hand, Property Fields are also written in the Subscription table, and thus participate in the Activation subscription matching process, performed by the Messaging Engine.

Tip #2:

• Therefore, if routing or tracking are not needed, then use Distinguished Fields as they do not incur the overhead of subscription

As a final note around this topic. This discussion might actually make you wonder: if tracking and routing are not required, and I want to access a field form within an Ox, then why even use Distinguished Fields? Cannot I just use Xpath inside my Ox?

The answer is yes you can, but you really should not. Xpath loads the entire message into memory, while using Distinguished fields you get to access the required field directly from the context…which is after all, the reason behind the idea of Distinguished Fields.

Using Message Context for CBR

So by now you know that BizTalk context-based publish and subscribe model, relies on the Property Fields within the Message Context. This is done in two places in BizTalk.

First in an Ox, a Receive Shape which set to receive a message of a certain type, creates a subscription for this Orchestration based on the MessageType and Receive Port ID property fields:

In addition, you can further filter the receive conditions, by using the Filter property of the Receive Shape. Here you can select the property fields to filter on:

A Send Shape inside an Ox, creates a subscription based on the Transport ID property field, for the physical send port that will receive the message:

And finally, when you create a physical send port in the admin console, you can set filter expressions to create the subscription for this port. You can select the system property fields as well as any custom defined property fields:

Why Property Schemas?

A question might pop into your head: why use property schemas? Why not promote the property directly within the message schema, as we do in the case of Distinguished field?

To answer this question, you need to understand a simple, yet important concept called correlation.

Lets consider an example to explain the concept of correlation.

Say that within your orchestration, you need to send an outgoing message with a field O_ID, while you will be expecting a return message indicating the success of the order with a field called OID. Assuming at a certain moment, you will have multiple instances of the Orchestration running, so how will BizTalk know, to which Ox instance the return message shall be handed?

To solve this, BizTalk needs to correlate – or associate – outgoing and return messages, so that the return message is correctly delivered to the corresponding Ox instance.

Correlation is achieved by defining a property or a set of properties that provide the link between messages in the orchestration:

• In our example, we need to correlate the outgoing (O_ID) and the return message (OID) based on the order id.
• However, both message schemas use a different name (O_ID for outgoing and OID for the return message).
• To solve this, a property schema is used to define a promoted property – say its called OrderID – that is then referenced by both outgoing and returning schemas.

Property Schema Base

The final thing you need to know about Property Schemas and Property Fields, is the Property Schema Base.

Promoted Fields, can have one of three base types:

• MessageDataPropertyBase indicates that a property has its value from the message payload, such as OrderID. Such properties are promoted using custom property schemas, and the Property Schema Base property is set to MessageDataPropertyBase – which is also the default value.
•  MessageContextPropertyBase, indicates that a property does not have its value from the message payload. This can be something like, RecordCount which is calculated for example based on the number of records in a message. Such property is also created using a custom property schema, however, the Property Schema Base property is set to MessageContextPropertyBase; and promotion takes place inside a custom pipeline component instead of BizTalk editor. It can be also something like, MessageType, which is set automatically by the pipeline. In this case the pipeline will use the system property schema I showed you before. System property schemas automatically have the Property Schema Base set to MessageContextPropertyBase
• The final type – PartContextPropertyBase – is similar to MessageContextPropertyBase in that it indicates that the property will not have its value coming from the message payload, however, unlike MessageContextPropertyBase where the property is promoted inside the message context; for PartContextPropertyBase the property is promoted inside the message part context.

You can set the Property Schema Base property of a custom Property Schema, from the properties windows, as shown in this image:

As for system property schemas, if you open any of those from the administration console, you will see that as expected all properties have their Property Schema Base set to MessageContextPropertyBase:

Message Parts

All messages in BizTalk are considered to be multipart messages.

In an Ox when you create a message, its as if your creating a multipart message with only 1 part which is the body part. You can also create a multipart message, in which case you will explicitly specify the message parts of which the message is composed of. In this case, 1 part only has to be the body part. This body part will hold the payload of the message – that is the data delivered by the adapter – and will indicate the type of the message.

There are many reasons why you would use a multipart message:

• One common reason is to receive MIME encoded messages which contain attachments. The message body contains the payload and its what you actually want to process in BizTalk while you want to deal differently with the attachments; for example, pass them along without change to the destination endpoint.
• Multipart messages are also used to send and receive emails with attachments. Adapters such as POP3 and SMTP natively support multipart messages. MIME decoder is included as part of POP3 adapter, this means that the adapter will automatically generate the multipart message with the body and attachments from the email. Similarly, SMTP adapter will automatically map a multipart message being sent from BizTalk into the corresponding email body and attachments because it has built-in MIME support. Note that other adapters, such as the File adapter for example, do not support multipart messages. So If you try to send a multipart message through a File adapter, only the body part will be sent, because the adapter does not understand message parts. However, you can use a custom pipeline with the MIME encoding component with the send File adapter. In which case the final message sent out will be MIME encoded and it will include the body and the parts. You will see this in the next demo.
• One other reason to use multipart messages, is actually more of a tip or best practice, as it makes your process more flexible to change. When you use XML schemas to represent a message that is being sent or received; this binds the logical ports to this message type. So if for any reason, you want to change that message type, you will have to delete all port connections and change the message type as well as the port’s operation message type…in short you will have to redo the whole thing again. To eliminate this overhead of change, you can define your message as multipart which has one part only – which is the body part of course. You then set the body part type to the XML schema. This will allow you to change the message type of the body part without doing any changes to the Ox. This is because using this trick, you will add a level of abstraction such that the Multipart message type is now wrapping the Schema message type. This way the Receive and Send shapes are expecting to receive the multipart message type instead of the message schema type; as such they will not be affected by the change.

Message Types

As I have already explained, BizTalk accepts any binary stream of data as messages.

Once inside an Ox, these messages can be one of 4 types:

• XSD Schemas
• Multipart messages
• A new type which we haven’t discussed yet, is .NET class; here you can use one of the .NET Framework classes, or your own .NET class
• And finally, you can the type set to Web message, in case you are adding you add a web reference to an asmx web service. Since you will probably be using WCF with BizTalk rather than ASMX, the chances are you won’t be using this option very often.

Although most of the times you will be using XSD – or multipart with body part type set to XSD, there are situations where .NET classes will be more appropriate.

• First obvious reason is that you already have a .NET class that you want to reuse
• A second more common reason is message creation. When you need to create a message based on XSD from scratch inside an OX, your options are not exactly easy enough. If however your message is based on a .NET class you can construct it using the Expression shape by just initiating an instance of the class, just as you would usually do using C#.
• Final reason, is if you want to use an XMLDocument, either to represent a binary message or an untyped message

An XSD however becomes mandatory in cases of message publication. When the message must be sent to a party in an interoperable scenario, you want to agree with this party about the schema to be exchanged. For these interoperable scenarios, XSD becomes the only viable option.

UnTyped Messages

Using UnTyped messages means that you want to receive or send a message which is not tied to a specific XSD schema or .NET type.

Instead you set your message type to XmlDocument, so that you can receive any type.

Say for example, that you have an Ox where you want to be able to receive more than 1 type of a message. In order to do so, you set the message type to be received to XmlDocument.

Recall that this also means that you can receive a binary message.