Table of contents
To introduce the encoding and decoding process, we describe how a trivial message with only 1 field would be FAST encoded and decoded.
Here's the message we'll start with:
The SOH (ASCII 1) is the FIX separator byte.
This message requires a very simple template (expressed in XML and described in greater detail in later sections):
<template dictionary="1" id="1" name="template1"> <string id="58" name="Text"> <default value=""></default> </string> </template>
The high order bit of each byte of the message is reserved to indicate the end of the field.
So each of the fields described below will only use 7 bit bytes and the 8th bit will be set on the last byte of the field.
To encode the message, we need to produce a Presence Map (PMap) that goes onto the front of the compressed FAST message followed by the encoded fields.
PMap is described in detail in section 3.1, but simply has 1 bit for each field to indicate whether the field has been included in subsequent bytes, or has been omitted.
Note: In our example, if the Text field is omitted, it will default to an empty string. (Because of the xml element: <default value=""/> in the above template.)
When encoding the message the template ID is usually put into the message right after the PMap. (We'll discuss this more later.) This requires a PMap bit, too.
So for our trivial one field message, we need a PMap with 2 bits:
- indicate template ID is in the message
- indicate Text field is in the message
- These are the high order bits and the rest are zeros: 110 0000.
This is a 1 byte PMap, so its 8th bit is set to 1 to indicate it is the last byte of the PMap: 1110 0000 = 0xE0.
- The next field is the template field (for template 1): 000 0001.
Again, this is a 1 byte field so we set the 8th bit to indicate the end: 1000 0001 = 0x81.
- The next field is the text field contents: "HelloWorld".
In the hex format it is: H=0x48, e=0x65, l=0x6C, l=0x6C, o=0x6F, W=0x57, o=0x6F, r=0x72, l=0x6C, d=0x64.
The last byte will have its high order bit set to "1". Using this, we have the following binary representation of our text field:
01001000 01100101 01101100 01101100 01101111 01010111 01101111 01110010 01101100 11100100
The encoded elements strung together are shown below:
Picture 2.1 - "HelloWorld message encoding"
Looking at the just the above binary data and the template file, we can reconstruct the original message content by reversing the encoding process.
Our input (encoded) FAST message is:
HEX format: 0xE0 0x81 0x48 0x65 0x6C 0x6C 0x6F 0x57 0x6F 0x72 0x6C 0xE4 Binary format: 11100000 10000001 01001000 01100101 01101100 01101100 01101111 01010111 01101111 01110010 01101100 11100100
Lets decode it step by step:
- Extracting Message PMap. Message PMap must be present before each encoded message so the first step of decoding process must be defining PMap.
We get the first byte from the stream: "11100000", check its stop bit (high order bit or first bit).
It is set to "1", so our encoded PMap consists of 1 byte: 11100000. We remove the stop bit which yields the decoded message PMap: 1100000
- Decoding template ID. We get the first bit from our PMap which is "1" and it indicates presence of template ID value in the input message.
Using the same way as for PMap decoding we decode template ID:
Template ID is used to define FAST message template which contains refferences for field decoding.
- We get the next byte after PMap from the stream. It is the second byte: "10000001" and the stop bit of this byte is "1". So our template ID is the 1 byte value.
- After removing stop bit from template ID encoded value we receive: "0000001" = "1" which is the actual value of template ID.
- "HelloWorld" field decoding. The next bit (second bit in this case) from our PMap is set to "1" and it indicates that the "HelloWorld" field is present in the input message too.
As our text field consists of more than one byte we must get next input bytes (after template ID) one by one. This is required to find the byte which has stop bit set to "1" and indicates the end of "HelloWorld" encoded field.
Byte which has high order bit (stop bit) set to "1" is the last byte in input stream so our encoded text field is:
01001000 01100101 01101100 01101100 01101111 01010111 01101111 0111001001101100 11100100
Stop bit decoding of this text field is in following. We must replace the stop bit of last byte of text field from "1" to "0".
After this we receive actual value of our text field:
Binary format: 01001000 01100101 01101100 01101100 01101111 01010111 01101111 01110010 01101100 01100100 HEX format: 0x48 0x65 0x6C 0x6C 0x6F 0x57 0x6F 0x72 0x6C 0x64 Character format: HelloWorld
We combine the decoded output fields yielding message: 58=HelloWorld<SOH>.
The following diagram summarized these steps:
Picture 2.2 - "HelloWorld message decoding"
The next section will describe the different FAST elements (such as the FAST Template and the PMap) in much greater detail and walk through more complex decoding examples.