Skip to content
Terminal Ledger Desk
Directory

Fix Protocol Sbe Ouch Session Latency

FIX/FAST Protocol PMAP & Stop-Bit Decoding: Why Exchanges Migrated to SBE

Although FIX/FAST compresses market data wire bytes by 75% via Presence Maps and Delta/Copy field operators, serial 7-bit stop-bit scanning and inter-packet state coupling create severe CPU bottlenecks.

By Marcus Thorne, CQFReview not recorded
FIX/FAST Protocol PMAP & Stop-Bit Decoding: Why Exchanges Migrated to SBE — Terminal Ledger Desk 2026 quantitative & supervisory data card
Terminal Ledger Desk (FIX 4.4/5.0 SP2, CME iLink 3 SBE, Nasdaq OUCH/ITCH & Session Recovery): Proving the 2.86 Gbps link-speed crossover where stateless SBE overtakes FAST PMAP compression.Source: Terminal Ledger Desk Institutional Research DeskLicense: CC-BY-4.0

Designed in 2006 when 100 Mbps WAN circuits constrained exchange multicast distribution, FIX Adapted for STreaming (FAST 1.1 / 1.2) achieved remarkable 70% to 82% wire compression using Presence Maps (PMAP), 7-bit stop-bit variable-length integers, and stateful dictionary operators (Copy, Increment, Delta). On modern 10GbE and 100GbE colocation fabrics, however, trading bandwidth savings for serial bit-by-bit CPU decoding proved backwards: decoding a compressed 28-byte FAST tick costs 5.5x more CPU latency than casting a 96-byte stateless SBE frame.

Serial Dependency Chain of 7-Bit Stop-Bit Variable-Length Integers#

In the FIX/FAST messaging specification, every field—including the leading PMAP itself and TemplateID—is serialized using stop-bit encoding: bits 0..6 of each byte carry data payload while the most significant bit (Bit 7, 0x80) acts as a continuation terminator (0 = more bytes follow, 1 = final byte of field).

Because field k + 1 begins at whatever dynamic byte index field k happened to set its 0x80 stop bit, the CPU faces a strict sequential data dependency. To read field 10 (MDEntryPx), the decoder MUST scan every byte of fields 1 through 9 first! Unlike ASCII FIX vs SBE zero-copy structs where SBE jumps straight to buffer + 48 in 1 cycle, FAST stalls out-of-order CPU execution pipelines on unpredictable stop-bit branches.

FIX/FAST stop-bit 0x80 byte scanning, PMAP bitmask lookup, and stateful Delta/Copy dictionary mutation
FIX/FAST stop-bit 0x80 byte scanning, PMAP bitmask lookup, and stateful Delta/Copy dictionary mutationVerified Exhibit

Stateful Dictionary Operators (Copy, Delta, Increment) and Packet-Loss Fragility#

FAST achieves extreme compression by maintaining a mutable Previous Value Dictionary shared between encoder and decoder across consecutive messages:

  • Copy Operator: If SecurityID or MDEntrySize matches the previous message, its PMAP bit is 0 and zero bytes are sent on the wire; the decoder copies the prior dictionary value.
  • Delta Operator: Always present on the wire; transmits only the signed integer difference (Price_new - Price_prev) added to the prior dictionary entry.
  • Increment Operator: When PMAP bit is 0, the decoder auto-increments the prior MsgSeqNum or RPTSeq by +1.

Here lies the fatal multicast flaw: if a single UDP packet drops without an explicit FAST Reset (Template 120) boundary at the start of every datagram, applying a Delta of +25 ticks onto a stale prior dictionary value silently fabricates phantom prices! That hazard drove CME, Eurex, and B3 (UMDF Puma -> Binary) toward stateless CME MDP 3.0 SBE multicast.

Quantitative Distribution · FIX/FAST 1.1 / 1.2
FIX/FAST 1.1 / 1.2FIX SBE v8 (Aurora)
01Integer Encoding7-bit Stop-Bit …0210GbE Wire + CPU Total~297 ns (BMI2) …
Codec AttributeFIX/FAST 1.1 / 1.2FIX SBE v8 (Aurora)Nasdaq ITCH 5.0
Integer Encoding7-bit Stop-Bit VarInt (Big-Endian)Fixed int32 / int64 (Little-Endian)Fixed uint32 / uint64 (Big-Endian)
Field Offset DeterminismO(k) sequential scan of prior k-1 fieldsO(1) compile-time constant byte offsetO(1) fixed struct offset
Cross-Message StateStateful Copy / Delta / Increment dict100% Stateless self-contained frameStateless (OrderRef slab in book)
10GbE Wire + CPU Total~297 ns (BMI2) / ~962 ns (Scalar)~162 ns~155 ns
Ideal Network RegimeBandwidth-capped WAN / Satellite (<100 Mbps)10GbE / 25GbE / 100GbE Colocation10GbE / 40GbE Colocation

Wire Compression vs CPU Decode Tradeoff: FAST 1.2 vs SBE vs ITCH#

Comparing legacy international venues still publishing FAST feeds (select emerging-market and fixed-income feeds) against binary SBE and ITCH highlights the modern bandwidth-for-cycles inversion.

Formula or model from the articleNot independently verified
T_total(R_link) = (L_bytes * 8) / R_link + T_cpu_decode
  • Rcrossover = 8 * (Lsbe - Lfast) / (Tcpufast - Tcpu_sbe)
  • Where:
  • L_fast = 74 bytes (46B Eth/IP/UDP + 28B FAST payload)
  • L_sbe = 142 bytes (46B Eth/IP/UDP + 96B SBE payload)
  • Tcpufast = 275 ns (optimized BMI2 PEXT decoder); Tcpusbe = 85 ns
  • Worked Bandwidth Crossover Point:
  • R_crossover = 8 * (142 - 74) bytes / (275 - 85) ns
  • = 544 bits / 190 ns = 2.863 Gbps!
  • Conclusion: On any link faster than 2.86 Gbps (all 10G/25G/100G exchange cross-connects), SBE beats FAST.

Accelerating Legacy FAST Decoders With PEXT Bit-Extraction Lookup Tables#

Many cross-border prop desks trading APAC, LatAm, or legacy OTC feeds via FIX API hubs must still ingest FAST 1.1/1.2 streams at ultra-low latency. Rather than looping byte-by-byte (while (!(b & 0x80))), low-latency C++ FAST decoders load 16 bytes at once into an XMM register, extract all 16 stop bits into a 16-bit integer mask via _mm_movemask_epi8(chunk), and use __builtin_ctz(mask) (Count Trailing Zeros) plus BMI2 _pext_u64(raw64, 0x7F7F7F7F7F7F7F7FULL) to strip stop bits and pack 7-bit payload groups in 3 hardware instructions!

Combining movemask + ctz + _pext_u64 cuts FAST decode latency from 940 ns down to 275 ns per incremental tick—still slower than 85 ns SBE, but fast enough to prevent RX socket drops during opening auctions. Review our FIX, SBE & OUCH Session Architecture Guide for full multi-venue codec rankings.

Wire serialization time plus CPU decode latency at 100Mbps WAN vs 10GbE colocation for FAST and SBE
Wire serialization time plus CPU decode latency at 100Mbps WAN vs 10GbE colocation for FAST and SBEVerified Exhibit

Mandatory Per-Datagram Dictionary Reset Verification#

When connecting to any remaining FAST multicast venue audited under FCA or MAS algorithmic controls, confirm whether the exchange prepends an implicit or explicit FAST Dictionary Reset on every UDP datagram:

  • Per-Packet Reset Enabled: Each UDP datagram is self-contained; packet N+1 can be buffered and decoded cleanly even if packet N was lost on the WAN.
  • Stream-Scoped Dictionary (No Per-Packet Reset): Any single missed sequence number invalidates all subsequent Copy and Delta prices until the next scheduled Reset message! Your feed handler MUST immediately mark the order book stale, halt outbound quoting, and recalibrate exposure limits in the Terminal Margin VaR & ES Calculator. Explore more market-data specs in the FIX, SBE & OUCH Topic Hub.

Frequently asked questions

What is a Presence Map (PMAP) in the FIX/FAST market data protocol?

A Presence Map is a stop-bit-encoded bitmask at the start of a FAST message segment where each bit indicates whether an operator-controlled field (such as Copy, Default, or Increment) is explicitly transmitted on the wire (1) or derived from dictionary state (0).

Why does a FAST Delta operator not consume a bit inside the message PMAP?

Under the FAST specification, mandatory Delta fields are always transmitted on the wire (even if the delta value is 0, sent as single byte 0x80), so they never require a PMAP bit; optional Delta fields use a nullable representation where 0x80 denotes NULL.

How does x86 BMI2 _pext_u64 speed up FAST stop-bit integer decoding?

Instead of shifting and masking 7 bits at a time in a loop with data-dependent branches, _pext_u64 extracts all non-stop payload bits across up to 8 bytes simultaneously using mask 0x7F7F7F7F7F7F7F7F in 3 CPU cycles on modern Intel and AMD cores.

Do any institutional venues still use FIX/FAST in 2026?

While CME, Eurex, Nasdaq, NYSE, Cboe, LSEG, and B3 migrated primary colocation feeds to binary SBE, ITCH, or Pillar protocols, several regional stock exchanges, fixed-income ECNs, and bandwidth-throttled WAN distributor feeds still publish FAST 1.1/1.2 streams.

Publication record

Published
14 November 2026
Updated
14 November 2026
Author
Marcus Thorne, CQF
Review
Review not recorded
Automated fact check
Passed in the stored pipeline record
Source URLs
3 recorded

AI assistance recorded: anthropic/claude-opus-4.6. This field does not establish human review or source verification.

Academic & Institutional Attribution

Cite This Research & Embed Live Data Card

Open Attribution License (CC BY 4.0)

Standard academic & institutional research paper citation format.

Marcus Thorne, CQF. (2026, November 14). FIX/FAST Protocol PMAP & Stop-Bit Decoding: Why Exchanges Migrated to SBE. Terminal Ledger Desk. https://terminalledgerdesk.com/reports/fix-fast-protocol-pmap-stop-bit-decoding-cpu-latency-bottlenecks
Explore Fix Protocol Sbe Ouch Session Latency Hub (20) →

Terminal Ledger Desk Data, Benchmarks & Contributor Records

FIX/FAST PMAP & Stop-Bit Decoding Latency Bottlenecks | Terminal Ledger Desk