Skip to content
Terminal Ledger Desk
Directory

Fix Protocol Sbe Ouch Session Latency

Nasdaq TotalView-ITCH 5.0 & MoldUDP64: L3 Multicast Order Book Reconstruction

Nasdaq TotalView-ITCH 5.0 broadcasts over 500 million daily order-by-order events inside MoldUDP64 multicast datagrams, enabling colocated trading engines to rebuild exact L3 queue priority in sub-microsecond latency.

By Marcus Thorne, CQFReview not recorded
Nasdaq TotalView-ITCH 5.0 & MoldUDP64: L3 Multicast Order Book Reconstruction — Terminal Ledger Desk 2026 quantitative & supervisory data card
Terminal Ledger Desk (FIX 4.4/5.0 SP2, CME iLink 3 SBE, Nasdaq OUCH/ITCH & Session Recovery): L3 equity book reconstruction via 1.0 GiB HugePage OrderRef slabs, A/B multicast, and Glimpse spins.Source: Terminal Ledger Desk Institutional Research DeskLicense: CC-BY-4.0

Building an institutional US equities Smart Order Router or market-making engine requires consuming order-by-order (Level 3) depth rather than aggregated SIP top-of-book quotes. Nasdaq TotalView-ITCH 5.0 delivers every individual resting order addition (A/F), partial execution (E/C), cancellation (X), deletion (D), and replacement (U) encapsulated inside MoldUDP64 IP multicast packets, allowing engines inside Carteret and Equinix NY4 to compute exact queue-ahead share volume in under 190 nanoseconds per tick.

MoldUDP64 Datagram Coalescing and 64-Bit Sequence Tracking#

At peak 09:30:00 and 16:00:00 auction bursts, transmitting one Ethernet/IP/UDP header per 36-byte ITCH message would overwhelm switch packet-per-second budgets. MoldUDP64 solves packet overhead by packing multiple length-prefixed ITCH messages into a single MTU-sized (<= 1,400 byte) UDP payload preceded by a 20-byte header: 10-byte Session, 8-byte big-endian SequenceNumber of the first contained message, and 2-byte MessageCount.

If a datagram carries SequenceNumber = 10,500,001 and MessageCount = 18, the next datagram on that multicast group MUST begin at 10,500,019. Setting MessageCount = 0 denotes an end-of-millisecond heartbeat carrying the next expected sequence number so idle subscribers detect silent drops within 1 millisecond—far tighter than TCP FIX Heartbeat and TestRequest timers.

Nasdaq TotalView-ITCH 5.0 MoldUDP64 multicast demultiplexing and O(1) L3 order book state machine
Nasdaq TotalView-ITCH 5.0 MoldUDP64 multicast demultiplexing and O(1) L3 order book state machineVerified Exhibit

Two-Table L3 Book Reconstruction: OrderID Map + Price-Level Tree#

Observe a critical bandwidth optimization in ITCH 5.0: while Add Order (Type A) provides OrderReferenceNumber (8B), BuySellIndicator, Shares (4B), StockLocate (2B), and Price (4B), subsequent Order Cancel (X) and Order Delete (D) messages transmit only OrderReferenceNumber, omitting Stock, Side, and Price! Consequently, an L3 book builder cannot route ticks by symbol alone.

High-speed engines maintain a flat global open-addressed hash table (or segmented slab indexed by OrderReferenceNumber) storing { stockLocate, side, priceNodePtr, remainingShares } alongside per-symbol price-level doubly-linked lists. When Delete (D) arrives, a single pointer dereference decrements the price level in O(1) time without tree traversal—mirroring the zero-copy ethos of our FIX, SBE & OUCH Session Architecture Guide.

Quantitative Distribution · Payload Bytes
Payload BytesL3 Order Book Mutation
01Stock Directory39 bytes02Add Order (No MPID / MPID)36 / 40 bytes03Order Executed31 bytes04Order Cancel / Delete23 / 19 bytes05Order Replace35 bytes
ITCH 5.0 Message TypeCodePayload BytesKey Wire FieldsL3 Order Book Mutation
Stock DirectoryR39 bytesStockLocate (2B), Stock (8B), LotSizeInitializes Locate -> SymbolBook slot at 04:00 EST
Add Order (No MPID / MPID)A / F36 / 40 bytesOrderRefNum (8B), Side, Shares, PriceInserts slab node + appends tail of price FIFO
Order ExecutedE31 bytesOrderRefNum (8B), ExecutedShares, MatchNumDecrements resting head; removes node if Qty == 0
Order Cancel / DeleteX / D23 / 19 bytesOrderRefNum (8B) (+ CanceledShares on X)O(1) slab lookup -> unlinks doubly-linked node
Order ReplaceU35 bytesOrigOrderRefNum, NewOrderRefNum, Shares, PriceDeletes OrigRef + inserts NewRef at tail priority

MoldUDP64 Gap Recovery vs Nasdaq Glimpse Mid-Day Spin Synchronization#

Recovering from UDP buffer drops or restarting a strategy process at 11:15 EST requires coordinating live multicast buffering with unicast recovery services.

Formula or model from the articleNot independently verified
M_slab_ram = N_max_live_orders * sizeof(OrderNode_Aligned)
  • BWmold = Ritchpeak * (Litchavg + 2 + (Lethipudp + 20) / K_coalesce) * 8
  • Where:
  • Nmaxlive_orders = 33,554,432 slots (2^25 pre-allocated slab nodes)
  • sizeof(OrderNode) = 32 bytes (nextIdx 4B, prevIdx 4B, price 4B, shares 4B, refNum 8B, locate 2B, flags 6B)
  • Ritchpeak = 6,800,000 messages/second (microburst rate)
  • Litchavg = 27 bytes; Kcoalesce = 28 messages/MoldUDP64 packet; Lethipudp = 46 bytes
  • Worked Memory & Microburst Bandwidth Sizing:
  • Mslabram = 33,554,432 * 32 bytes = 1,073,741,824 bytes (1.00 GiB pinned in 512 x 2MB HugePages)
  • Bytes/msg = 27 + 2 + (66 / 28) = 31.36 bytes/message
  • BW_mold = 6,800,000 * 31.36 * 8 = 1.706 Gbps per multicast feed (3.41 Gbps dual A+B).

Why Unicast UDP Retransmission Requests Must Never Run on the Hot Path#

MoldUDP64 provides a unicast UDP Retransmission Server where subscribers send a 20-byte request specifying missed SequenceNumber and MessageCount. However, exchange retransmission servers intentionally throttle and deprioritize unicast repair traffic during market spikes to protect primary multicast fans. Relying on unicast repair inside Equinix NY4/Carteret racks stalls book validity for 5 to 45 milliseconds.

Instead, Tier-1 desks subscribe simultaneously to dual physically diverse multicast fabrics (Feed A and Feed B) and perform first-arrival sequence arbitration in NIC/FPGA memory—identical in principle to CME MDP 3.0 SBE A/B channel arbitration. Only if both A and B drop the identical sequence simultaneously—or when cold-starting an engine mid-session—does the process connect to Nasdaq Glimpse over SoupBinTCP to download a point-in-time L3 book spin aligned to a specific ITCH sequence number before draining buffered multicast ticks into Nasdaq OUCH 5.0 order entry.

ITCH 5.0 book recovery duration across A/B feed arbitration, unicast UDP retransmission, Glimpse spin, and full-day Mold replay
ITCH 5.0 book recovery duration across A/B feed arbitration, unicast UDP retransmission, Glimpse spin, and full-day Mold replayVerified Exhibit

Memory Sizing & Kernel Socket Buffer Engineering for 500M+ Daily Ticks#

Preventing kernel RcvbufErrors (netstat -su) during FOMC volatility spikes requires tuning three Linux and application layers audited against SEC market-data integrity rules:

  1. Socket Receive Ring (rmem_max): Raise net.core.rmem_max = 134217728 (128 MB) or bypass the kernel completely onto Solarflare ef_vi / Mellanox VMA user-space RX rings.
  2. 2-Byte StockLocate Indexing: Never key symbol books by 8-byte ASCII tickers ("AAPL "); ingest Stock Directory (Type R) at 04:00 EST and index a contiguous 16,384-element C++ array directly by 16-bit StockLocate (1..N).
  3. Queue-Depletion Signal Validation: Feed real-time L3 queue-imbalance alpha metrics into the Algorithmic Kelly Allocation Calculator and compare multi-asset terminal architectures in our FIX, SBE & OUCH Topic Hub and proprietary terminal directory.

Frequently asked questions

Why does ITCH Order Replace (Type U) change the OrderReferenceNumber instead of editing in place?

On Nasdaq, modifying price or increasing share size forfeits time priority and places the new order at the back of the price queue. Assigning a brand-new monotonic OrderReferenceNumber preserves strict chronological ordering across all resting liquidity.

How does Nasdaq Glimpse align its point-in-time snapshot with live MoldUDP64 multicast traffic?

An engine first starts recording live MoldUDP64 packets into a memory ring buffer, then logs into Glimpse to receive all currently resting orders followed by an End-of-Snapshot message citing sequence S_snap. The engine discards buffered multicast messages <= S_snap and applies S_snap + 1 onward.

What is the difference between ITCH Add Order Type A and Type F?

Type A (36 bytes) represents an anonymous non-attributed resting order displayed under the NSDQ generic market participant code, whereas Type F (40 bytes) appends a 4-character Market Participant Identifier (MPID) for attributed market-maker quotes.

How does TotalView-ITCH 5.0 latency compare against the consolidated UTP SIP Level-1 feed?

Colocated Carteret consumers decode ITCH 5.0 directly off the matching engine multicast fan in ~12–18 microseconds, beating the consolidated UTP SIP top-of-book quote feed by an additional geographic and aggregation margin during cross-market sweeps.

Publication record

Published
14 November 2026
Updated
14 November 2026
Author
Marcus Thorne, CQF
Review
Review not recorded
Automated fact check
Passed in the stored pipeline record
Source URLs
3 recorded

AI assistance recorded: anthropic/claude-opus-4.6. This field does not establish human review or source verification.

Academic & Institutional Attribution

Cite This Research & Embed Live Data Card

Open Attribution License (CC BY 4.0)

Standard academic & institutional research paper citation format.

Marcus Thorne, CQF. (2026, November 14). Nasdaq TotalView-ITCH 5.0 & MoldUDP64: L3 Multicast Order Book Reconstruction. Terminal Ledger Desk. https://terminalledgerdesk.com/reports/nasdaq-totalview-itch-5-0-moldudp64-multicast-book-reconstruction

Regulatory Directory Cross-Reference

Referenced Institutions & Regulatory Registries

Referenced Supervisory Authorities

Explore Fix Protocol Sbe Ouch Session Latency Hub (20) →

Terminal Ledger Desk Data, Benchmarks & Contributor Records

Nasdaq TotalView-ITCH 5.0 & MoldUDP64 Book Reconstruction | Terminal Ledger Desk