Context Mixing Visualizer

PAQ / Logistic Mixing Compression & Decompression Interactive Demo

How it Works Interactive Simulator
1. Multiple Context Models

Instead of relying on a single context length, Context Mixing runs multiple predictors in parallel (e.g., Order-0 overall bias, Order-1 previous bit, Order-2 previous byte/word).

2. Logistic Mixing

Predictions $P_i(1)$ are stretched into logit domain $x_i = \ln(P_i/(1-P_i))$, combined with adaptive weights $w_i$, and squashed back via $P(1) = 1/(1+e^{-\sum w_i x_i})$.

3. Online Learning

After each bit $y \in \{0, 1\}$ is processed, model counts update, and weight values adapt using gradient descent: $w_i \leftarrow w_i + \eta \cdot x_i \cdot (y - P(1))$.

Simulation Controls & Input

Speed: 700ms
Bit Progress: 0 / 0
Bit Stream Visualization Grey = Pending | Highlighted = Active | Green = Processed

Individual Context Models Predictions

Bit #0
Model 0: Order-0 Context (Overall) Context: [Global Frequency]
Counts (0 / 1) 1 : 1
Probability $P_0(1)$ 0.500
Stretched $x_0$ 0.000
Model 1: Order-1 Context (1 Bit) Active Context: "None"
Context Counts (0 / 1) 1 : 1
Probability $P_1(1)$ 0.500
Stretched $x_1$ 0.000
Model 2: Order-2 Context (2 Bits) Active Context: "None"
Context Counts (0 / 1) 1 : 1
Probability $P_2(1)$ 0.500
Stretched $x_2$ 0.000

Logistic Mixing Calculation

Weighted Sum ($S$):
$S = (w_0 \cdot x_0) + (w_1 \cdot x_1) + (w_2 \cdot x_2) = 0.000$
Squash Function: $P(1) = \frac{1}{1 + e^{-S}}$
P(0) = 0.5000 P(1) = 0.5000

Adaptive Model Weights

$w_0$ (Order-0) 1.0000
$w_1$ (Order-1) 1.0000
$w_2$ (Order-2) 1.0000
Weight Update Delta ($\Delta w_i = \eta \cdot x_i \cdot (y - P(1))$):
Awaiting first bit step...

Compression Efficiency

Total Compressed Size 0.00 bits
Bitrate Ratio 0.00 bits / bit

Note: Using ideal arithmetic/range coding, each predicted bit $y$ requires exactly $-\log_2 P(y)$ bits of compressed storage.

Decompressor Synchronization

The decompressor reconstructs the exact same context tables and weights bit-by-bit in real time:

[Awaiting stream start...]
Status: READY

Educational Guide: How Context Mixing Works

1. The Problem with Single Context Models

Traditional predictive compressors (like PPM or Huffman) use a fixed context order. High-order contexts (e.g., last 4 characters) are extremely accurate when matches are found, but fail completely when encountering unseen sequences. Low-order contexts (e.g., last 1 character) are reliable but less specific.

Context Mixing solves this dilemma: It runs multiple independent models of varying orders simultaneously and dynamically combines their predictions.

2. Why Logistic (Non-linear) Mixing?

Instead of simple linear averaging $P_{mix} = \frac{P_1 + P_2}{2}$, modern compressors like PAQ convert probabilities into log-odds (logit domain) using the $stretch$ function:

$stretch(p) = \ln\left(\frac{p}{1-p}\right)$

In logit space, confident predictions (near 0 or 1) stretch to large negative/positive infinity. When summed with weights, a single highly confident model can overrule multiple uncertain models!

3. Adaptive Weight Updates (Online Learning)

The mixer assigns weight $w_i$ to each model. After predicting bit $y \in \{0, 1\}$, weights update using stochastic gradient descent to minimize cross-entropy loss:

$w_i \leftarrow w_i + \eta \cdot x_i \cdot (y - P(1))$

Models that predicted correctly receive increased weight, while models that gave wrong high-confidence predictions are quickly penalized.

4. Symmetric Decompression Guarantee

Data compression is lossless because the Decompressor executes the exact same mathematical logic:

  • Calculates predictions $P_i(1)$ using the exact same context history.
  • Applies the same weights $w_i$ to get $P(1)$.
  • Reads the actual encoded bit $y$ from the arithmetic stream.
  • Updates context counts and weights identical to the compressor.