Streaming HDC-MiniROCKET: shifting a sequence vector in O(D)
Part 3 of 4 of HDC Time Series. 1. Reproducing HDC-MiniROCKET · 2. Choosing the time scale · 3. Streaming HDC-MiniROCKET · 4. Finding the shift · all my HDC posts
Generated with Claude Code, and I am not ashamed. My own blog posts of this kind are part of the training data and have sculpted its style.
Draft, October 2026, not published. Every result on this page is read from results/*.json; the page only averages, compares and formats them. The few other numbers are code constants, dataset facts, or my first cost run (in the git history), and say so.
This builds on the work of Kenny Schlegel, Peer Neubert and their coauthors. HDC-MiniROCKET (Schlegel, Neubert and Protzel, IJCNN 2022) turns a time series into one vector: each time step’s MiniROCKET features are bound to a vector that encodes the time step, and the results are summed. In 2026 Schlegel, Rachkovskij, Kleyko, Loutfi, Streif and Osipov gave a real-valued position encoding, the Sinusoid variant (kind B), with an exact shift operator ⋄ (arXiv 2608.28334). The last paragraph of their discussion (Sec. 7, p. 21 of v1) names the thing I try here:
“Finally, the shift-equivariance property is particularly relevant in streaming or continuously processed data scenarios. In such settings, new data arrive sequentially, and the temporal representation must be updated by shifting the existing sequence representation, similar to recurrent or state-space models. An equivariant encoding allows this shift to be applied directly in the representation space without recomputing the full sequence encoding, potentially enabling more efficient processing of long or continuously evolving data streams.”
So this is their idea, worked out and measured. I would rather hear whether they are already on it than publish around them. Notation (D, T, s, kind B, ⋄) is as in Part 1.
What I found.
- The update is exact in real arithmetic and costs O(D) per sample, for an unbounded stream and for a sliding window. Exact means: the state equals the bundling of the per-step features that were fed in. It equals batch HDC-MiniROCKET of a series prefix only after a read-time flush that adds the zero-padded right edge. Without the flush the state differs, and a classifier trained on whole series notices.
- For a sliding window the stream does not equal batch HDC-MiniROCKET of the window. That is the limit from Part 1 (the shift operator moves positions, not signals) in streaming form. I have not measured the gap.
- In floating point the plain recursion drifts: linearly in time on an unbounded stream, about 1 % relative error after a million float32 steps. A known trick from signal processing, the modulated sliding DFT, keeps the relative float32 error below 10⁻⁴ over a million steps.
- Against re-encoding the window on every sample, the update wins by tens to hundreds of times at D = 9,996 and windows of 256 samples or more, in numpy on a loaded machine. At short windows (64 samples, or up to 256 at small D) the gain is a few times at most, and in one case the stream is slower. When a prediction is needed only every k samples, the gain shrinks k-fold.
- Early classification: the streamed state is good enough. With one classifier per prefix length, the cheap causal state (no flush) is within about a point of the batch prefix. The time encoding does not help on this 20-dataset subset, neither on prefixes nor on full series (one seed, fixed s: no visible effect, not proof of none).
The update
Write one frequency ω of the kind-B encoding as a complex number: the cos half is the real part and the sin half the imaginary part. A sequence is then z = Σ f_t e^{iθt}, where f_t is the feature at step t and θ = ω s / L is the angle per step. L is the length that s refers to: Part 1’s positions p_t = s t / (T − 1) mean L = T − 1 for a series of length T, and the encoder’s default for a window of length W is L = W − 1 (Experiment 1 used L = W; for drift this makes no difference). The shift operator ⋄ is multiplication by e^{iθδ}. Put the newest sample at position 0 and older ones at negative positions. Then one new sample means
\[z_{t+1} = e^{-i\theta} z_t + f_{t+1},\]
and a sliding window of length W, with the oldest sample at position 0, means
\[z_{t+1} = e^{-i\theta} z_t + f_{t+1} e^{i\theta(W-1)} - f_{t-W+1} e^{-i\theta}.\]
Both are exact for the features that were fed in, because the shift is exact and bundling is a sum. Per frequency that is one complex multiply and two multiply-adds. The derivation, with MiniROCKET’s per-dilation delays, is in docs/DESIGN.md.
MiniROCKET’s pooling, PPV (the proportion of positive values), is not linear, but its nonlinearity is a threshold applied to each time step on its own, with a bias fixed at training time. After the threshold, PPV is a mean and HDC-MiniROCKET a sum. So the streaming PPV is a running integer count, exact with no rounding, and streaming HDC-MiniROCKET is the recursion above with the ±1 features as input.
The edges are where stream and batch part ways. Half of MiniROCKET’s features use zero padding, and the dilation d of a feature can reach (T − 1)/8.
- Right edge. The outputs for the last 4d samples need samples that have not arrived. The causal state leaves them out. A flush adds them when a prediction is made, treating the future as zeros the way batch does, at a cost of up to O(D d_max) per read. I have specified the flush and checked its result (below), but not implemented it as a streaming operation that computes the tail from raw samples, and not timed it.
- Left edge of a sliding window. Batch on the window sees zeros before the window start. The stream computed those features from the real samples that were there. So a windowed stream is not batch on the window, flush or not. This is Part 1’s finding about the shift operator, met again in a stream: ⋄ moves positions exactly, but when the window slides, MiniROCKET’s edge features change, and no shift accounts for that. The recursion never asks ⋄ to predict those features (each feature is computed once, from real samples), so nothing is wrong. But “stream = batch” is true only for a growing prefix, not for a window. How much the difference matters is open.
For a growing prefix I checked the chain against the shared reference implementation (hdc-ts-core). Streaming core’s per-step features with their delays, followed by the right-edge outputs as core computes them with zero padding, gives core’s batch vector to float32 rounding. That checks the delays, positions and bookkeeping. There is a unit test on synthetic series, for the full series and for prefixes, and a check on real data below. “Batch on a prefix of length t” means core’s MiniROCKET fitted at the full length T, applied to the first t samples with zero padding after them, with the first t rows of the length-T position table. Core computes this for t ≥ 4 d_max + 1, roughly T/2. For shorter prefixes, the same formula is my definition.
Experiment 1: does it stay exact?
I streamed a million ±1 element vectors through the encoder in float32 and float64, with D = 256 frequencies, and compared the result with the same sum computed in 80-bit extended precision (with the same float64 angles θ, so it measures the arithmetic of the recursion).
- recurrent is the formula above.
- anchored stores every sample at its absolute time and applies one exact shift when the vector is read. Its phasor e^{iθt} is reset to the exact value every K steps. That is Duda’s modulated sliding DFT (2010) at random frequencies. Every 65,536 steps (a code constant) it moves its time origin, see below.
- rebase recomputes the window from a buffer every 16 W steps.
There are two element streams: iid ±1, and persistent, where each component flips with probability 0.002 per step.
| stream | elements | encoder | rel. error at 10⁶ | max over the run | log-log slope, t ≥ 10⁴ |
|---|---|---|---|---|---|
| unbounded | iid | anchored K=1024/float32 | 1.9e-05 | 1.9e-05 | 0.13 |
| unbounded | iid | anchored K=1024/float64 | 3.7e-14 | 3.7e-14 | 0.14 |
| unbounded | iid | anchored K=64/float32 | 1.7e-05 | 1.7e-05 | 0.49 |
| unbounded | iid | anchored K=64/float64 | 3.3e-14 | 3.3e-14 | 0.43 |
| unbounded | iid | recurrent/float32 | 8.8e-03 | 8.8e-03 | 0.98 |
| unbounded | iid | recurrent/float64 | 1.9e-11 | 1.9e-11 | 1.01 |
| unbounded | persistent | anchored K=1024/float32 | 2.2e-05 | 2.2e-05 | 0.15 |
| unbounded | persistent | anchored K=1024/float64 | 3.8e-14 | 3.8e-14 | 0.14 |
| unbounded | persistent | anchored K=64/float32 | 2.1e-05 | 2.1e-05 | 0.43 |
| unbounded | persistent | anchored K=64/float64 | 3.5e-14 | 3.5e-14 | 0.47 |
| unbounded | persistent | recurrent/float32 | 1.1e-02 | 1.1e-02 | 1.02 |
| unbounded | persistent | recurrent/float64 | 1.8e-11 | 1.8e-11 | 0.98 |
| window | iid | anchored K=1024/float32 | 6.5e-05 | 6.5e-05 | 0.39 |
| window | iid | anchored K=1024/float64 | 3.1e-13 | 3.1e-13 | 0.63 |
| window | iid | anchored K=64/float32 | 3.5e-05 | 3.5e-05 | 0.52 |
| window | iid | anchored K=64/float64 | 3.0e-13 | 3.0e-13 | 0.79 |
| window | iid | recurrent+rebase/float32 | 8.1e-06 | 6.5e-05 | – (sawtooth) |
| window | iid | recurrent+rebase/float64 | 1.6e-14 | 1.3e-13 | – (sawtooth) |
| window | iid | recurrent/float32 | 5.2e-04 | 5.2e-04 | 0.51 |
| window | iid | recurrent/float64 | 1.1e-12 | 1.1e-12 | 0.53 |
| window | persistent | anchored K=1024/float32 | 6.6e-05 | 6.6e-05 | 0.37 |
| window | persistent | anchored K=1024/float64 | 2.8e-13 | 2.8e-13 | 0.61 |
| window | persistent | anchored K=64/float32 | 3.6e-05 | 3.6e-05 | 0.48 |
| window | persistent | anchored K=64/float64 | 1.4e-13 | 1.4e-13 | 0.69 |
| window | persistent | recurrent+rebase/float32 | 1.5e-05 | 9.8e-05 | – (sawtooth) |
| window | persistent | recurrent+rebase/float64 | 3.1e-14 | 2.0e-13 | – (sawtooth) |
| window | persistent | recurrent/float32 | 9.1e-04 | 9.1e-04 | 0.54 |
| window | persistent | recurrent/float64 | 1.6e-12 | 1.6e-12 | 0.51 |
The errors follow a simple model (DESIGN.md section 4). The recurrent form rotates every stored sample once per step, and each rotation is off by about one rounding unit. So on an unbounded stream the error grows linearly: 8.8e-03 in float32 after a million steps, slope 1 on the log-log plot. With a window, every sample is rotated at most W times and then subtracted. What is left over accumulates like a random walk, slope ½, reaching 9.1e-04. The anchored form never rotates a stored sample. Its relative error in float32 stays at or below 6.6e-05 over the million steps, though it still grows slowly (slopes up to ½, the rounding of the running sum), so longer streams would need a look. Pushing costs about the same as recurrent (next section), but reading the anchored state costs D complex exponentials, so it pays off when reads are rarer than pushes. The worst single frequency is worse than that average: up to 4.0e-04 of the typical per-frequency magnitude. float64 is fine with any of them: the worst relative error over all float64 runs is 1.9e-11. Rebase bounds the error, as a sawtooth that resets every 16 W steps, so only its maximum means anything.
One thing I got wrong first: anchored in float64 grew faster than expected, because the exact phase θt loses precision as t grows. Moving the time origin every 65,536 steps, one exact rotation of the state, fixed it.
Experiment 2: what does it cost?
The baseline re-encodes the whole window on every new sample, from a ring buffer and a precomputed position table: no trigonometry, one fused multiply-sum per half of the buffer. That is a cheap batch method, not the cheapest possible one. All timings are numpy on one thread, minimum over five repeats, on a machine shared with other jobs (load average 11.5 to 13.5 on 20 logical CPUs). The streaming push alone, whose code did not change, moved by up to 2.6× between my two runs (the first is in the git history), so read the table for orders of magnitude only.
| D | W | batch µs | recurrent µs | anchored µs | speed-up (recurrent) |
|---|---|---|---|---|---|
| 256 | 64 | 12.5 | 13.1 | 10.1 | 0.96× |
| 256 | 256 | 23.7 | 8.4 | 9.6 | 2.82× |
| 256 | 1024 | 115.4 | 9.3 | 10.2 | 12.45× |
| 256 | 4096 | 467.9 | 8.7 | 9.7 | 53.77× |
| 1024 | 64 | 47.7 | 14.9 | 16.5 | 3.21× |
| 1024 | 256 | 93.5 | 22.8 | 17.5 | 4.11× |
| 1024 | 1024 | 2773.1 | 34.2 | 40.2 | 81.19× |
| 1024 | 4096 | 3119.2 | 16.1 | 18.3 | 193.61× |
| 4096 | 64 | 103.1 | 44.4 | 47.9 | 2.32× |
| 4096 | 256 | 455.3 | 43.2 | 44.9 | 10.55× |
| 4096 | 1024 | 4763.1 | 44.9 | 48.6 | 106.16× |
| 4096 | 4096 | 34766.4 | 43.2 | 73.1 | 804.49× |
| 9996 | 64 | 448.2 | 140.0 | 142.5 | 3.20× |
| 9996 | 256 | 8165.4 | 133.6 | 151.8 | 61.13× |
| 9996 | 1024 | 23706.3 | 148.6 | 162.5 | 159.50× |
| 9996 | 4096 | 53594.9 | 150.8 | 167.5 | 355.34× |
At MiniROCKET’s size, D = 9,996, the update takes 134–151 µs per sample whatever the window, against 8.2 ms (W = 256) to 54 ms (W = 4096) for re-encoding: 61× to 355×. The gain is not universal:
- Short windows. At W = 64 it is 3.2× at D = 9,996. At D ≤ 1,024 and W ≤ 256 it ranges from 0.96× (slower than batch) to 4.1×. A push there costs about the same whatever D is, which is numpy call overhead, not arithmetic. A compiled loop should be much cheaper; I have not written one.
- Not every sample needs a prediction. The baseline re-encodes on every sample. If a prediction is needed only every k samples, batch pays once per k samples and the stream still pays every sample. At k = 32 and D = 9,996 the ratio is 1.9× at W = 256 and 11× at W = 4096; break-even is at k equal to the speed-up. The stream’s push is timed without reading the state. Reading the window-frame state is nearly free for recurrent and costs D complex exponentials for anchored (above); the absolute frame of Experiment 3 costs both of them that.
- Nothing here is specific to ⋄. The gain comes from bundling being a sum and the shift being exact. Any position code with an exact shift, complex FPE included, streams the same way.
For context, my numpy streaming MiniROCKET feature map (valid outputs only, not core’s padded features) costs 71 µs per sample at D = 9,996, the same order as the HDC update. So the per-sample pipeline is O(D). A flushed read adds up to O(D d_max) once per prediction, and that I have not timed.
Experiment 3: early classification
Here a test series arrives one sample at a time, and the classifier guesses after a fraction q of it. I used core’s 20-dataset UCR subset, one seed, kind B with s = 1 and s = 3 fixed, and core’s ridge classifier. MiniROCKET’s biases, the classifiers and the rescaling constants come from the training set only. Rescaled means each feature is multiplied by (number of outputs at full length) / (number so far), the analogue of PPV’s mean. One caveat on q: the experiment code gives a prefix at least 9 samples, so on the two datasets with T = 24 the columns q = 0.1–0.3 all see 37.5 % of the series, and on the two with T = 80 the column q = 0.1 sees 11 %.
The check. On two Adiac test series (s = 3), the flushed prefix vector at full length equals core’s batch HDC-MiniROCKET vector to 5.9e-08 relative, and the streaming encoder matches the causal prefix state to 2.4e-07. Prefixes shorter than the series are covered by the unit test.
One classifier for whole series fails on prefixes. That is the expected outcome, and it is why early classification usually trains one classifier per prefix length. With half of each test series seen, the full-series classifier gets 30.5% (HDC, s = 3, flushed and rescaled; 30.2% without rescaling), against 87.4% on full series. Plain MiniROCKET fails too, with no time encoding at all: 52.2% at half, from 86.9%. So most of the drop is that a prefix’s features do not look like a whole series’ features. HDC loses more on top of that, plausibly because the classifier weighs late positions that a prefix lacks. I have not tested that explanation. None of this is a streaming effect, since the flushed stream equals the batch method on the prefix.
Without the flush, the causal state loses even on full series (73.6% instead of 87.4%). The causal state uses only samples seen so far, so it leaves out each feature’s last 4d outputs. At the largest dilations that is about half of a padded feature’s outputs.
With one classifier per prefix length, the causal state is enough. Retrained on flushed prefixes, the classifiers reach 66.0% at q = 0.1 (64.4% without the four datasets where the 9-sample floor applies) and 83.5% at half. Accuracy reaches 95 % of its full-length value at a median fraction of 0.40 of the series. Retrained on the causal state, they differ from the flushed ones by -1.2 to +0.5 points across q. So an O(D)-per-sample stream can feed an early classifier without the flush. The flush is only needed to reuse a classifier trained on whole series.
The time encoding does not help here, on prefixes or on full series. Retrained HDC minus retrained plain MiniROCKET is between -0.8 and +1.1 points across q for s = 3 (-0.8 to +1.1 for s = 1), including 87.3% against 86.8% at q = 1. That is 20 datasets, one seed and a fixed s. It shows no visible effect on this subset and does not establish that there is none.
| mean accuracy (%) | q=0.1 | q=0.2 | q=0.3 | q=0.4 | q=0.5 | q=0.6 | q=0.7 | q=0.8 | q=0.9 | q=1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| one classifier, flushed (= batch on prefix), rescaled, HDC s=3 | 28.9 | 30.1 | 30.0 | 30.8 | 30.5 | 34.0 | 38.0 | 45.8 | 60.7 | 87.4 |
| one classifier, causal, rescaled, HDC s=3 | 28.7 | 29.6 | 30.5 | 31.0 | 29.9 | 33.3 | 35.9 | 43.9 | 57.7 | 73.6 |
| one classifier, MiniROCKET PPV | 34.0 | 35.0 | 40.2 | 43.4 | 52.2 | 55.0 | 59.2 | 61.7 | 73.6 | 86.9 |
| retrained per q, flushed, HDC s=3 | 66.0 | 73.9 | 78.4 | 81.5 | 83.5 | 84.5 | 85.1 | 86.6 | 86.9 | 87.3 |
| retrained per q, causal, HDC s=3 | 64.8 | 74.3 | 77.7 | 81.3 | 83.1 | 84.0 | 85.2 | 86.1 | 87.2 | 87.2 |
| retrained per q, MiniROCKET PPV | 64.9 | 74.6 | 78.7 | 81.8 | 83.0 | 83.9 | 84.9 | 86.7 | 87.2 | 86.8 |
What is new here, and what is not
Most of the parts are known:
- Sliding DFT. The window recursion is a sliding DFT at random frequencies, and its numerical trouble is textbook signal processing (Jacobsen and Lyons 2003; Duda 2010).
- Recurrent HDC encodings. With permutations, including the sliding-window update, they are old (Frady, Kleyko and Sommer 2018; Thomas, Dasgupta and Rosing 2021). They are exact in integers, but nearby positions are unrelated. Shift-equivariant, similarity-preserving sequence vectors by recursive binding are in Rachkovskij and Kleyko (2022).
- State-space models. The unbounded recursion is a diagonal linear state-space model with |λ| = 1 (S4D, LRU, RetNet without decay). Schlegel and coauthors point out this link themselves (quoted above).
- Early classification. One classifier per prefix length is the standard setup, and one MiniROCKET per 5 % prefix has been used (Renault et al., arXiv 2602.00918).
What I did not find elsewhere is the combination: the kind-B encoding streamed with MiniROCKET’s per-dilation delays; the observation that PPV and its HDC version stream exactly, with the padded edges as the only gap; the drift measurements; and the result that the causal streaming state feeds per-prefix classifiers about as well as the batch prefix. In particular I found no streaming or incremental MiniROCKET. That rests on web searches and abstracts, not a full reading of the literature, so it means “not found”, not “does not exist”.
Open questions
- A real stream. I planned a sliding window over the UCI smartphone recordings with postural transitions (HAPT; per its documentation 61 continuous recordings at 50 Hz with labelled segments; DESIGN.md 5.4) and have not run it. It needs a left-edge convention for the padded features in the window encoder, which does not exist yet. It would show how far the windowed stream is from batch on the window, and whether s > 0 helps over plain MiniROCKET in a stream.
- Does time encoding help early classification on all 128 UCR datasets, with several seeds and s chosen per dataset (Part 2)?
- Why does the full-series classifier fail more with HDC than with PPV? An ablation with shuffled positions would tell.
- What does the push cost compiled, without numpy overhead? And what does the read-time flush cost, implemented as a streaming operation?
- Is a stopping rule on the per-prefix classifiers (stop when confident) competitive with TEASER- or ECEC-style early classifiers?
Generated by Claude on Benjamin Schwerdtner’s behalf.
source: notebooks/streaming_hdc_minirocket.clj