Performance notes on the Faust NNX backend

This document records what makes a generated Faustax module fast or slow. The results describe the code generator (Faust’s -lang nnx), not the Faustax API. We record the results here because we measure them in Faustax.

All numbers on this page come from one machine: an Intel i9-12900KF (24 threads) and an NVIDIA RTX 4080 SUPER under WSL2. The software is jax 0.10.2 and a Faust 2.87.0 development build. The signal is 1 s of mono audio at 44.1 kHz (44,100 scan steps). Each time is the median of 5 runs after one compile/warmup call. Two scripts reproduce every measurement below:

• examples/benchmarks/scan_cost_probe.py — the delay-line layouts in a synthetic scan body.

• examples/benchmarks/codegen_probes.py — the table reads, the select casts, the control-rate hoisting, the scan iteration overhead, and the layout comparison on the shipped DSP files.

The processor and voice numbers come from bench.py and bench_voices.py; see Performance.

The delay-line layout in the scan carry decides the cost

A short delay line reaches the scan carry in one of three layouts: roll, IOTA, or scalars. Rows B, C, and D below run the same 10-section biquad cascade in each layout. Row A is a smaller body with only two delay lines:

scan body	ns/step
A: compressor-like, 2 arrays (len 2), 4 .at[].set per step	12
B: EQ-like, 10 arrays (len 3), roll writes: ~30 .at[].set per step	11,266
C: EQ-like, 10 arrays, IOTA writes: 10 .at[].set per step	21
D: EQ-like, state as 20 scalars, zero .at[].set	30

The roll layout writes the new sample at index 0 and then copies each element down one slot. This layout costs about 400 times more than the other two. The cost is not linear in the op count, and it does not change with the arithmetic in the body. The same 10-array roll body costs 11,697 ns/step with no extra arithmetic and 11,279 ns/step with 40 extra flops per section. The scalar body scales with the arithmetic instead: 31 ns/step and 292 ns/step for the same two cases. A fixed cost of this size is consistent with XLA CPU giving up the in-place update of the while-body above a small number of copy-down writes. The probe does not identify the exact trigger.

The compiler avoids the roll layout in two ways. The default build compiles a delay line of length 4 or less (kMaxSize in the analysis pass) to individual scalar carry entries with direct assignment (layout D). This output is equivalent to the fRec*_perm scalar temporaries of the C++ backend. The analysis pass is NNXScalarizeAnalysis in compiler/generator/nnx_base_instructions.hh, which both nnx/nnx_instructions.hh and linen/linen_instructions.hh include. The -mcd 0 flag selects the IOTA layout instead (layout C): one write for each array at a rotating index, and reads at the previous indices.

Both layouts avoid the pathology, and they differ in the forward and gradient directions:

processor	layout	carry entries	fwd b1	fwd b8	grad b8
ParametricEQ	scalars (default)	62 scalars	65.4 ms	65.3 ms	1,192 ms
ParametricEQ	arrays (-mcd 0)	3 scalars, 30 arrays	42.7 ms	47.7 ms	2,224 ms
Compressor	scalars (default)	4 scalars	0.6 ms	4.5 ms	92 ms
Compressor	arrays (-mcd 0)	3 scalars, 1 array	0.7 ms	2.6 ms	104 ms
Freeverb	scalars (default)	55 scalars, 12 arrays	136.6 ms	285.1 ms	—
Freeverb	arrays (-mcd 0)	15 scalars, 32 arrays	597.3 ms	601.0 ms	—

The outputs of the two layouts agree to float32 tolerance (max |diff| 1.9e-5 for the EQ, 2.4e-7 for Freeverb, and exactly 0 for the Compressor). Scalars win the gradient by 1.9x on the EQ and the forward pass by 4.4x on Freeverb. Freeverb keeps its large IOTA buffers as arrays under both settings; only its comb-damping one-pole states scalarize. The IOTA layout wins the EQ forward pass by 1.5x and the batch-8 Compressor forward pass by 1.7x. Keep the default scalar layout, because training uses the gradient and because Freeverb-style DSP loses the most from arrays.

What costs nothing

• Loop-invariant control-rate recomputation. The generated tick() recomputes every fSlow*/iSlow* control expression (including tan, sin, and pow) for each sample, and the params travel in the scan carry. A body that recomputes 84 control expressions per sample (the ParametricEQ count) costs 1.03x the hoisted version. A body that recomputes 317 of them (the DX7 count) costs 0.96x the hoisted version. Both outputs are bit-identical to the hoisted version. XLA’s while-loop invariant code motion already hoists values that go through the carry unchanged. The code generator does not need a separate control-rate function.

• The bool → int32 → != 0 cast sequence. 1,794 convert ops stay in the optimized HLO of the DX7. A select costs 1.99 ns with a direct predicate and 2.00 ns through the cast sequence.

• The lowering path of a one-element table read. t[idx], t.at[idx].get(mode="promise_in_bounds"), lax.dynamic_index_in_dim, and jnp.take all cost the same, within 3%.

• Params in the scan carry. A body that reads a 147-entry params dict (the DX7 count) from the carry costs the same as the body that reads the dict from a closure (0.96x to 1.03x across runs), and the output is bit-identical. The generated modules also carry an is_stochastic class flag and skip the per-sample RNG key split for deterministic DSPs. This flag exists for API cleanliness, not for speed, because the key array in the carry cost nothing either.

• Stateless DSP. The Gain module (no recursion) runs at thousands of times real time, because XLA vectorizes the scan.

What costs something

• Read-only tables in the scan carry. A one-element read costs 8.1 ns from a closed-over table and 24–28 ns from a table in the carry. The DX7 reads 86 elements per step from its 9 ROM tables, so the carry layout costs an estimated 1.5 µs/step, which is 6% of its step. The carry also holds each ROM table twice: the generated module keeps an iDx7Algo1SIG<n>Wave0 entry and an identical itbl<n>Dx7Algo1SIG<n> entry, which is 18 array entries for 9 tables. Suggested codegen change: emit read-only wavetables once, as module constants outside the scan carry.

• unroll on GPU. Unrolling divides the dominant per-iteration launch overhead. It is worth 4x on the batch-1 EQ forward pass and 5.4x on the batch-8 Compressor forward pass.

• unroll on CPU, for large bodies only. The 4,000-op DX7 body gets 1.81x at unroll 2 (1,047 → 577 ms). Small effect bodies lose: the batch-1 EQ forward pass costs 61.5 ms at unroll 1 and 90.8 ms at unroll 8, and the batch-1 Compressor costs 0.8 ms at unroll 1 and 4.4 ms at unroll 8. XLA compile time also increases steeply with body size × unroll, and it reaches minutes for the DX7 above unroll 2.

A large instrument: DX7 (2025 dx7.lib, dx.algorithm(1))

The generated 6-operator FM voice has a tick of 820 lines and 815 assignments. The op census per sample is 770 arithmetic ops, 359 bitwise ops, 1,033 comparisons, 650 jnp.where, ~935 integer casts, 231 rounds, 86 one-element table reads, and 7 sin. The voice has 147 UI params and 9 int ROM tables (sizes 4 to 100). All 133 filter, phase, and envelope states emit as scalars, and the tick contains zero .at[].set() writes. Of the 815 assignments, 317 are control-rate (185 fSlow and 132 iSlow).

backend	voices	unroll	wall ms	RT aggregate
CPU	1	1	1,047.3	1.0x
CPU	1	2	577.2	1.7x
CPU	16	1	1,014.3	15.8x
CPU	64	1	1,963.5	32.6x
GPU	1	1	4,185.2	0.2x
GPU	1	2	2,355.9	0.4x
GPU	1,024	2	1,275.5	802.8x

One voice costs 23.7 µs/step on CPU at unroll 1. Voice-level vmap is the primary method to get instrument throughput. On CPU, 16 voices cost the wall time of one voice, and a 16-voice polyphonic patch runs at 15.8x real time. On GPU, 1,024 voices cost less wall time than one voice at the same unroll setting. Performance holds the full voice tables.

The GPU pays a large per-iteration cost and no batch cost

A minimal scan body isolates the iteration overhead of the loop:

body	CPU µs/step	GPU µs/step
minimal, unroll 1	0.003	13.34
minimal, unroll 2	0.002	6.69
minimal, unroll 8	0.001	1.70
minimal, unroll 16	0.106	0.86
carry width 64, unroll 1	0.049	13.19
carry width 1,024, unroll 1	0.747	13.38
carry width 8,192, unroll 1	5.810	13.66

One GPU scan iteration costs about 13 µs, which is three to four orders of magnitude more than one CPU iteration. A 44,100-step scan therefore has a floor of ~0.6 s per call on GPU at unroll 1, whatever the body does. Unrolling divides this floor almost exactly. The GPU cost per step does not change from carry width 1 to width 8,192, while the CPU cost grows with the width. This is the mechanism behind the flat batch scaling of the GPU tables in Performance.

These measurements give this guidance: below ~64 parallel lanes (voices × batch items), run the scan on CPU. Above ~64 parallel lanes, the GPU is faster because of its flat scaling. Install the GPU wheels with uv sync --group gpu (the gpu dependency group pins jax[cuda13]).