Future work

This document lists proposed tasks that are scoped but not started. Each task description gives sufficient context so that a later session (or a contributor) can start the task without new research.

Emit ops.allpole / ops.state_space from the generated NNX tick

Status: proposed. The task is cross-repo (it changes the Faust compiler). Do a feasibility study and a prototype before you commit to the task.

Motivation

The generated NNX modules differentiate their per-sample scan with ordinary reverse-mode autodiff (BPTT). BPTT stores the scan carry at every step. Linear IIR recursions are the main component of EQs, reverbs, and dynamics. For these recursions, faustax.ops already implements a better rule. The VJP of an all-pole recursion is the same filter, run backwards, with time-shifted coefficients (ops.allpole, from torchlpc). The VJP of a general linear state space is its transposed adjoint, run backwards (ops.state_space). The proposal is that the compiler recognizes those recursions in the emitted tick and calls the primitive over the whole block. Then the generated gradients get the bounded residual. For all-pole shapes, the generated gradients also get the measured speed/memory improvement. No person must rewrite a DSP by hand.

Measured results that scope the task

examples/benchmarks/bench_state_space.py measured that the dense state_space adjoint has time parity with BPTT-through-scan on CPU. The cause is that XLA already differentiates a linear scan efficiently. Thus the benefit is not uniform:

• The benefit is large and proven for all-pole / one-pole ballistics shapes — the torchlpc/torchcomp rules. For example, BPTT through the generated Compressor scan costs ~19× more than the custom-VJP dynamics.compexp_gain chain at batch 8 (see Custom gradient primitives). Detect these shapes first.

• The benefit is small for generic small biquads. For these, the benefit is bounded memory and clean higher-order derivatives, not CPU time.

Thus, target the patterns with the large benefit first: a single linear recursion with one delay line (ops.allpole), and the switched attack/release one-pole (ops.ballistics / dynamics).

Outline of the work

1. Change the NNX backend codegen in the Faust repo (compiler/generator/nnx/nnx_instructions.hh, nnx_code_container.cpp, nnx_base_instructions.hh). Detect when an fRec update inside tick is an affine function of its own delayed outputs. Such an update is a linear all-pole section with one shared delay line. Do not emit the scalar recursion inside the scan. Instead, emit a call to allpole(x, a, zi) over the block. Gather a from the coefficients. The coefficients can be parameter-modulated and thus time-varying.

2. Emit the FIR read-out on the same delay line as an ops.fir. Then a full direct-form section is lfilter = fir ∘ allpole. ops.lfilter already uses this decomposition.

3. The generated modules currently embed the architecture file textually. The modules must import faustax.ops (a new runtime dependency for the emitted Python), or the modules must inline the primitive. Select one of the two options. Record the selection in architecture/jax/minimal.py.

Validation plan

• Exactness first: Regenerate a known IIR DSP (src/faustax/dsp/parametric_eq.dsp, a reverb, src/faustax/dsp/compressor.dsp). Diff the forward output against the output of the current scalar-scan codegen. The outputs must match to floating-point precision. The output must also pass the impulse-conformance tests against the C++ reference.

• Gradients: Compare the gradients against BPTT through the old codegen (the math is the same). Then benchmark the gradient time and the peak memory against the state dimension and the block length. Extend examples/benchmarks/bench_state_space.py for this benchmark.

Risks and open questions

• Pattern-matching robustness: Reject nonlinear couplings, delay lines that sections share, and fractional/variable delays. Handle coefficients that are time-varying because a slider modulates them (→ LPV allpole/state_space).

• State convention: The zi / warm-up of the primitive must reproduce the initial carry of the scan exactly. This includes one-sample-block mode.

• Determinism: Keep the float results sufficiently bit-stable for the conformance suite.

Related, smaller task

Before the compiler change, route one real generated module through the ops by hand. For example, compute the gain of the Compressor with dynamics.compexp_gain / ops.ballistics instead of its BPTT scan. Then benchmark the gradient time and the memory on that module. This test changes the abstract 19×/parity results into a concrete module-level result. This test also decreases the risk of the compiler pattern-matching above.

An optional frequency-sampled path for small-batch GPU gradients

Status: proposed, and deliberately not started. Read the “Reasons to not do this” section before you write code.

Motivation

Performance measures one gap that the architecture of Faustax causes. On GPU at batch 1, the dasp-pytorch EQ costs 7.4 ms forward and 23.5 ms for the gradient. The Faustax EQ costs 377.9 ms and 758.1 ms, which is approximately 32× more. The cause is the sequential scan, and NNX backend notes measures its per-iteration cost. The gap closes as the batch grows (5.5× at batch 256), and it reverses for the reverb at every batch size. Thus the gap matters only for small-batch EQ gradients on GPU.

dasp-pytorch closes this gap with the frequency sampling method. It builds the biquad coefficients elementwise, evaluates the transfer function of the cascade on an FFT grid, and multiplies in the frequency domain. The whole idea is approximately 200 lines of dasp_pytorch/signal.py: biquad, fft_sosfreqz, and sosfilt_via_fsm. The method needs no recurrence, so it needs no custom VJP: every operation is already a differentiable primitive.

Outline of the work

Add one function, not a second processor hierarchy. faustax.freqsampled.parametric_eq(x, sample_rate, **params) is approximately 150 lines and closes the measured gap. src/faustax/reverb.py is the pattern to follow: it already ports dasp_pytorch.functional.noise_shaped_reverberation, its 12-band octave filterbank, and its FFT convolution. The translation from torch to JAX is mechanical, with four known differences:

1. The masked in-place writes of the compressor (x_sc[idx] = ...) become jnp.where.

2. n_fft = 2 ** ceil(log2(...)) must be a Python int, because shapes are static under jit.

3. The preallocated sos[:, i, :] = ... writes become a list and one jnp.stack.

4. The unseeded torch.randn of dasp-pytorch becomes a keyed RNG, as in reverb.py.

Validation plan

examples/benchmarks/bench_vs_dasp.py already renders both libraries with the same physical parameter values, and dasp-pytorch is already a dev dependency. Thus an equivalence test is a small addition to an existing harness: assert that the new function matches dasp_pytorch.functional.parametric_eq to float32 tolerance. Then measure the same function against the exact recurrence, and record how the approximation error changes with the parameter values.

Reasons to not do this

• Two implementations of the same effect have different numerics. The docs must then answer “which EQ do I mean” on every page.

• A frequency-sampled effect does not stream. It is a block circular convolution with no carry, so it cannot hold the streaming parity that Real-time deployment sells, and it cannot take a per-sample time-varying parameter.

• The parameter metadata needs a second mechanism. Processor introspects the ranges from the Faust slider declarations. A function with no Faust source has no declarations, so its ranges become hand-maintained, as dasp_pytorch/modules.py maintains them.

• Matching the dasp-pytorch compressor means reproducing its bugs. Its smoother ignores release_ms. The parity table of the README calls the application of release_ms an intentional upgrade.

An alternative that duplicates no DSP definition

Any linear time-invariant module has an impulse response. Render the response once by scanning an impulse, then apply it to the whole batch by FFT convolution. This method needs no second definition of any effect, and it works for any LTI Faust module. The arithmetic is favorable: the GPU costs 1.70 µs per scan iteration at unroll 8, so an 8,192-tap response costs approximately 14 ms, against the 377.9 ms that the batch-1 EQ costs now.

The open question is the length of the response, and it decides whether the method works. The length is a property of the parameter values, not a constant. A low shelf at 20 Hz with a Q of 6 rings for tens of thousands of samples, so a truncation at 8,192 taps is an approximation with an error that the slider values control. Measure that error across the slider ranges before you build anything on this idea.

Compile arbitrary Faust in-process with DawDreamer

Status: blocked on an upstream release.

Motivation

faustax.compile already has the provider. _compile_dawdreamer calls boxFromDSP and boxToSource through the libfaust that DawDreamer bundles, and provider="auto" selects it when no faust binary resolves. The provider raises today, because the released DawDreamer wheels predate the NNX backend. DawDreamer 0.9.0 does not ship the backend. A 0.9.1 release built against a Faust that has the backend would make the provider work.

The result is that a user could compile arbitrary Faust with pip install, and with no Faust build from master-dev. This changes a contributor-only capability into a user-facing one. The widget modulation recipe needs exactly this capability: a slider becomes an input channel only when you author and compile your own DSP. examples/parameter_estimation/automation.py --compile demonstrates the loop today and needs FAUST_BIN to run. The learnable-soundfile and learnable-menu workflows have the same constraint.

Outline of the work

1. Add a compile extra that pins dawdreamer>=0.9.1. Do not make DawDreamer a base dependency: the base install advertises five runtime dependencies, and DawDreamer is a large binary wheel that a user of the shipped effects never loads.

2. Resolve the NNX architecture file. _compile_dawdreamer needs an explicit arch_file= today, and a user with no Faust checkout has no such file.

3. Enable the compile tests that skip without a compiler, and run them against the DawDreamer provider in CI.

4. Update the installation sections and Development to describe the two providers.

Open questions

• Where does the architecture file come from? Either DawDreamer ships architecture/jax/minimal.py next to its libfaust, or Faustax vendors a copy. A vendored copy must track the backend, and a stale copy generates modules that do not match the checked-in ones. Check what the wheel contains before you select an option.

• Where do the Faust libraries come from? boxFromDSP receives the -I directories. The CLI provider resolves stdfaust.lib from the Faust checkout. Confirm that the DawDreamer wheel bundles the libraries, and confirm which version they are.

• Which platforms have wheels? The base install supports Python 3.11 through 3.14 on every platform that JAX supports. The compile extra covers only the platforms that DawDreamer builds for.

• Which provider does auto prefer? The CLI provider is first today. A contributor with a Faust checkout and the extra installed gets the checkout, which is the correct default for regeneration work.