Learnable soundfiles and menus

The NNX backend converts two Faust constructs into trainable parameters. The first construct is a soundfile with the [param:1] tag. The second construct is an nentry menu, which the backend relaxes to a Gumbel-softmax over its discrete grid.

Learnable soundfiles ([param:1])

A Faust soundfile usually contains read-only sample data that loads at load time. Examples are a wavetable, an impulse response, and a bank of parameter frames. Add the [param:1] tag to the soundfile label. The NNX backend then instantiates the buffer as an nnx.Param. The contents of the buffer are differentiable. Gradient descent optimizes the buffer contents together with the slider values:

// read-only (default): fixed sample data
osc = soundfile("wt[url:{'wavetable.wav'}]", 1);

// learnable: the buffer becomes an nnx.Param, trained end-to-end
osc = soundfile("wt[url:{'wavetable.wav'}][param:1]", 1);

The initial contents load from the WAV file. This is the warm start. Training then moves the samples to minimize the loss. The gradient reaches each sample of each channel. Thus a table-driven Faust program becomes a model that fits its data, not only its knobs. For example, a wavetable oscillator can learn its waveform, and a convolution can learn its impulse response. A multi-channel table can hold structured per-voice data (frequencies, decay rates, levels). Training optimizes this data together with the rest of the graph. The [param:0] tag is the default and keeps a soundfile fixed.

Learnable buffers are standard nnx.Param leaves under the soundfile zone. Thus they are compatible with all the features above. Use nnx.split(module, nnx.Param, ...) (or an explicit nnx.Param filter) to keep the integer length/offset metadata out of the gradient. save_params serializes the learnable buffers. nnx.grad updates the learnable buffers in place:

module = MyTableDSP(sample_rate=48000, soundfile_dirs=["assets"])  # warm start from WAV
graphdef, params, rest = nnx.split(module, nnx.Param, ...)         # params includes the buffer

def loss_fn(params):
    m = nnx.merge(graphdef, params, rest)
    y = m(num_samples=N)          # 0-input generators: render N samples
    return spectral_loss(y, target)

grads = jax.grad(loss_fn)(params)  # grads["<soundfile_zone>"]["fBuffers"] is the table gradient

Learnable menus (nentry as Gumbel-softmax)

A Faust nentry is a categorical control. Examples are a preset selector, a waveform switch, and a filter-type menu. The NNX backend makes the nentry differentiable with a Gumbel-softmax over its discrete grid. Thus gradient descent can select an entry instead of a continuous knob value:

// grid [0, 6, 12, 18] dB from (min, max, step) = (0, 18, 6); the menu index
// is learnable, and [tau:learnable] trains the softmax temperature too.
drive_db = nentry("drive[style:menu{'0 dB':0;'6 dB':6;'12 dB':12;'18 dB':18}][tau:learnable]",
                  0, 0, 18, 6);

Each menu adds two learnable parts to the module. The first part is <zone>_logits, which has one logit for each entry. The argmax of the logits gives the selection. The second part is <zone>_tau, which is the temperature. In training (deterministic=False, with an nentry RNG stream present), the value is a soft sample dot(softmax((logits + gumbel) / tau), grid). The soft sample is a mixture of grid values and is differentiable in the logits and in tau. At eval (deterministic=True), the value becomes the hard grid[argmax(logits)]. The two training conditions are necessary for gradients to flow. If one condition is missing, the backend silently uses the non-differentiable hard path:

model = ModeDrive(sample_rate=44100, rngs=nnx.Rngs(0, nentry=7))  # training: samples
loss, grads = nnx.value_and_grad(loss_fn)(model)                  # reaches *_logits, *_tau

You can also control the selection from outside the module through normalized_params={"drive_logits": ..., "drive_tau": ...} (the RL / external-controller path). Pass a gumbel_key to unnormalize_params to get soft sampling. A soft sample is a convex mixture of grid values. Thus an interior target is ambiguous: 12 dB is also equal to “half of 6, half of 18”. One solution is to recover an extreme entry (examples/learnable_structure/learn_mode_select.py). A different solution is to anneal tau toward one-hot (examples/learnable_structure/learn_mode_select_interior.py). Then the low-temperature samples punish the mixture, and the logits commit to the interior entry.