Audio Graph Architecture

Last updated: May 2026 — Gain/pan/spatial removed from Processor trait; now handled by dedicated Mixer and Spatial plugin nodes. This document is the canonical reference for any AI agent or developer working on the audio graph. Read it before making any changes.

1. Overview

The audio graph is a directed acyclic graph (DAG) of processing nodes. Each node holds exactly one Box<dyn Processor> that handles all DSP. The graph handles topology, buffer management, connection routing, topological sort, and optional parallel execution via Rayon.

Key Invariants

All DSP state (mute, plugins) is accessed through the Processor trait. AudioNode is a thin metadata wrapper — it does NOT have public gain, pan, muted, or plugins fields.
Gain, pan, and spatial positioning are NOT on the Processor trait. They are handled by dedicated plugin nodes:
- Mixer plugin (songbird-plugins): gain + equal-power pan
- SpatialPanner plugin (songbird-plugins): azimuth/elevation/distance for HRTF
The signal chain within a node is: instrument → effects → strip → mixer The Mixer is always the last plugin in the chain.

2. File Structure

File	Contents
`mod.rs`	`AudioGraph` — topology, scratch pool, connection routing, topo sort, parallel processing
`node.rs`	`AudioNode` — thin wrapper: id + name + `Box<dyn Processor>` + solo + metadata
`processor.rs`	`Processor` trait — the single processing abstraction
`chain_processor.rs`	`PluginChainProcessor` — chain of plugins (default, “track” mode)
`single_processor.rs`	`SingleProcessor` — single plugin per node (modular mode)
`pdc.rs`	Plugin Delay Compensation
`signal_router.rs`	Signal routing matrix (connections, gains, buses, presets)
`thread_pool.rs`	Rayon-based parallel processing for independent levels
`track_group.rs`	Track grouping and folder tracks

3. Architecture Intent & Design Decisions

3.1 Why a Processor Trait?

The Processor trait decouples the processing model from the graph:

PluginChainProcessor — traditional DAW track: runs a chain of plugins in series. Muted state is stored on the processor. Gain and pan are handled by a Mixer plugin at the end of the chain. This is the default — every graph.add_node("name") creates one.
SingleProcessor — modular mode: wraps exactly one plugin per node. For gain control, add a separate Mixer plugin node. Enables modular routing: split signals, parallel FX chains, per-plugin metering.

3.2 Mixer & Spatial Plugins (Modular DAW Pattern)

Following Bitwig and modular DAW conventions, mixer controls are dedicated nodes in the signal graph rather than baked into every processor:

Mixer (songbirdMixer): params[0] = gain (linear), params[1] = pan (-1..1). Uses equal-power pan law: cos/sin(pan01 * π/2).
SpatialPanner (songbirdSpatialPanner): params[0] = azimuth, params[1] = elevation, params[2] = distance. Metadata-only for HRTF.

Runtime commands (SetTrackGain, SetTrackPan, SetNodeGain, etc.) find the Mixer/Spatial plugin by type_name() and update its params directly.

3.3 Performance Characteristics

One dyn Processor dispatch per node per block adds ~1ns (a single vtable lookup). The existing code already does N vtable dispatches per node (one per plugin.process() call), so this is negligible.

3.4 Why Solo Stays on AudioNode

Solo is a graph-level concern: when any node is solo’d, the graph skips processing all non-solo’d nodes entirely (doesn’t even call process()). It can’t live in the processor because the graph needs to read it before deciding whether to call process().

3.5 `plugins_mut()` vs `plugins_vec_mut()`

plugins_mut() -> &mut [Box<dyn Plugin>] — Returns a mutable slice. Use for iterating, indexing, get_mut(). Cannot call push(), remove(), insert(), or clear() because slices don’t own the data.
plugins_vec_mut() -> Option<&mut Vec<Box<dyn Plugin>>> — Returns the underlying Vec. Use when you need push(), remove(), insert(), clear(), or mem::swap(). Returns None for processors that don’t use a dynamic Vec.
push_plugin(plugin) — Convenience for the common case. Panics on SingleProcessor (which holds exactly one plugin).

4. Processing Pipeline

Each audio callback:

1. Check session hot-swap
2. Drain commands (Play, Stop, Seek, SetGain, etc.)
3. Advance transport
4. Schedule clips (ClipScheduler → ScheduleOutput)
5. Process graph:
   a. Inject audio clips into FX node scratch buffers
   b. Collect MIDI events per node
   c. For each topological level (parallel via rayon):
      - Zero scratch buffer
      - Sum inputs from connected sources
      - Call processor.process(buffer, midi_events)
      - Copy to output scratch (master node is skipped here —
        it runs out-of-band in step 7)
6. Sum terminal nodes — except the master fader — to output
   L/R (HRTF / ILD / plain sum depending on spatial mode)
7. Master fader: capture the just-summed L/R into the master
   node's scratch, downmix if mono, run its plugin chain
   (`process_master_chain`), then sum master scratch back to
   the device output. Skipped when no master slot is registered.
8. Emit metering events

5. Processor Model

                    ┌─────────────────┐
                    │   Processor     │ (trait, processor.rs)
                    │─────────────────│
                    │ process()       │
                    │ muted()/set_..  │  ← processor zeros buffer internally
                    │ plugins()       │  ← immutable slice access
                    │ plugins_mut()   │  ← mutable slice access
                    │ plugins_vec_mut │  ← Vec access for push/remove/swap
                    │ push_plugin()   │
                    │ plugin_count()  │
                    │ is_instrument() │
                    │ latency_samples │
                    │ has_tail()      │
                    └────────┬────────┘
                             │
              ┌──────────────┴──────────────┐
              │                             │
   ┌──────────┴──────────┐     ┌────────────┴──────────┐
   │ PluginChainProcessor│     │   SingleProcessor     │
   │  (chain_processor)  │     │ (single_processor.rs) │
   │─────────────────────│     │───────────────────────│
   │ plugins: Vec<Plugin>│     │ plugins: Vec<Plugin>  │ (always len=1)
   │ muted: bool         │     │ muted: bool           │
   └─────────────────────┘     └───────────────────────┘
   Default "track" mode        Modular mode

   Plugin chain order: [Instrument, FX..., Strip, Mixer]
                        ↑ generates audio    ↑ gain+pan (last)

Node vs. Processor Responsibilities

Concern	Where	Why
Processing (DSP)	Processor	Different processor types handle DSP differently
Gain / Pan	Mixer plugin (last in chain)	Modular: dedicated node, equal-power pan law
Spatial	SpatialPanner plugin	Metadata for HRTF renderer
Mute	Processor	Processor zeros its own buffer
Solo	AudioNode	Graph-level: must be checked before calling process()
Plugin mgmt	Processor	Encapsulates plugin chain or single plugin
Topology	AudioGraph	Connections, topo sort, scratch pool
Parallel exec	AudioGraph	Rayon per-level dispatch

6. Gain/Pan Access Patterns

Reading gain/pan from a node

let mixer = node.processor.plugins()
    .iter()
    .find(|p| p.type_name() == "songbirdMixer");
if let Some(m) = mixer {
    let gain = m.params()[0].value;
    let pan = m.params()[1].value;
}

Setting gain/pan on a node

for plugin in node.processor.plugins_mut() {
    if plugin.type_name() == "songbirdMixer" {
        plugin.params_mut()[0].set(0.5);   // gain
        plugin.params_mut()[1].set(-0.3);  // pan
        break;
    }
}

Adding a Mixer to a new node

// The mixer's apply_to_graph() pushes a Mixer plugin automatically.
// For manual node setup:
node.processor.push_plugin(
    Box::new(songbird_plugins::Mixer::with_values(0.8, 0.0)),
);

7. Cross-Crate Access Patterns

The Processor trait is used by multiple crates outside the engine. Here’s how each crate accesses node state:

Crate	File	Access Pattern
`songbird-engine`	`graph/mod.rs`	`node.processor.process()`
`songbird-engine`	`audio_io.rs`, `callback_state.rs`	Find Mixer plugin by type_name, update params
`songbird-engine`	`debug/snapshot.rs`	Read gain/pan from Mixer plugin params
`songbird-engine`	`mixer/mod.rs`	`apply_to_graph()` pushes Mixer; `sync_params_to_graph()` updates Mixer params
`songbird-sync`	`graph_sync.rs`	`build_graph_from_project()` pushes Mixer + Spatial plugins
`songbird-wasm`	`lib.rs`	Find Mixer plugin by type_name, read/write params
`songbird-export`	`bounce.rs`, `lib.rs`	Read/write Mixer plugin mute param, `node.processor.plugins()`

What NOT to do

// ❌ WRONG — gain/pan are NOT on the processor
node.processor.set_gain(0.5);
node.processor.set_pan(-1.0);
node.processor.gain();

// ✅ CORRECT — find the Mixer plugin and update its params
for plugin in node.processor.plugins_mut() {
    if plugin.type_name() == "songbirdMixer" {
        plugin.params_mut()[0].set(0.5);  // gain
        plugin.params_mut()[1].set(-1.0); // pan
        break;
    }
}

// ❌ WRONG — mute is NOT on the processor
node.processor.set_muted(true);

// ✅ CORRECT — mute is on the Mixer plugin (param index 3)
for plugin in node.processor.plugins_mut() {
    if plugin.type_name() == "songbirdMixer" {
        plugin.params_mut()[3].set(1.0);  // 1.0 = muted, 0.0 = unmuted
        break;
    }
}

// ✅ For Vec operations (push, remove, insert, clear):
node.processor.plugins_vec_mut().unwrap().clear();
node.processor.plugins_vec_mut().unwrap().insert(0, plugin);

Watch out: Not everything has a `.processor`

Several types have their own plugins, gain, pan, muted fields that are not AudioNode. Do NOT change these to use processor.xxx():

Type	Location	Has own `plugins`/`gain`
`TrackTemplate`	`project/track_template.rs`	`self.plugins: Vec<TemplatePlugin>`
`ChainPreset`	`graph/signal_router.rs`	`self.plugins: Vec<PluginChainEntry>`
`songbird_state::Track`	`songbird-state`	`track.plugins: Vec<PluginInstance>`, `track.gain: f64`
`ClipState`	`songbird-state`	`clip.gain: f64`, `clip.muted: bool`
`ScheduledMidiClip`	`clip_scheduler`	`clip.gain: f64`, `clip.muted: bool`
`ClipEffectChain`	`scheduling/clip_effects`	`chain.muted: bool`, `chain.pan: f32`
`Connection`	`graph/mod.rs`	`conn.gain: f64`
`MixerTrackResponse`	tests	`t.gain`, `t.pan`, `t.muted`

If you see data.tracks[0].plugins or track.gain, those are project data — they’re fine as-is.

8. Migration Status

✅ COMPLETE — Plugin-Based Gain/Pan/Spatial (May 2026)

Gain, pan, and spatial positioning have been removed from the Processor trait and PluginChainProcessor. They are now handled by dedicated plugin nodes following the Bitwig/modular DAW pattern. What was changed:

Area	Details
`Processor` trait	Removed `gain()`, `set_gain()`, `pan()`, `set_pan()`, `spatial_position()`, `set_spatial()`, `muted()`, `set_muted()`
`PluginChainProcessor`	Removed `gain`, `pan`, `spatial_*` fields
`graph/mod.rs`	Removed gain/pan multiply loops from `process_single_node`, `process_level_parallel`, `process_single_node_with_midi`
`Mixer` plugin	Gain + equal-power pan; renamed from Fader; added `with_values(gain, pan)` constructor
`SpatialPanner` plugin	New plugin for HRTF metadata (azimuth, elevation, distance)
`audio_io.rs`	`SetNodeGain`/`SetNodePan` now target Mixer plugin params
`callback_state.rs`	`SetTrackGain`/`SetTrackPan`/`SetTrackSpatial` now target plugin params
`mixer/mod.rs`	`apply_to_graph()` pushes Mixer; `sync_params_to_graph()` updates Mixer params
`graph_sync.rs`	`build_graph_from_project()` pushes Mixer + Spatial plugins
`project_loader.rs`	Inserts instrument/FX/strip plugins before Mixer (correct chain order)
`songbird-wasm`	Updated to read/write Mixer plugin params
All test files	Updated across engine, sync, export (3468+ tests pass)

⏳ PENDING — Next Steps

Task	Priority	Details
Performance profiling	Medium	Profile Mixer plugin overhead vs old baked-in gain/pan. Expected: negligible.
Modular graph usage	Low	`SingleProcessor` is implemented but not yet used by `build_graph_from_project()`.
PDC integration	Low	`pdc.rs` may need updates if processor types report different latencies.

9. Testing the Graph

Quick validation

cd rust && cargo check -p songbird-engine --tests

Full test suite

cd rust && cargo test -p songbird-engine

Cross-crate (recommended for graph changes)

cd rust && cargo check --workspace --tests
cd rust && cargo test --workspace

Headless server (integration testing)

./utils/agent-headless.sh
# Then open: http://localhost:8080?ws=localhost:9090

10. Adding a New Processor Type

Create a new file in graph/, e.g., my_processor.rs
Implement the Processor trait (see chain_processor.rs as template)
Add pub mod my_processor; to graph/mod.rs
Use it: AudioNode::with_processor(id, "name", Box::new(MyProcessor::new()))
Update build_graph_from_project() in songbird-sync/graph_sync.rs if this processor type should be created from project data

Trait methods to implement

Method	Required?	Default
`process()`	Yes	—
`processor_name()`	Yes	—
`processor_type()`	Yes	—
~~`muted()` / `set_muted()`~~	Removed	Now on Mixer plugin param[3]
`plugins()` / `plugins_mut()`	No	empty slice
`plugins_vec_mut()`	No	None
`push_plugin()`	No	panics
`is_instrument()`	No	false
`latency_samples()`	No	0
`has_tail()`	No	false
`plugin_count()`	No	`plugins().len()`

Architecture

Core (zero-dep foundations)

Engine & DSP

Data & Sync

Machine Learning

Integration

Orchestration

Application Shells

Audio Graph Architecture

Audio Graph Architecture

1. Overview

Key Invariants

2. File Structure

3. Architecture Intent & Design Decisions

3.1 Why a Processor Trait?

3.2 Mixer & Spatial Plugins (Modular DAW Pattern)

3.3 Performance Characteristics

3.4 Why Solo Stays on AudioNode

3.5 `plugins_mut()` vs `plugins_vec_mut()`

4. Processing Pipeline

5. Processor Model

Node vs. Processor Responsibilities

6. Gain/Pan Access Patterns

Reading gain/pan from a node

Setting gain/pan on a node

Adding a Mixer to a new node

7. Cross-Crate Access Patterns

What NOT to do

Watch out: Not everything has a `.processor`

8. Migration Status

✅ COMPLETE — Plugin-Based Gain/Pan/Spatial (May 2026)

⏳ PENDING — Next Steps

9. Testing the Graph

Quick validation

Full test suite

Cross-crate (recommended for graph changes)

Headless server (integration testing)

10. Adding a New Processor Type

Trait methods to implement

​Audio Graph Architecture

​1. Overview

​Key Invariants

​2. File Structure

​3. Architecture Intent & Design Decisions

​3.1 Why a Processor Trait?

​3.2 Mixer & Spatial Plugins (Modular DAW Pattern)

​3.3 Performance Characteristics

​3.4 Why Solo Stays on AudioNode

​3.5 plugins_mut() vs plugins_vec_mut()

​4. Processing Pipeline

​5. Processor Model

​Node vs. Processor Responsibilities

​6. Gain/Pan Access Patterns

​Reading gain/pan from a node

​Setting gain/pan on a node

​Adding a Mixer to a new node

​7. Cross-Crate Access Patterns

​What NOT to do

​Watch out: Not everything has a .processor

​8. Migration Status

​✅ COMPLETE — Plugin-Based Gain/Pan/Spatial (May 2026)

​⏳ PENDING — Next Steps

​9. Testing the Graph

​Quick validation

​Full test suite

​Cross-crate (recommended for graph changes)

​Headless server (integration testing)

​10. Adding a New Processor Type

​Trait methods to implement

Audio Graph Architecture

1. Overview

Key Invariants

2. File Structure

3. Architecture Intent & Design Decisions

3.1 Why a Processor Trait?

3.2 Mixer & Spatial Plugins (Modular DAW Pattern)

3.3 Performance Characteristics

3.4 Why Solo Stays on AudioNode

3.5 `plugins_mut()` vs `plugins_vec_mut()`

4. Processing Pipeline

5. Processor Model

Node vs. Processor Responsibilities

6. Gain/Pan Access Patterns

Reading gain/pan from a node

Setting gain/pan on a node

Adding a Mixer to a new node

7. Cross-Crate Access Patterns

What NOT to do

Watch out: Not everything has a `.processor`

8. Migration Status

✅ COMPLETE — Plugin-Based Gain/Pan/Spatial (May 2026)

⏳ PENDING — Next Steps

9. Testing the Graph

Quick validation

Full test suite

Cross-crate (recommended for graph changes)

Headless server (integration testing)

10. Adding a New Processor Type

Trait methods to implement