> ## Documentation Index > Fetch the complete documentation index at: https://songbird.studiocollective.xyz/llms.txt > Use this file to discover all available pages before exploring further. # Audio Graph Architecture # Audio Graph Architecture > **Last updated**: May 2026 — Gain/pan/spatial removed from Processor trait; > now handled by dedicated Mixer and Spatial plugin nodes. > > This document is the canonical reference for any AI agent or developer > working on the audio graph. Read it before making any changes. *** ## 1. Overview The audio graph is a **directed acyclic graph (DAG)** of processing nodes. Each node holds exactly one **`Box`** that handles all DSP. The graph handles topology, buffer management, connection routing, topological sort, and optional parallel execution via Rayon. ### Key Invariants 1. **All DSP state (mute, plugins) is accessed through the `Processor` trait.** `AudioNode` is a thin metadata wrapper — it does NOT have public `gain`, `pan`, `muted`, or `plugins` fields. 2. **Gain, pan, and spatial positioning are NOT on the Processor trait.** They are handled by dedicated plugin nodes: * **`Mixer`** plugin (`songbird-plugins`): gain + equal-power pan * **`SpatialPanner`** plugin (`songbird-plugins`): azimuth/elevation/distance for HRTF 3. The signal chain within a node is: `instrument → effects → strip → mixer` The Mixer is always the **last** plugin in the chain. *** ## 2. File Structure | File | Contents | | --------------------- | ----------------------------------------------------------------------------------------- | | `mod.rs` | `AudioGraph` — topology, scratch pool, connection routing, topo sort, parallel processing | | `node.rs` | `AudioNode` — thin wrapper: id + name + `Box` + solo + metadata | | `processor.rs` | `Processor` trait — the single processing abstraction | | `chain_processor.rs` | `PluginChainProcessor` — chain of plugins (default, "track" mode) | | `single_processor.rs` | `SingleProcessor` — single plugin per node (modular mode) | | `pdc.rs` | Plugin Delay Compensation | | `signal_router.rs` | Signal routing matrix (connections, gains, buses, presets) | | `thread_pool.rs` | Rayon-based parallel processing for independent levels | | `track_group.rs` | Track grouping and folder tracks | *** ## 3. Architecture Intent & Design Decisions ### 3.1 Why a Processor Trait? The `Processor` trait decouples the processing model from the graph: * **`PluginChainProcessor`** — traditional DAW track: runs a chain of plugins in series. Muted state is stored on the processor. Gain and pan are handled by a **Mixer plugin** at the end of the chain. **This is the default — every `graph.add_node("name")` creates one.** * **`SingleProcessor`** — modular mode: wraps exactly one plugin per node. For gain control, add a separate Mixer plugin node. Enables modular routing: split signals, parallel FX chains, per-plugin metering. ### 3.2 Mixer & Spatial Plugins (Modular DAW Pattern) Following Bitwig and modular DAW conventions, mixer controls are dedicated nodes in the signal graph rather than baked into every processor: * **Mixer** (`songbirdMixer`): params\[0] = gain (linear), params\[1] = pan (-1..1). Uses equal-power pan law: `cos/sin(pan01 * π/2)`. * **SpatialPanner** (`songbirdSpatialPanner`): params\[0] = azimuth, params\[1] = elevation, params\[2] = distance. Metadata-only for HRTF. Runtime commands (`SetTrackGain`, `SetTrackPan`, `SetNodeGain`, etc.) find the Mixer/Spatial plugin by `type_name()` and update its params directly. ### 3.3 Performance Characteristics One `dyn Processor` dispatch per node per block adds **\~1ns** (a single vtable lookup). The existing code already does N vtable dispatches per node (one per `plugin.process()` call), so this is negligible. ### 3.4 Why Solo Stays on AudioNode Solo is a **graph-level** concern: when any node is solo'd, the graph skips processing all non-solo'd nodes entirely (doesn't even call `process()`). It can't live in the processor because the graph needs to read it *before* deciding whether to call `process()`. ### 3.5 `plugins_mut()` vs `plugins_vec_mut()` * `plugins_mut() -> &mut [Box]` — Returns a mutable **slice**. Use for iterating, indexing, `get_mut()`. Cannot call `push()`, `remove()`, `insert()`, or `clear()` because slices don't own the data. * `plugins_vec_mut() -> Option<&mut Vec>>` — Returns the underlying `Vec`. Use when you need `push()`, `remove()`, `insert()`, `clear()`, or `mem::swap()`. Returns `None` for processors that don't use a dynamic Vec. * `push_plugin(plugin)` — Convenience for the common case. Panics on `SingleProcessor` (which holds exactly one plugin). *** ## 4. Processing Pipeline Each audio callback: ```text theme={null} 1. Check session hot-swap 2. Drain commands (Play, Stop, Seek, SetGain, etc.) 3. Advance transport 4. Schedule clips (ClipScheduler → ScheduleOutput) 5. Process graph: a. Inject audio clips into FX node scratch buffers b. Collect MIDI events per node c. For each topological level (parallel via rayon): - Zero scratch buffer - Sum inputs from connected sources - Call processor.process(buffer, midi_events) - Copy to output scratch (master node is skipped here — it runs out-of-band in step 7) 6. Sum terminal nodes — except the master fader — to output L/R (HRTF / ILD / plain sum depending on spatial mode) 7. Master fader: capture the just-summed L/R into the master node's scratch, downmix if mono, run its plugin chain (`process_master_chain`), then sum master scratch back to the device output. Skipped when no master slot is registered. 8. Emit metering events ``` *** ## 5. Processor Model ```text theme={null} ┌─────────────────┐ │ Processor │ (trait, processor.rs) │─────────────────│ │ process() │ │ muted()/set_.. │ ← processor zeros buffer internally │ plugins() │ ← immutable slice access │ plugins_mut() │ ← mutable slice access │ plugins_vec_mut │ ← Vec access for push/remove/swap │ push_plugin() │ │ plugin_count() │ │ is_instrument() │ │ latency_samples │ │ has_tail() │ └────────┬────────┘ │ ┌──────────────┴──────────────┐ │ │ ┌──────────┴──────────┐ ┌────────────┴──────────┐ │ PluginChainProcessor│ │ SingleProcessor │ │ (chain_processor) │ │ (single_processor.rs) │ │─────────────────────│ │───────────────────────│ │ plugins: Vec│ │ plugins: Vec │ (always len=1) │ muted: bool │ │ muted: bool │ └─────────────────────┘ └───────────────────────┘ Default "track" mode Modular mode Plugin chain order: [Instrument, FX..., Strip, Mixer] ↑ generates audio ↑ gain+pan (last) ``` ### Node vs. Processor Responsibilities | Concern | Where | Why | | ---------------- | ---------------------------- | ------------------------------------------------------- | | Processing (DSP) | Processor | Different processor types handle DSP differently | | Gain / Pan | Mixer plugin (last in chain) | Modular: dedicated node, equal-power pan law | | Spatial | SpatialPanner plugin | Metadata for HRTF renderer | | Mute | Processor | Processor zeros its own buffer | | Solo | AudioNode | Graph-level: must be checked *before* calling process() | | Plugin mgmt | Processor | Encapsulates plugin chain or single plugin | | Topology | AudioGraph | Connections, topo sort, scratch pool | | Parallel exec | AudioGraph | Rayon per-level dispatch | *** ## 6. Gain/Pan Access Patterns ### Reading gain/pan from a node ```rust theme={null} let mixer = node.processor.plugins() .iter() .find(|p| p.type_name() == "songbirdMixer"); if let Some(m) = mixer { let gain = m.params()[0].value; let pan = m.params()[1].value; } ``` ### Setting gain/pan on a node ```rust theme={null} for plugin in node.processor.plugins_mut() { if plugin.type_name() == "songbirdMixer" { plugin.params_mut()[0].set(0.5); // gain plugin.params_mut()[1].set(-0.3); // pan break; } } ``` ### Adding a Mixer to a new node ```rust theme={null} // The mixer's apply_to_graph() pushes a Mixer plugin automatically. // For manual node setup: node.processor.push_plugin( Box::new(songbird_plugins::Mixer::with_values(0.8, 0.0)), ); ``` *** ## 7. Cross-Crate Access Patterns The `Processor` trait is used by multiple crates outside the engine. Here's how each crate accesses node state: | Crate | File | Access Pattern | | ----------------- | ---------------------------------- | ------------------------------------------------------------------------------ | | `songbird-engine` | `graph/mod.rs` | `node.processor.process()` | | `songbird-engine` | `audio_io.rs`, `callback_state.rs` | Find Mixer plugin by type\_name, update params | | `songbird-engine` | `debug/snapshot.rs` | Read gain/pan from Mixer plugin params | | `songbird-engine` | `mixer/mod.rs` | `apply_to_graph()` pushes Mixer; `sync_params_to_graph()` updates Mixer params | | `songbird-sync` | `graph_sync.rs` | `build_graph_from_project()` pushes Mixer + Spatial plugins | | `songbird-wasm` | `lib.rs` | Find Mixer plugin by type\_name, read/write params | | `songbird-export` | `bounce.rs`, `lib.rs` | Read/write Mixer plugin mute param, `node.processor.plugins()` | ### What NOT to do ```rust theme={null} // ❌ WRONG — gain/pan are NOT on the processor node.processor.set_gain(0.5); node.processor.set_pan(-1.0); node.processor.gain(); // ✅ CORRECT — find the Mixer plugin and update its params for plugin in node.processor.plugins_mut() { if plugin.type_name() == "songbirdMixer" { plugin.params_mut()[0].set(0.5); // gain plugin.params_mut()[1].set(-1.0); // pan break; } } // ❌ WRONG — mute is NOT on the processor node.processor.set_muted(true); // ✅ CORRECT — mute is on the Mixer plugin (param index 3) for plugin in node.processor.plugins_mut() { if plugin.type_name() == "songbirdMixer" { plugin.params_mut()[3].set(1.0); // 1.0 = muted, 0.0 = unmuted break; } } // ✅ For Vec operations (push, remove, insert, clear): node.processor.plugins_vec_mut().unwrap().clear(); node.processor.plugins_vec_mut().unwrap().insert(0, plugin); ``` ### Watch out: Not everything has a `.processor` Several types have their own `plugins`, `gain`, `pan`, `muted` fields that are **not** AudioNode. Do NOT change these to use `processor.xxx()`: | Type | Location | Has own `plugins`/`gain` | | ----------------------- | --------------------------- | ------------------------------------------------------- | | `TrackTemplate` | `project/track_template.rs` | `self.plugins: Vec` | | `ChainPreset` | `graph/signal_router.rs` | `self.plugins: Vec` | | `songbird_state::Track` | `songbird-state` | `track.plugins: Vec`, `track.gain: f64` | | `ClipState` | `songbird-state` | `clip.gain: f64`, `clip.muted: bool` | | `ScheduledMidiClip` | `clip_scheduler` | `clip.gain: f64`, `clip.muted: bool` | | `ClipEffectChain` | `scheduling/clip_effects` | `chain.muted: bool`, `chain.pan: f32` | | `Connection` | `graph/mod.rs` | `conn.gain: f64` | | `MixerTrackResponse` | tests | `t.gain`, `t.pan`, `t.muted` | If you see `data.tracks[0].plugins` or `track.gain`, those are **project data** — they're fine as-is. *** ## 8. Migration Status ### ✅ COMPLETE — Plugin-Based Gain/Pan/Spatial (May 2026) Gain, pan, and spatial positioning have been removed from the `Processor` trait and `PluginChainProcessor`. They are now handled by dedicated plugin nodes following the Bitwig/modular DAW pattern. **What was changed:** | Area | Details | | ---------------------- | --------------------------------------------------------------------------------------------------------------------- | | `Processor` trait | Removed `gain()`, `set_gain()`, `pan()`, `set_pan()`, `spatial_position()`, `set_spatial()`, `muted()`, `set_muted()` | | `PluginChainProcessor` | Removed `gain`, `pan`, `spatial_*` fields | | `graph/mod.rs` | Removed gain/pan multiply loops from `process_single_node`, `process_level_parallel`, `process_single_node_with_midi` | | `Mixer` plugin | Gain + equal-power pan; renamed from Fader; added `with_values(gain, pan)` constructor | | `SpatialPanner` plugin | New plugin for HRTF metadata (azimuth, elevation, distance) | | `audio_io.rs` | `SetNodeGain`/`SetNodePan` now target Mixer plugin params | | `callback_state.rs` | `SetTrackGain`/`SetTrackPan`/`SetTrackSpatial` now target plugin params | | `mixer/mod.rs` | `apply_to_graph()` pushes Mixer; `sync_params_to_graph()` updates Mixer params | | `graph_sync.rs` | `build_graph_from_project()` pushes Mixer + Spatial plugins | | `project_loader.rs` | Inserts instrument/FX/strip plugins before Mixer (correct chain order) | | `songbird-wasm` | Updated to read/write Mixer plugin params | | All test files | Updated across engine, sync, export (3468+ tests pass) | ### ⏳ PENDING — Next Steps | Task | Priority | Details | | --------------------- | -------- | ---------------------------------------------------------------------------------- | | Performance profiling | Medium | Profile Mixer plugin overhead vs old baked-in gain/pan. Expected: negligible. | | Modular graph usage | Low | `SingleProcessor` is implemented but not yet used by `build_graph_from_project()`. | | PDC integration | Low | `pdc.rs` may need updates if processor types report different latencies. | *** ## 9. Testing the Graph ### Quick validation ```bash theme={null} cd rust && cargo check -p songbird-engine --tests ``` ### Full test suite ```bash theme={null} cd rust && cargo test -p songbird-engine ``` ### Cross-crate (recommended for graph changes) ```bash theme={null} cd rust && cargo check --workspace --tests cd rust && cargo test --workspace ``` ### Headless server (integration testing) ```bash theme={null} ./utils/agent-headless.sh # Then open: http://localhost:8080?ws=localhost:9090 ``` *** ## 10. Adding a New Processor Type 1. Create a new file in `graph/`, e.g., `my_processor.rs` 2. Implement the `Processor` trait (see `chain_processor.rs` as template) 3. Add `pub mod my_processor;` to `graph/mod.rs` 4. Use it: `AudioNode::with_processor(id, "name", Box::new(MyProcessor::new()))` 5. Update `build_graph_from_project()` in `songbird-sync/graph_sync.rs` if this processor type should be created from project data ### Trait methods to implement | Method | Required? | Default | | ----------------------------- | --------- | ----------------------------- | | `process()` | **Yes** | — | | `processor_name()` | **Yes** | — | | `processor_type()` | **Yes** | — | | ~~`muted()` / `set_muted()`~~ | Removed | Now on Mixer plugin param\[3] | | `plugins()` / `plugins_mut()` | No | empty slice | | `plugins_vec_mut()` | No | None | | `push_plugin()` | No | panics | | `is_instrument()` | No | false | | `latency_samples()` | No | 0 | | `has_tail()` | No | false | | `plugin_count()` | No | `plugins().len()` |