> ## Documentation Index > Fetch the complete documentation index at: https://songbird.studiocollective.xyz/llms.txt > Use this file to discover all available pages before exploring further. # Songbird Rust Engine (`rust/`) # Songbird Rust Engine (`rust/`) The Rust backend provides the complete audio engine, DSP plugins, clip management, state synchronization, and the dispatch layer for Songbird. The audio engine and entire backend are pure Rust; a thin C ABI FFI bridge (`songbird-host-ffi`) hosts third-party VST3/AU plugins via a JUCE wrapper. The same Rust core powers the Tauri desktop app, the headless WebSocket server, the CLI, and the WASM/web build. ## Tech Stack * **Rust 2021 edition** — Systems language with memory safety guarantees * **Tauri v2** — Desktop app shell * **cpal** — Cross-platform audio I/O * **midir** — Cross-platform MIDI I/O * **rayon** — Data-parallel graph processing * **serde / serde\_json** — Serialization (project files, IPC, state sync) * **ts-rs** — TypeScript bindings generated from Rust enums * **hound** — WAV file reading/writing * **git2** (libgit2) — Git operations for sync engine * **crossbeam-channel** — Lock-free communication between audio and UI threads ## Workspace Structure The workspace is organized into seven layers, bottom-up: ``` rust/ ├── Cargo.toml — Workspace root (34 crates) ├── Cargo.lock ├── README.md — This file ├── crates/ │ ├── core/ — Zero-dependency foundations │ │ ├── songbird-types/ — Shared types (ids, project defaults, value trees) │ │ ├── songbird-dsp/ — Pure DSP primitives (filters, envelopes, oscillators) │ │ ├── songbird-theory/ — Music theory (scales, chords, voicings) │ │ └── songbird-ffi/ — Low-level FFI helpers │ │ │ ├── engine/ — Audio engine + DSP │ │ ├── songbird-engine/ — Audio graph, transport, scheduling, automation, ring buffer │ │ ├── songbird-plugins/ — 28+ stock DSP plugins (effects + instruments) │ │ ├── songbird-modular/ — Node-graph DSP runtime (Songbird Modular editor) │ │ ├── songbird-clips/ — Clip model, .bird parser/serializer │ │ ├── songbird-export/ — Offline rendering (master, stems, bounce) │ │ ├── songbird-record/ — Audio + MIDI recording │ │ └── songbird-reverb-vst3/ — Stock reverb packaged as a VST3 │ │ │ ├── data/ — State, sync, collaboration, analytics │ │ ├── songbird-state/ — Project model, undo/redo, persistence │ │ ├── songbird-sync/ — Sync engine (dispatch bridge, channel guards, transports) │ │ ├── songbird-files/ — Project file I/O, archive format, VFS layer │ │ ├── songbird-collab/ — Multiplayer cursors, presence, git-based merge │ │ └── songbird-analytics/ — Dispatch-layer analytics + Sentry activation │ │ │ ├── ml/ — Machine learning + LLM integration │ │ ├── songbird-agent/ — LLM agent (Anthropic / Gemini / local) │ │ ├── songbird-inference/ — Local inference runtime │ │ └── songbird-separator/ — Stem separation │ │ │ ├── integration/ — Platform + external bridges │ │ ├── songbird-host-ffi/ — VST3/AU plugin hosting via JUCE FFI │ │ ├── songbird-plugin-registry/ — Unified plugin registry + cache │ │ ├── songbird-device-events/ — OS-level device-events bridge │ │ ├── songbird-voices/ — Voice allocation + MIDI routing helpers │ │ ├── songbird-lyria/ — Lyria audio-generation integration │ │ ├── songbird-veo/ — Veo video-generation integration │ │ └── songbird-composition/ — Gemini composition glue │ │ │ ├── orchestration/ — Project + session lifecycle (above engine/state, below app) │ │ ├── songbird-project/ — Project open/save as a single canonical path │ │ ├── songbird-session/ — Bird→session loader, scheduler lifecycle helpers │ │ ├── songbird-scenarios/ — JSON scenarios that drive Songbird into a starting state │ │ └── songbird-orchestrator/ — Shared EngineBackend: session build, device │ │ management, plugin swaps, master capture, │ │ plugin editor hosting. Used by ALL frontends. │ │ │ └── app/ — Application shells │ ├── songbird-app/ — Tauri v2 desktop app │ ├── songbird-cli/ — CLI for batch render + validation │ ├── songbird-headless/ — Standalone WebSocket server │ ├── songbird-mcp/ — MCP server bridge (UDS → dispatch) │ └── songbird-wasm/ — WASM/web runtime entry point └── vendor/ — Patched dependencies (midir) ``` ## Crate Dependency Graph Every edge below is an actual `[dependencies]` entry in the relevant `Cargo.toml` (dev-dependencies excluded). Crates are clustered by the `crates//` directory they live in. ``` ┌─ app/ — binary entry points ───────────────────────────────────────┐ │ songbird-app → orchestrator, session, project, scenarios, │ │ engine, plugins, clips, export, record, │ │ sync, state, analytics, files, collab, │ │ host-ffi, plugin-registry, lyria, │ │ inference, types │ │ songbird-headless → orchestrator, session, project, engine, │ │ plugins, export, record, sync, state, │ │ host-ffi, plugin-registry, inference, │ │ dsp, types │ │ songbird-cli → session, project, engine, export, clips, │ │ plugin-registry, sync, state │ │ songbird-wasm → session, project, engine, plugins, clips, │ │ plugin-registry, sync, state, types │ │ songbird-mcp → (no in-tree deps; talks to app via UDS) │ └────────────────────────────────────────────────────────────────────┘ ┌─ orchestration/ ───────────────────────────────────────────────────┐ │ songbird-orchestrator → engine, export, record, plugins, modular, │ │ session, sync, state, host-ffi, │ │ plugin-registry, device-events, dsp, types │ │ songbird-session → engine, plugins, clips, host-ffi, │ │ plugin-registry, sync, state, types │ │ songbird-project → plugins, clips, state, types │ │ songbird-scenarios → sync │ └────────────────────────────────────────────────────────────────────┘ ┌─ engine/ — audio + DSP ────────────────────────────────────────────┐ │ songbird-engine → clips, plugins, plugin-registry, state, │ │ dsp, ffi, types │ │ songbird-export → engine, session, plugins, plugin-registry, │ │ sync, state, clips, dsp, types │ │ songbird-record → engine, state │ │ songbird-clips → plugins, types │ │ songbird-modular → plugins, dsp, types │ │ songbird-plugins → lyria, dsp, types │ │ songbird-reverb-vst3 → plugins, dsp │ └────────────────────────────────────────────────────────────────────┘ ┌─ data/ — state + sync + collab ────────────────────────────────────┐ │ songbird-sync → state, project, engine, clips, plugins, │ │ plugin-registry, files, record, analytics, │ │ agent, inference, lyria, separator, veo, │ │ dsp, types │ │ songbird-state → files, types │ │ songbird-collab → state │ │ songbird-files → types │ │ songbird-analytics → (no in-tree deps) │ └────────────────────────────────────────────────────────────────────┘ ┌─ ml/ ──────────────────────────────────────────────────────────────┐ │ songbird-agent → inference │ │ songbird-separator → inference, dsp │ │ songbird-inference → (no in-tree deps) │ └────────────────────────────────────────────────────────────────────┘ ┌─ integration/ ─────────────────────────────────────────────────────┐ │ songbird-host-ffi → plugins, types │ │ songbird-plugin-registry → host-ffi, modular, plugins, state │ │ songbird-composition → theory │ │ songbird-device-events → (no in-tree deps) │ │ songbird-lyria → (no in-tree deps) │ │ songbird-veo → (no in-tree deps) │ │ songbird-voices → (no in-tree deps; no in-tree consumers)│ └────────────────────────────────────────────────────────────────────┘ ┌─ core/ — zero-dep foundations ─────────────────────────────────────┐ │ songbird-types songbird-dsp songbird-theory songbird-ffi └────────────────────────────────────────────────────────────────────┘ ``` A few things worth calling out from the graph: * **`songbird-orchestrator` is the shared `EngineBackend`** — `songbird-app` and `songbird-headless` both depend on it rather than duplicating session-build, device-management, and plugin-editor code. * **`songbird-sync` is the dispatch hub** — it's the only crate that depends on the ML, generation (Lyria, Veo, separator), and analytics crates, because it routes commands from those channels to their handlers. The engine and state crates know nothing about ML. * **`songbird-state` does not depend on `songbird-sync`** at runtime — the dependency is one-way (sync → state). Echo suppression and undo/redo live in `state`; routing lives in `sync`. * **`songbird-mcp` has zero Rust dependencies** — it's a standalone binary that talks to the running desktop app over a Unix Domain Socket bridge (`~/.songbird/mcp.sock`), so MCP `tools/call` requests reach the same `dispatch` without a compile-time link. * **`songbird-voices` and `songbird-composition` are isolated** — `voices` has no in-tree consumers yet; `composition` only depends on `theory`. Both are scaffolding for upcoming work. * **The `core/` layer (`types`, `dsp`, `theory`, `ffi`) is zero-dep** — every other crate ultimately roots into it, but it depends on nothing in the workspace. ## Architecture Overview ``` ┌─────────────────────────────────────────────────────────────────┐ │ React UI (TypeScript) │ │ ├─ Zustand stores ← → sync engine (send / sendRT) │ │ └─ rtFrame listener (~30Hz metering data) │ ├─────────────────────────────────────────────────────────────────┤ │ songbird-app (Tauri v2) / songbird-headless / songbird-wasm │ │ ├─ ONE Tauri #[command] (native_invoke) or WS / in-process │ │ └─ Routes every call through songbird_sync::dispatch │ ├─────────────────────────────────────────────────────────────────┤ │ songbird-sync │ │ ├─ dispatch(channel.command, payload) │ │ ├─ 20 per-channel command handlers │ │ └─ Echo suppression / guards / persistence per channel │ ├─────────────────────────────────────────────────────────────────┤ │ songbird-engine │ │ ├─ AudioGraph — connection-based DAG + topological sort │ │ ├─ Transport — play/stop/record/loop/seek/metronome │ │ ├─ ClipScheduler — timeline MIDI/audio clip playback │ │ ├─ ClipLauncher — Ableton-style session view grid │ │ ├─ AutomationState — breakpoint envelopes per parameter │ │ ├─ Mixer — track types, sends, groups │ │ ├─ MidiRouter — MIDI event routing + filtering │ │ └─ SpscRingBuffer — lock-free audio↔UI communication │ ├─────────────────────────────────────────────────────────────────┤ │ songbird-plugins │ │ ├─ Plugin trait — uniform interface for all DSP │ │ ├─ 20 effects — ChannelStrip, Reverb (3), Delay, Tape, … │ │ └─ 11 instruments — 4OSC, STK, 303, FM, Pluck, Sub, Wavetable,│ │ Sampler, DrumMachine, Metronome, … │ ├─────────────────────────────────────────────────────────────────┤ │ cpal / midir — Hardware audio + MIDI I/O │ └─────────────────────────────────────────────────────────────────┘ ``` ## Build & Development ```bash theme={null} cd rust # Build the entire workspace cargo build --workspace # Run all tests (5,240 tests) cargo test --workspace --exclude songbird-app -- --skip test_robust_session_channel_concurrent_access # Run clippy linting (zero warnings) cargo clippy --workspace --exclude songbird-app -- -D warnings # Build the Tauri app (requires Tauri CLI) cargo tauri build # Run specific crate tests cargo test -p songbird-engine cargo test -p songbird-plugins cargo test -p songbird-state # Run a specific test by name cargo test -p songbird-engine -- test_graph_parallel ``` ## Audio Processing Pipeline The audio callback processes data through this pipeline each block: ``` Transport.advance(num_samples) → ClipScheduler — resolves which clips are active at current position → AudioGraph.process() — topological sort, parallel level execution via rayon → For each node (track): 1. Gather MIDI events from ClipScheduler 2. Process through PluginChain (instruments → effects) 3. Apply gain, pan, mute/solo 4. Sum into output connections → Master output buffer → cpal audio callback writes to hardware ``` ### Real-Time Safety The audio callback path is designed for zero allocation: * **AudioBuffer** — inline `[*mut f32; 8]`, no heap allocation * **ScratchPool** — pre-allocated per-node buffers, assigned in `rebuild_topology()` * **SpscRingBuffer** — lock-free single-producer single-consumer for engine↔UI commands * **No Mutex in audio path** — all shared state accessed via lock-free ring buffers * **Plugin trait** — `process()` operates on pre-allocated buffers ## Dispatch Layer The Tauri app exposes a **single** `#[tauri::command]` (`native_invoke`) that funnels every UI ↔ backend call through `songbird_sync::dispatch::dispatch()`. Headless mode uses the same dispatch over a WebSocket transport; the WASM build uses an in-process transport. This is enforced as a Hard Rule — no new `#[tauri::command]` is allowed outside the sync engine. Commands are addressed by `"channel.action"` strings (e.g. `transport.play`, `mixer.volume`, `clip.delete`). Events flow the other way as `"channel:event"` (e.g. `transport:state`, `mixer:audio_clip_peaks`). The 20 channels live under `rust/crates/data/songbird-sync/src/channels/`; each has a `defs.rs` (events / command enums via ts-rs) and a `commands.rs` (dispatch arms). To add a new command: 1. Add a variant to the appropriate channel's `defs.rs` command enum. 2. Add a dispatch arm in the channel's `commands.rs`. 3. Run `./utils/sync-types` to regenerate TypeScript bindings. 4. Use `send('channel.action', payload)` from the React side. The full skill is in `.claude/skills/add-sync-path/SKILL.md`. ## State Management ``` React setState() → send('channel.action', payload) // sync engine client → native_invoke or WebSocket or in-process transport → songbird_sync::dispatch // single dispatcher → channel commands.rs // per-channel handler → ctx.store.mutate_described(...) // typed mutation → guards → echo check → subscribers → persist → Git commit (libgit2) for undoable mutations ``` The state pipeline is a single clean design: * **Typed mutations** — channels declare command/event enums via ts-rs; TS types are generated, not hand-written. * **Echo suppression** — version counters and source tags prevent loops between the React and engine sides. * **Guard flags** — slider-drag guard, persistence gate, and other guards configured per-channel. * **Undo/redo** — every undoable mutation is committed to an in-process libgit2 repo; undo/redo are git operations against `refs/heads/main` and `refs/redo-tip`. See `crates/data/songbird-state/SPEC.md` for the full invariant set. ## Testing The workspace has 5,240 tests organized by purpose: | Test Suite | Location | Count | What It Tests | | ----------------------- | ------------------------------- | ------- | --------------------------------------------------------------------------- | | Unit tests | Each crate's `mod tests` | \~2,500 | Individual module correctness | | E2E engine tests | `engine_e2e_tests.rs` | 15 | Full pipeline: transport → graph → clips → output | | Audio output tests | `audio_output_tests.rs` | 25 | Waveform correctness, DSP parity, golden output | | IPC roundtrip tests | `ipc_roundtrip_tests.rs` | 87 | Serialize/deserialize all IPC response types | | IPC workflow tests | `ipc_workflow_tests.rs` | 15 | Full user workflows through IPC layer | | IPC coverage tests | `ipc_handler_coverage_tests.rs` | 54 | Every sync-engine channel command wired correctly | | Engine controller tests | `engine_controller.rs` | 179 | Offline rendering, callbacks, cross-crate integration, robustness hardening | | Edge case tests | `edge_case_tests.rs` | 25 | NaN, Inf, empty buffers, extreme values | | Stress tests | `stress_tests.rs` | 15 | Concurrent mutations, MIDI floods, large graphs | | Performance tests | `perf_tests.rs` | 15 | Graph throughput, parallel speedup, zero-alloc | | RT safety tests | `rt_safety_tests.rs` | 25 | Zero allocation in audio callback path | | Project format tests | `project_format_tests.rs` | 15 | JSON round-trip, forward compat, Unicode | | Plugin factory tests | `songbird-plugins/lib.rs` | 3 | Create, round-trip state for all stock plugins | ```bash theme={null} # Run all tests cargo test --workspace --exclude songbird-app # Run only E2E tests cargo test -p songbird-engine -- engine_e2e # Run only audio output verification cargo test -p songbird-engine -- audio_output # Run only IPC handler coverage cargo test -p songbird-engine -- ipc_handler_coverage ``` ## Key Design Decisions ### Graph Topology Songbird uses a **connection-based DAG** with `Connection` structs carrying port indices, rather than a flat track-centric model. This enables sidechain routing, parallel compression, and multi-bus routing without architecture changes. Topological sort via Kahn's algorithm is cached in `rebuild_topology()`. ### Parallel Processing (via rayon) Independent branches of the audio graph execute concurrently. The topological sort assigns **levels** — nodes at the same level have no dependencies and can be processed in parallel via `rayon::scope`. All metadata is cached so `process()` does zero allocation. ### Nested Device Chains (Bitwig-inspired) The `PluginChain` supports `ChainNode` variants that can be either a `Plugin` or a `Container` holding sub-chains. This enables parallel processing chains, multiband splits, and mid/side processing at the track level. ### Plugin Trait All plugins — stock and external — implement the same `Plugin` trait: ```rust theme={null} pub trait Plugin: Send { fn type_name(&self) -> &str; fn name(&self) -> &str; fn initialize(&mut self, sample_rate: f64); fn reset(&mut self); fn process(&mut self, buffer: &mut AudioBuffer, midi_events: &[MidiEvent]); fn params(&self) -> Vec; fn set_param(&mut self, name: &str, value: f64); fn save_state(&self) -> Vec; fn load_state(&mut self, data: &[u8]); fn tail_seconds(&self) -> f64; fn latency_samples(&self) -> usize; fn accepts_midi(&self) -> bool; fn produces_midi(&self) -> bool; } ``` ### FFI Bridge (for C++ Plugin Hosting) Rust owns the audio thread. Third-party VST3/AU plugins are hosted via JUCE through a C ABI FFI layer (`songbird-host-ffi`). The bridge uses non-interleaved buffer layout matching JUCE's `AudioBuffer`. The `Plugin` trait does not assume shared memory, enabling future out-of-process hosting. ## Headless WebSocket Server The `songbird-headless` crate runs a standalone WebSocket server that serves the same protocol as the Tauri app, enabling browser-based UI without Tauri: ```bash theme={null} # Run headless server cargo run -p songbird-headless -- --port 9000 --no-audio # Flags: # --port WebSocket port (default: 9000) # --project Load a .bird project file # --no-audio Disable cpal audio output (timer-based transport fallback) ``` **Protocol:** Text frames use `{ "eventId": "", "payload": }` format (same as the desktop dispatch). Binary RT frames broadcast at \~30fps with metering, transport position, recording peaks, and Link status. See [ARCHITECTURE.md](ARCHITECTURE.md) for the full system design.