Skip to main content

​
Songbird Rust Engine (rust/)

The Rust backend provides the complete audio engine, DSP plugins, clip management, state synchronization, and the dispatch layer for Songbird. The audio engine and entire backend are pure Rust; a thin C ABI FFI bridge (songbird-host-ffi) hosts third-party VST3/AU plugins via a JUCE wrapper. The same Rust core powers the Tauri desktop app, the headless WebSocket server, the CLI, and the WASM/web build.

​
Tech Stack

  • Rust 2021 edition — Systems language with memory safety guarantees
  • Tauri v2 — Desktop app shell
  • cpal — Cross-platform audio I/O
  • midir — Cross-platform MIDI I/O
  • rayon — Data-parallel graph processing
  • serde / serde_json — Serialization (project files, IPC, state sync)
  • ts-rs — TypeScript bindings generated from Rust enums
  • hound — WAV file reading/writing
  • git2 (libgit2) — Git operations for sync engine
  • crossbeam-channel — Lock-free communication between audio and UI threads

​
Workspace Structure

The workspace is organized into seven layers, bottom-up: 
rust/
├── Cargo.toml                  — Workspace root (34 crates)
├── Cargo.lock
├── README.md                   — This file
├── crates/
│   ├── core/                   — Zero-dependency foundations
│   │   ├── songbird-types/     — Shared types (ids, project defaults, value trees)
│   │   ├── songbird-dsp/       — Pure DSP primitives (filters, envelopes, oscillators)
│   │   ├── songbird-theory/    — Music theory (scales, chords, voicings)
│   │   └── songbird-ffi/       — Low-level FFI helpers
│   │
│   ├── engine/                 — Audio engine + DSP
│   │   ├── songbird-engine/    — Audio graph, transport, scheduling, automation, ring buffer
│   │   ├── songbird-plugins/   — 28+ stock DSP plugins (effects + instruments)
│   │   ├── songbird-modular/   — Node-graph DSP runtime (Songbird Modular editor)
│   │   ├── songbird-clips/     — Clip model, .bird parser/serializer
│   │   ├── songbird-export/    — Offline rendering (master, stems, bounce)
│   │   ├── songbird-record/    — Audio + MIDI recording
│   │   └── songbird-reverb-vst3/ — Stock reverb packaged as a VST3
│   │
│   ├── data/                   — State, sync, collaboration, analytics
│   │   ├── songbird-state/     — Project model, undo/redo, persistence
│   │   ├── songbird-sync/      — Sync engine (dispatch bridge, channel guards, transports)
│   │   ├── songbird-files/     — Project file I/O, archive format, VFS layer
│   │   ├── songbird-collab/    — Multiplayer cursors, presence, git-based merge
│   │   └── songbird-analytics/ — Dispatch-layer analytics + Sentry activation
│   │
│   ├── ml/                     — Machine learning + LLM integration
│   │   ├── songbird-agent/     — LLM agent (Anthropic / Gemini / local)
│   │   ├── songbird-inference/ — Local inference runtime
│   │   └── songbird-separator/ — Stem separation
│   │
│   ├── integration/            — Platform + external bridges
│   │   ├── songbird-host-ffi/  — VST3/AU plugin hosting via JUCE FFI
│   │   ├── songbird-plugin-registry/ — Unified plugin registry + cache
│   │   ├── songbird-device-events/ — OS-level device-events bridge
│   │   ├── songbird-voices/    — Voice allocation + MIDI routing helpers
│   │   ├── songbird-lyria/     — Lyria audio-generation integration
│   │   ├── songbird-veo/       — Veo video-generation integration
│   │   └── songbird-composition/ — Gemini composition glue
│   │
│   ├── orchestration/          — Project + session lifecycle (above engine/state, below app)
│   │   ├── songbird-project/   — Project open/save as a single canonical path
│   │   ├── songbird-session/   — Bird→session loader, scheduler lifecycle helpers
│   │   ├── songbird-scenarios/ — JSON scenarios that drive Songbird into a starting state
│   │   └── songbird-orchestrator/ — Shared EngineBackend: session build, device
│   │                               management, plugin swaps, master capture,
│   │                               plugin editor hosting. Used by ALL frontends.
│   │
│   └── app/                    — Application shells
│       ├── songbird-app/       — Tauri v2 desktop app
│       ├── songbird-cli/       — CLI for batch render + validation
│       ├── songbird-headless/  — Standalone WebSocket server
│       ├── songbird-mcp/       — MCP server bridge (UDS → dispatch)
│       └── songbird-wasm/      — WASM/web runtime entry point
└── vendor/                     — Patched dependencies (midir)

​
Crate Dependency Graph

Every edge below is an actual [dependencies] entry in the relevant Cargo.toml (dev-dependencies excluded). Crates are clustered by the crates/<group>/ directory they live in. 
┌─ app/ — binary entry points ───────────────────────────────────────┐
│ songbird-app         → orchestrator, session, project, scenarios,  │
│                        engine, plugins, clips, export, record,     │
│                        sync, state, analytics, files, collab,      │
│                        host-ffi, plugin-registry, lyria,           │
│                        inference, types                            │
│ songbird-headless    → orchestrator, session, project, engine,     │
│                        plugins, export, record, sync, state,       │
│                        host-ffi, plugin-registry, inference,       │
│                        dsp, types                                  │
│ songbird-cli         → session, project, engine, export, clips,    │
│                        plugin-registry, sync, state                │
│ songbird-wasm        → session, project, engine, plugins, clips,   │
│                        plugin-registry, sync, state, types         │
│ songbird-mcp         → (no in-tree deps; talks to app via UDS)     │
└────────────────────────────────────────────────────────────────────┘

┌─ orchestration/ ───────────────────────────────────────────────────┐
│ songbird-orchestrator → engine, export, record, plugins, modular,  │
│                         session, sync, state, host-ffi,            │
│                         plugin-registry, device-events, dsp, types │
│ songbird-session      → engine, plugins, clips, host-ffi,          │
│                         plugin-registry, sync, state, types        │
│ songbird-project      → plugins, clips, state, types               │
│ songbird-scenarios    → sync                                       │
└────────────────────────────────────────────────────────────────────┘

┌─ engine/ — audio + DSP ────────────────────────────────────────────┐
│ songbird-engine       → clips, plugins, plugin-registry, state,    │
│                         dsp, ffi, types                            │
│ songbird-export       → engine, session, plugins, plugin-registry, │
│                         sync, state, clips, dsp, types             │
│ songbird-record       → engine, state                              │
│ songbird-clips        → plugins, types                             │
│ songbird-modular      → plugins, dsp, types                        │
│ songbird-plugins      → lyria, dsp, types                          │
│ songbird-reverb-vst3  → plugins, dsp                               │
└────────────────────────────────────────────────────────────────────┘

┌─ data/ — state + sync + collab ────────────────────────────────────┐
│ songbird-sync         → state, project, engine, clips, plugins,    │
│                         plugin-registry, files, record, analytics, │
│                         agent, inference, lyria, separator, veo,   │
│                         dsp, types                                 │
│ songbird-state        → files, types                               │
│ songbird-collab       → state                                      │
│ songbird-files        → types                                      │
│ songbird-analytics    → (no in-tree deps)                          │
└────────────────────────────────────────────────────────────────────┘

┌─ ml/ ──────────────────────────────────────────────────────────────┐
│ songbird-agent        → inference                                  │
│ songbird-separator    → inference, dsp                             │
│ songbird-inference    → (no in-tree deps)                          │
└────────────────────────────────────────────────────────────────────┘

┌─ integration/ ─────────────────────────────────────────────────────┐
│ songbird-host-ffi         → plugins, types                         │
│ songbird-plugin-registry  → host-ffi, modular, plugins, state      │
│ songbird-composition      → theory                                 │
│ songbird-device-events    → (no in-tree deps)                      │
│ songbird-lyria            → (no in-tree deps)                      │
│ songbird-veo              → (no in-tree deps)                      │
│ songbird-voices           → (no in-tree deps; no in-tree consumers)│
└────────────────────────────────────────────────────────────────────┘

┌─ core/ — zero-dep foundations ─────────────────────────────────────┐
│ songbird-types     songbird-dsp     songbird-theory     songbird-ffi
└────────────────────────────────────────────────────────────────────┘
A few things worth calling out from the graph:
  • songbird-orchestrator is the shared EngineBackend — songbird-app and songbird-headless both depend on it rather than duplicating session-build, device-management, and plugin-editor code.
  • songbird-sync is the dispatch hub — it’s the only crate that depends on the ML, generation (Lyria, Veo, separator), and analytics crates, because it routes commands from those channels to their handlers. The engine and state crates know nothing about ML.
  • songbird-state does not depend on songbird-sync at runtime — the dependency is one-way (sync → state). Echo suppression and undo/redo live in state; routing lives in sync.
  • songbird-mcp has zero Rust dependencies — it’s a standalone binary that talks to the running desktop app over a Unix Domain Socket bridge (~/.songbird/mcp.sock), so MCP tools/call requests reach the same dispatch without a compile-time link.
  • songbird-voices and songbird-composition are isolated — voices has no in-tree consumers yet; composition only depends on theory. Both are scaffolding for upcoming work.
  • The core/ layer (types, dsp, theory, ffi) is zero-dep — every other crate ultimately roots into it, but it depends on nothing in the workspace.

​
Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│  React UI (TypeScript)                                          │
│  ├─ Zustand stores ← → sync engine (send / sendRT)              │
│  └─ rtFrame listener (~30Hz metering data)                      │
├─────────────────────────────────────────────────────────────────┤
│  songbird-app (Tauri v2) / songbird-headless / songbird-wasm    │
│  ├─ ONE Tauri #[command] (native_invoke) or WS / in-process    │
│  └─ Routes every call through songbird_sync::dispatch          │
├─────────────────────────────────────────────────────────────────┤
│  songbird-sync                                                  │
│  ├─ dispatch(channel.command, payload)                          │
│  ├─ 20 per-channel command handlers                             │
│  └─ Echo suppression / guards / persistence per channel         │
├─────────────────────────────────────────────────────────────────┤
│  songbird-engine                                                │
│  ├─ AudioGraph      — connection-based DAG + topological sort   │
│  ├─ Transport       — play/stop/record/loop/seek/metronome      │
│  ├─ ClipScheduler   — timeline MIDI/audio clip playback         │
│  ├─ ClipLauncher    — Ableton-style session view grid           │
│  ├─ AutomationState — breakpoint envelopes per parameter        │
│  ├─ Mixer           — track types, sends, groups                │
│  ├─ MidiRouter      — MIDI event routing + filtering            │
│  └─ SpscRingBuffer  — lock-free audio↔UI communication          │
├─────────────────────────────────────────────────────────────────┤
│  songbird-plugins                                               │
│  ├─ Plugin trait    — uniform interface for all DSP             │
│  ├─ 20 effects      — ChannelStrip, Reverb (3), Delay, Tape, … │
│  └─ 11 instruments  — 4OSC, STK, 303, FM, Pluck, Sub, Wavetable,│
│                       Sampler, DrumMachine, Metronome, …        │
├─────────────────────────────────────────────────────────────────┤
│  cpal / midir       — Hardware audio + MIDI I/O                 │
└─────────────────────────────────────────────────────────────────┘

​
Build & Development

cd rust

# Build the entire workspace
cargo build --workspace

# Run all tests (5,240 tests)
cargo test --workspace --exclude songbird-app -- --skip test_robust_session_channel_concurrent_access

# Run clippy linting (zero warnings)
cargo clippy --workspace --exclude songbird-app -- -D warnings

# Build the Tauri app (requires Tauri CLI)
cargo tauri build

# Run specific crate tests
cargo test -p songbird-engine
cargo test -p songbird-plugins
cargo test -p songbird-state

# Run a specific test by name
cargo test -p songbird-engine -- test_graph_parallel

​
Audio Processing Pipeline

The audio callback processes data through this pipeline each block: 
Transport.advance(num_samples)
  → ClipScheduler — resolves which clips are active at current position
  → AudioGraph.process() — topological sort, parallel level execution via rayon
    → For each node (track):
      1. Gather MIDI events from ClipScheduler
      2. Process through PluginChain (instruments → effects)
      3. Apply gain, pan, mute/solo
      4. Sum into output connections
  → Master output buffer
  → cpal audio callback writes to hardware

​
Real-Time Safety

The audio callback path is designed for zero allocation:
  • AudioBuffer — inline [*mut f32; 8], no heap allocation
  • ScratchPool — pre-allocated per-node buffers, assigned in rebuild_topology()
  • SpscRingBuffer — lock-free single-producer single-consumer for engine↔UI commands
  • No Mutex in audio path — all shared state accessed via lock-free ring buffers
  • Plugin trait — process() operates on pre-allocated buffers

​
Dispatch Layer

The Tauri app exposes a single #[tauri::command] (native_invoke) that funnels every UI ↔ backend call through songbird_sync::dispatch::dispatch(). Headless mode uses the same dispatch over a WebSocket transport; the WASM build uses an in-process transport. This is enforced as a Hard Rule — no new #[tauri::command] is allowed outside the sync engine. Commands are addressed by "channel.action" strings (e.g. transport.play, mixer.volume, clip.delete). Events flow the other way as "channel:event" (e.g. transport:state, mixer:audio_clip_peaks). The 20 channels live under rust/crates/data/songbird-sync/src/channels/; each has a defs.rs (events / command enums via ts-rs) and a commands.rs (dispatch arms). To add a new command:
  1. Add a variant to the appropriate channel’s defs.rs command enum.
  2. Add a dispatch arm in the channel’s commands.rs.
  3. Run ./utils/sync-types to regenerate TypeScript bindings.
  4. Use send('channel.action', payload) from the React side.
The full skill is in .claude/skills/add-sync-path/SKILL.md.

​
State Management

React setState()
  → send('channel.action', payload)            // sync engine client
  → native_invoke or WebSocket or in-process transport
  → songbird_sync::dispatch                    // single dispatcher
  → channel commands.rs                         // per-channel handler
  → ctx.store.mutate_described(...)             // typed mutation
  → guards → echo check → subscribers → persist
  → Git commit (libgit2) for undoable mutations
The state pipeline is a single clean design:
  • Typed mutations — channels declare command/event enums via ts-rs; TS types are generated, not hand-written.
  • Echo suppression — version counters and source tags prevent loops between the React and engine sides.
  • Guard flags — slider-drag guard, persistence gate, and other guards configured per-channel.
  • Undo/redo — every undoable mutation is committed to an in-process libgit2 repo; undo/redo are git operations against refs/heads/main and refs/redo-tip.
See crates/data/songbird-state/SPEC.md for the full invariant set.

​
Testing

The workspace has 5,240 tests organized by purpose:
Test SuiteLocationCountWhat It Tests
Unit testsEach crate’s mod tests~2,500Individual module correctness
E2E engine testsengine_e2e_tests.rs15Full pipeline: transport → graph → clips → output
Audio output testsaudio_output_tests.rs25Waveform correctness, DSP parity, golden output
IPC roundtrip testsipc_roundtrip_tests.rs87Serialize/deserialize all IPC response types
IPC workflow testsipc_workflow_tests.rs15Full user workflows through IPC layer
IPC coverage testsipc_handler_coverage_tests.rs54Every sync-engine channel command wired correctly
Engine controller testsengine_controller.rs179Offline rendering, callbacks, cross-crate integration, robustness hardening
Edge case testsedge_case_tests.rs25NaN, Inf, empty buffers, extreme values
Stress testsstress_tests.rs15Concurrent mutations, MIDI floods, large graphs
Performance testsperf_tests.rs15Graph throughput, parallel speedup, zero-alloc
RT safety testsrt_safety_tests.rs25Zero allocation in audio callback path
Project format testsproject_format_tests.rs15JSON round-trip, forward compat, Unicode
Plugin factory testssongbird-plugins/lib.rs3Create, round-trip state for all stock plugins
# Run all tests
cargo test --workspace --exclude songbird-app

# Run only E2E tests
cargo test -p songbird-engine -- engine_e2e

# Run only audio output verification
cargo test -p songbird-engine -- audio_output

# Run only IPC handler coverage
cargo test -p songbird-engine -- ipc_handler_coverage

​
Key Design Decisions

​
Graph Topology

Songbird uses a connection-based DAG with Connection structs carrying port indices, rather than a flat track-centric model. This enables sidechain routing, parallel compression, and multi-bus routing without architecture changes. Topological sort via Kahn’s algorithm is cached in rebuild_topology().

​
Parallel Processing (via rayon)

Independent branches of the audio graph execute concurrently. The topological sort assigns levels — nodes at the same level have no dependencies and can be processed in parallel via rayon::scope. All metadata is cached so process() does zero allocation.

​
Nested Device Chains (Bitwig-inspired)

The PluginChain supports ChainNode variants that can be either a Plugin or a Container holding sub-chains. This enables parallel processing chains, multiband splits, and mid/side processing at the track level.

​
Plugin Trait

All plugins — stock and external — implement the same Plugin trait: 
pub trait Plugin: Send {
    fn type_name(&self) -> &str;
    fn name(&self) -> &str;
    fn initialize(&mut self, sample_rate: f64);
    fn reset(&mut self);
    fn process(&mut self, buffer: &mut AudioBuffer, midi_events: &[MidiEvent]);
    fn params(&self) -> Vec<PluginParam>;
    fn set_param(&mut self, name: &str, value: f64);
    fn save_state(&self) -> Vec<u8>;
    fn load_state(&mut self, data: &[u8]);
    fn tail_seconds(&self) -> f64;
    fn latency_samples(&self) -> usize;
    fn accepts_midi(&self) -> bool;
    fn produces_midi(&self) -> bool;
}

​
FFI Bridge (for C++ Plugin Hosting)

Rust owns the audio thread. Third-party VST3/AU plugins are hosted via JUCE through a C ABI FFI layer (songbird-host-ffi). The bridge uses non-interleaved buffer layout matching JUCE’s AudioBuffer<float>. The Plugin trait does not assume shared memory, enabling future out-of-process hosting.

​
Headless WebSocket Server

The songbird-headless crate runs a standalone WebSocket server that serves the same protocol as the Tauri app, enabling browser-based UI without Tauri: 
# Run headless server
cargo run -p songbird-headless -- --port 9000 --no-audio

# Flags:
#   --port <N>      WebSocket port (default: 9000)
#   --project <path> Load a .bird project file
#   --no-audio       Disable cpal audio output (timer-based transport fallback)
Protocol: Text frames use { "eventId": "<command>", "payload": <json> } format (same as the desktop dispatch). Binary RT frames broadcast at ~30fps with metering, transport position, recording peaks, and Link status. See ARCHITECTURE.md for the full system design.