Skip to main content

Lib (react_ui/src/lib/)

Non-React utility modules — native bridge, AI integration, theme system, and shared helpers.

Structure

lib/
├── index.ts          — Re-exports (Juce, isPlugin, cn, etc.)
├── utils.ts          — Shared utilities: cn() for class merging, isPlugin detection
├── native/           — Native bridge layer (nativeInvoke, SliderState, etc.)
├── ai/               — AI copilot (multi-provider, intent-driven)
└── theme/            — Dark/light theme CSS and system detection

Modules

native/ — Native Bridge Bootstrap

Bootstraps the window.__SONGBIRD__ global that the sync engine’s transports depend on.
FilePurpose
window.d.tsTypeScript declarations for window.__SONGBIRD__ global — backend event system, native function dispatch.
check_native_interop.jsBootstraps window.__SONGBIRD__ at import time. Warns in dev mode when running outside the backend bridge.
Note: native.js and native.d.ts have been deleted. All command dispatch now goes through send() from @/sync/api or invoke() from @/sync/transports/promiseHandler.

ai/ — AI Copilot (Multi-Provider)

Modular AI copilot with intent-driven speed optimization and multi-provider abstraction.

Architecture

ai/
├── types.ts           — Shared interfaces (LLMProvider, Intent, SpeedProfile, ToolCall)
├── index.ts           — Barrel exports for AI module
├── agents/            — Specialized AI agents
│   ├── base.ts        — Base agent class with shared behavior
│   ├── composer.ts    — Composition agent: writes .bird patterns, arrangements, and melodies
│   ├── mix-engineer.ts— Mixing agent: adjusts levels, panning, EQ, compression
│   ├── sound-designer.ts — Sound design agent: tweaks synth parameters, creates presets
│   └── audio-creator.ts — Audio creation agent: generates audio via Lyria
├── core/              — Core AI infrastructure
│   ├── copilot.ts     — Main orchestrator: shortcuts → intent → profile → provider
│   ├── router.ts      — Intent classifier (regex/keyword → Intent → SpeedProfile)
│   ├── prompts.ts     — Modular system prompt builder (conditional sections)
│   └── compactor.ts   — Conversation compaction for long threads
├── tools/             — Tool declarations and execution
├── bird/              — .bird file parsing and validation utilities
├── cache/             — Plugin parameter cache (pre-bake to skip round-trips)
├── docs/              — AI documentation and brainstorm notes
└── providers/
    ├── index.ts       — Provider registry & factory
    └── gemini.ts      — Gemini provider (LLMProvider implementation)

Request Pipeline

User message
  → shortcuts.ts     (instant: /bpm 120 → done, no API call)
  → router.ts        (classify intent: mixer/param/bird_edit/chat)
  → SpeedProfile     (model tier + thinking + tool subset)
  → prompts.ts       (build tailored prompt — 2KB for simple, 8KB for composition)
  → tools.ts         (select 1-3 tools instead of 9)
  → provider.stream  (Gemini/OpenAI/local with resolved config)
  → tool-executor.ts (handle function calls → native bridge)
  → result

Speed Optimizations

OptimizationFileImpact
Slash commands (skip LLM)shortcuts.ts100% faster for /bpm, /vol, etc.
Dynamic tool subsettools.ts30-50% TTFT — 2 tools instead of 9
Disable thinking for simple taskscopilot.ts20-40% — skip reasoning tokens
Smart model routingrouter.ts40-60% — Flash Lite for trivial
Modular prompt (smaller)prompts.ts10-20% TTFT — 2KB vs 8KB
Plugin param cacheparam-cache.tsEliminates 1 round-trip (~2-4s)
Conversation compactioncopilot.ts10-30% TTFT for long threads
Context cachingproviders/gemini.ts30-60% TTFT on repeated calls

Provider Interface

interface LLMProvider {
  name: string;
  models: Record<ModelTier, string>;
  streamMessage(tier: ModelTier, request: StreamRequest): Promise<LLMResponse>;
  createCache?(prompt: string, tools?: ToolDeclaration[]): Promise<string | null>;
}
Currently implemented: Gemini. Extensible to OpenAI, Anthropic, or on-device models.

Specialized Agents

The AI copilot uses a multi-agent architecture where different agents handle different types of requests:
AgentFileSpecialization
Composeragents/composer.tsWriting .bird patterns, melodies, chord progressions, arrangements
Mix Engineeragents/mix-engineer.tsAdjusting levels, panning, EQ, compression, spatial effects
Sound Designeragents/sound-designer.tsTweaking synth parameters, creating presets, macro mapping
Audio Creatoragents/audio-creator.tsGenerating audio via Lyria (text-to-music, style transfer)
Inline generation (Option+click-drag on a track lane) routes by track type: audio → ml.generate_lyria3 sync command (Lyria 3 Pro in Rust), MIDI → clip.create_midi_clip + ml.chat with intentOverride: bird_edit so the composer role fills the empty clip.

theme/ — Theme System

FilePurpose
index.tsTheme detection and application. Supports auto (follows system), dark, and light modes. Injects CSS variables.
dark.cssDark theme CSS custom properties (colors, shadows, surfaces).
light.cssLight theme CSS custom properties.

Conventions

  • isPlugin — Boolean flag (true when running inside the native WebView, false in browser dev mode). Guards all native function calls.
  • cn() — Class name merging utility (wraps clsx + tailwind-merge). Use this instead of raw template literals for conditional Tailwind classes.
  • No React in lib/ — This layer should remain framework-agnostic. No hooks, no JSX. Components that need these utilities import from @/lib.