Songbird Mobile

Mobile foundation for the Songbird DAW — iOS and Android apps built with Tauri v2, sharing the same React UI and Rust audio engine as the desktop app.

Architecture

┌─────────────────────────────────────────────────────────────┐
│  React UI (shared with desktop)                             │
│  Vite + React 19 + Tailwind v4 + Zustand                   │
│  Rendered in WKWebView (iOS) / Android System WebView       │
│  ├─ Touch-responsive layout (phone / tablet adaptive)       │
│  ├─ Haptic feedback via Tauri IPC → native plugin           │
│  └─ Safe area inset awareness                               │
└─────────────────┬───────────────────────────────────────────┘
                  │ Tauri IPC (invoke / events)
┌─────────────────▼───────────────────────────────────────────┐
│  Rust Backend (shared crates from rust/)                    │
│  ├─ songbird-engine    — Audio graph, transport, tempo      │
│  ├─ songbird-state     — Project model, undo/redo           │
│  ├─ songbird-clips     — Bird parser, MIDI generation       │
│  ├─ songbird-plugins   — Stock plugin DSP                   │
│  └─ songbird-export    — Audio export                       │
│                                                             │
│  Mobile-Specific Rust (mobile/shared)                       │
│  ├─ audio_session      — Platform audio session management  │
│  ├─ haptics            — Touch feedback abstractions        │
│  └─ platform           — Device capability queries          │
└─────────────────┬───────────────────────────────────────────┘
                  │ Tauri Mobile Plugin Bridge
┌─────────────────▼───────────────────────────────────────────┐
│  Native Platform Layer                                      │
│  iOS:     Swift — AVAudioSession, CoreMIDI, UIFeedback      │
│  Android: Kotlin — AudioManager, android.media.midi, Vibrator│
└─────────────────────────────────────────────────────────────┘

Why Tauri v2 (not React Native / Flutter / Capacitor)

Criterion	Tauri v2	React Native	Flutter	Capacitor
UI code reuse	100% (same React app)	0% (rewrite)	0% (Dart)	~90% (wrapper)
Audio engine	Shared Rust crates	FFI bridge needed	FFI bridge needed	JS bridge (slow)
Binary size	~10MB	~30MB+	~15MB+	~20MB+
Latency	Native (Rust → cpal)	JS bridge overhead	Dart FFI overhead	JS bridge overhead
Already using	Yes (desktop app)	No	No	No

Project Structure

mobile/
├── README.md               ← You are here
├── Cargo.toml              ← Workspace root (references rust/ crates)
├── shared/                 ← Shared mobile Rust code
│   ├── Cargo.toml
│   └── src/
│       ├── lib.rs
│       ├── commands.rs        — Shared IPC commands (transport, mixer, state, etc.)
│       ├── audio_session.rs   — Audio session config/events/traits
│       ├── haptics.rs         — Haptic feedback styles/traits
│       └── platform.rs        — Device info, screen metrics, thermal state
├── ios/                    ← iOS Tauri app
│   ├── Cargo.toml
│   ├── tauri.conf.json
│   ├── build.rs
│   ├── capabilities/default.json
│   ├── src/
│   │   ├── main.rs             — Binary entry (desktop testing)
│   │   ├── lib.rs              — Mobile entry point (uses shared commands)
│   │   └── commands.rs         — iOS-only commands (get_device_info)
│   ├── gen/apple/              — Xcode project scaffold
│   │   ├── Songbird_iOS/
│   │   │   ├── AppDelegate.swift
│   │   │   └── Info.plist
│   │   └── Assets.xcassets/
│   └── swift-plugins/          — Native Swift plugins
│       └── AudioSessionPlugin/
└── android/                ← Android Tauri app
    ├── Cargo.toml
    ├── tauri.conf.json
    ├── build.rs
    ├── capabilities/default.json
    ├── src/
    │   ├── main.rs
    │   ├── lib.rs              — Mobile entry point (uses shared commands)
    │   └── commands.rs         — Android-only commands (get_device_info)
    ├── gen/android/            — Gradle project scaffold
    │   ├── build.gradle.kts
    │   ├── settings.gradle.kts
    │   └── app/
    │       ├── build.gradle.kts
    │       └── src/main/
    │           ├── AndroidManifest.xml
    │           └── kotlin/com/songbird/app/
    └── kotlin-plugins/         — Native Kotlin plugins
        └── AudioSessionPlugin/

Getting Started

Prerequisites

iOS (requires macOS):

Xcode 16+ with iOS 16+ SDK
Rust with aarch64-apple-ios target: rustup target add aarch64-apple-ios
Tauri CLI: cargo install tauri-cli
CocoaPods (if using Swift plugins): gem install cocoapods

Android (macOS, Linux, or Windows):

Android Studio with SDK 26+ (Android 8.0 Oreo)
Android NDK (installed via Android Studio SDK Manager)

Rust with Android targets:

rustup target add aarch64-linux-android  # ARM64 devices
rustup target add x86_64-linux-android   # Emulator

Tauri CLI: cargo install tauri-cli
Set ANDROID_HOME and NDK_HOME environment variables

Build & Run

# First time: build the React UI
cd react_ui && npm install && npm run build && cd ..

# iOS — build via utils script
./utils/build-ios.sh              # debug build
./utils/build-ios.sh --release    # release build
./utils/build-ios.sh --dev        # dev server + hot-reload

# Android — build via utils script
./utils/build-android.sh          # debug build (APK)
./utils/build-android.sh --release # release build (AAB + APK)
./utils/build-android.sh --dev    # dev server + hot-reload

# Or manually:
cd mobile/ios && cargo tauri ios dev      # iOS simulator
cd mobile/android && cargo tauri android dev  # Android emulator

Desktop Development

The mobile app crates can also be compiled and run on the desktop host for rapid iteration without a simulator:

cd mobile/ios   # or mobile/android
cargo run       # Runs as a desktop Tauri window

This uses the same Tauri WebView but without the native mobile plugins. Mobile-specific commands (haptics, audio session) return placeholder values when running on desktop.

Mobile-Specific IPC Commands

These commands extend the shared desktop command set with mobile-only functionality:

Command	Description	Platform
`get_device_info`	Screen metrics, audio capabilities, thermal state	Both
`configure_audio_session`	Set audio category (playback/record), sample rate, buffer size	Both
`trigger_haptic`	Fire haptic feedback (light/medium/heavy/selection/success)	Both

All shared commands (transport, mixer, state, tempo, audio session, haptics) live in songbird-mobile-shared::commands and are used by both iOS and Android apps. Only get_device_info is platform-specific (returns different defaults per platform).

Audio Session Lifecycle

iOS (AVAudioSession)

App Launch → configure(PlayAndRecord, 44.1kHz, 5ms)
  → setCategory(.playAndRecord)
  → setPreferredSampleRate(44100)
  → setPreferredIOBufferDuration(0.005)
  → setActive(true)

Phone Call → handleInterruption(.began) → pause transport
          → handleInterruption(.ended, shouldResume: true) → resume

Headphones Unplugged → handleRouteChange(.oldDeviceUnavailable) → pause

Background → UIBackgroundMode: audio → engine continues

Android (AudioManager)

App Launch → requestAudioFocus(AUDIOFOCUS_GAIN)
  → AudioAttributes(USAGE_MEDIA, CONTENT_TYPE_MUSIC)
  → cpal opens AAudio/Oboe stream

Phone Call → onAudioFocusChange(AUDIOFOCUS_LOSS_TRANSIENT) → pause
           → onAudioFocusChange(AUDIOFOCUS_GAIN) → resume

Background → ForegroundService(mediaPlayback) → engine continues

Workspace & Dependency Graph

The mobile crates live in their own Cargo workspace (mobile/Cargo.toml) that references the core rust/ crates via path dependencies. This keeps a single Cargo.lock for mobile while sharing the same engine code:

mobile/Cargo.toml (workspace)
├── mobile/shared ──→ rust/crates/engine/songbird-engine
│                  ──→ rust/crates/songbird-state
├── mobile/ios ────→ mobile/shared (shared commands + types)
└── mobile/android ─→ mobile/shared (shared commands + types)

Roadmap

Phase 1: Foundation (this PR)

Mobile workspace structure with iOS + Android Tauri apps
Shared mobile crate (audio session, haptics, platform, IPC commands)
Shared IPC commands — no duplication between iOS and Android
iOS Xcode project scaffold + Swift AudioSessionPlugin
Android Gradle project scaffold + Kotlin AudioSessionPlugin
Build scripts: utils/build-ios.sh, utils/build-android.sh

Phase 2: Audio Engine Integration

cpal audio output on iOS (CoreAudio via AVAudioSession)
cpal audio output on Android (AAudio/Oboe)
Background audio playback (iOS UIBackgroundMode, Android ForegroundService)
Audio interruption handling (phone calls, Siri, alarms)
MIDI device support (CoreMIDI on iOS, android.media.midi on Android)

Phase 3: Touch UI Adaptations

Responsive layout: phone vs tablet detection
Touch-optimized piano roll (larger hit targets, pinch-to-zoom)
Touch-optimized mixer (swipe between tracks, gesture faders)
Safe area insets for notch/home indicator/nav bar
Haptic feedback integration for note entry and controls

Phase 4: Platform Features

AUv3 plugin hosting on iOS (load third-party Audio Unit plugins)
Inter-app audio on iOS
Files app integration (iOS) / SAF integration (Android)
.bird file type association (open Songbird files from Files/file manager)
Share sheet support (export stems, share projects)

Phase 5: Release

App Store submission (iOS)
Play Store submission (Android)
App icon and launch screen assets
In-app purchase integration (if applicable)

​Songbird Mobile

​Architecture

​Why Tauri v2 (not React Native / Flutter / Capacitor)

​Project Structure

​Getting Started

​Prerequisites

​Build & Run

​Desktop Development

​Mobile-Specific IPC Commands

​Audio Session Lifecycle

​iOS (AVAudioSession)

​Android (AudioManager)

​Workspace & Dependency Graph

​Roadmap

​Phase 1: Foundation (this PR)

​Phase 2: Audio Engine Integration

​Phase 3: Touch UI Adaptations

​Phase 4: Platform Features

​Phase 5: Release