Skip to main content

​
Songbird Mobile

Mobile foundation for the Songbird DAW — iOS and Android apps built with Tauri v2, sharing the same React UI and Rust audio engine as the desktop app.

​
Architecture

┌─────────────────────────────────────────────────────────────┐
│  React UI (shared with desktop)                             │
│  Vite + React 19 + Tailwind v4 + Zustand                   │
│  Rendered in WKWebView (iOS) / Android System WebView       │
│  ├─ Touch-responsive layout (phone / tablet adaptive)       │
│  ├─ Haptic feedback via Tauri IPC → native plugin           │
│  └─ Safe area inset awareness                               │
└─────────────────┬───────────────────────────────────────────┘
                  │ Tauri IPC (invoke / events)
┌─────────────────▼───────────────────────────────────────────┐
│  Rust Backend (shared crates from rust/)                    │
│  ├─ songbird-engine    — Audio graph, transport, tempo      │
│  ├─ songbird-state     — Project model, undo/redo           │
│  ├─ songbird-clips     — Bird parser, MIDI generation       │
│  ├─ songbird-plugins   — Stock plugin DSP                   │
│  └─ songbird-export    — Audio export                       │
│                                                             │
│  Mobile-Specific Rust (mobile/shared)                       │
│  ├─ audio_session      — Platform audio session management  │
│  ├─ haptics            — Touch feedback abstractions        │
│  └─ platform           — Device capability queries          │
└─────────────────┬───────────────────────────────────────────┘
                  │ Tauri Mobile Plugin Bridge
┌─────────────────▼───────────────────────────────────────────┐
│  Native Platform Layer                                      │
│  iOS:     Swift — AVAudioSession, CoreMIDI, UIFeedback      │
│  Android: Kotlin — AudioManager, android.media.midi, Vibrator│
└─────────────────────────────────────────────────────────────┘

​
Why Tauri v2 (not React Native / Flutter / Capacitor)

CriterionTauri v2React NativeFlutterCapacitor
UI code reuse100% (same React app)0% (rewrite)0% (Dart)~90% (wrapper)
Audio engineShared Rust cratesFFI bridge neededFFI bridge neededJS bridge (slow)
Binary size~10MB~30MB+~15MB+~20MB+
LatencyNative (Rust → cpal)JS bridge overheadDart FFI overheadJS bridge overhead
Already usingYes (desktop app)NoNoNo

​
Project Structure

mobile/
├── README.md               ← You are here
├── Cargo.toml              ← Workspace root (references rust/ crates)
├── shared/                 ← Shared mobile Rust code
│   ├── Cargo.toml
│   └── src/
│       ├── lib.rs
│       ├── commands.rs        — Shared IPC commands (transport, mixer, state, etc.)
│       ├── audio_session.rs   — Audio session config/events/traits
│       ├── haptics.rs         — Haptic feedback styles/traits
│       └── platform.rs        — Device info, screen metrics, thermal state
├── ios/                    ← iOS Tauri app
│   ├── Cargo.toml
│   ├── tauri.conf.json
│   ├── build.rs
│   ├── capabilities/default.json
│   ├── src/
│   │   ├── main.rs             — Binary entry (desktop testing)
│   │   ├── lib.rs              — Mobile entry point (uses shared commands)
│   │   └── commands.rs         — iOS-only commands (get_device_info)
│   ├── gen/apple/              — Xcode project scaffold
│   │   ├── Songbird_iOS/
│   │   │   ├── AppDelegate.swift
│   │   │   └── Info.plist
│   │   └── Assets.xcassets/
│   └── swift-plugins/          — Native Swift plugins
│       └── AudioSessionPlugin/
└── android/                ← Android Tauri app
    ├── Cargo.toml
    ├── tauri.conf.json
    ├── build.rs
    ├── capabilities/default.json
    ├── src/
    │   ├── main.rs
    │   ├── lib.rs              — Mobile entry point (uses shared commands)
    │   └── commands.rs         — Android-only commands (get_device_info)
    ├── gen/android/            — Gradle project scaffold
    │   ├── build.gradle.kts
    │   ├── settings.gradle.kts
    │   └── app/
    │       ├── build.gradle.kts
    │       └── src/main/
    │           ├── AndroidManifest.xml
    │           └── kotlin/com/songbird/app/
    └── kotlin-plugins/         — Native Kotlin plugins
        └── AudioSessionPlugin/

​
Getting Started

​
Prerequisites

iOS (requires macOS):
  • Xcode 16+ with iOS 16+ SDK
  • Rust with aarch64-apple-ios target: rustup target add aarch64-apple-ios
  • Tauri CLI: cargo install tauri-cli
  • CocoaPods (if using Swift plugins): gem install cocoapods
Android (macOS, Linux, or Windows):
  • Android Studio with SDK 26+ (Android 8.0 Oreo)
  • Android NDK (installed via Android Studio SDK Manager)
  • Rust with Android targets: 
    rustup target add aarch64-linux-android  # ARM64 devices
rustup target add x86_64-linux-android   # Emulator
  • Tauri CLI: cargo install tauri-cli
  • Set ANDROID_HOME and NDK_HOME environment variables

​
Build & Run

# First time: build the React UI
cd react_ui && npm install && npm run build && cd ..

# iOS — build via utils script
./utils/build-ios.sh              # debug build
./utils/build-ios.sh --release    # release build
./utils/build-ios.sh --dev        # dev server + hot-reload

# Android — build via utils script
./utils/build-android.sh          # debug build (APK)
./utils/build-android.sh --release # release build (AAB + APK)
./utils/build-android.sh --dev    # dev server + hot-reload

# Or manually:
cd mobile/ios && cargo tauri ios dev      # iOS simulator
cd mobile/android && cargo tauri android dev  # Android emulator

​
Desktop Development

The mobile app crates can also be compiled and run on the desktop host for rapid iteration without a simulator: 
cd mobile/ios   # or mobile/android
cargo run       # Runs as a desktop Tauri window
This uses the same Tauri WebView but without the native mobile plugins. Mobile-specific commands (haptics, audio session) return placeholder values when running on desktop.

​
Mobile-Specific IPC Commands

These commands extend the shared desktop command set with mobile-only functionality:
CommandDescriptionPlatform
get_device_infoScreen metrics, audio capabilities, thermal stateBoth
configure_audio_sessionSet audio category (playback/record), sample rate, buffer sizeBoth
trigger_hapticFire haptic feedback (light/medium/heavy/selection/success)Both
All shared commands (transport, mixer, state, tempo, audio session, haptics) live in songbird-mobile-shared::commands and are used by both iOS and Android apps. Only get_device_info is platform-specific (returns different defaults per platform).

​
Audio Session Lifecycle

​
iOS (AVAudioSession)

App Launch → configure(PlayAndRecord, 44.1kHz, 5ms)
  → setCategory(.playAndRecord)
  → setPreferredSampleRate(44100)
  → setPreferredIOBufferDuration(0.005)
  → setActive(true)

Phone Call → handleInterruption(.began) → pause transport
          → handleInterruption(.ended, shouldResume: true) → resume

Headphones Unplugged → handleRouteChange(.oldDeviceUnavailable) → pause

Background → UIBackgroundMode: audio → engine continues

​
Android (AudioManager)

App Launch → requestAudioFocus(AUDIOFOCUS_GAIN)
  → AudioAttributes(USAGE_MEDIA, CONTENT_TYPE_MUSIC)
  → cpal opens AAudio/Oboe stream

Phone Call → onAudioFocusChange(AUDIOFOCUS_LOSS_TRANSIENT) → pause
           → onAudioFocusChange(AUDIOFOCUS_GAIN) → resume

Background → ForegroundService(mediaPlayback) → engine continues

​
Workspace & Dependency Graph

The mobile crates live in their own Cargo workspace (mobile/Cargo.toml) that references the core rust/ crates via path dependencies. This keeps a single Cargo.lock for mobile while sharing the same engine code: 
mobile/Cargo.toml (workspace)
├── mobile/shared ──→ rust/crates/engine/songbird-engine
│                  ──→ rust/crates/songbird-state
├── mobile/ios ────→ mobile/shared (shared commands + types)
└── mobile/android ─→ mobile/shared (shared commands + types)

​
Roadmap

​
Phase 1: Foundation (this PR)

  • Mobile workspace structure with iOS + Android Tauri apps
  • Shared mobile crate (audio session, haptics, platform, IPC commands)
  • Shared IPC commands — no duplication between iOS and Android
  • iOS Xcode project scaffold + Swift AudioSessionPlugin
  • Android Gradle project scaffold + Kotlin AudioSessionPlugin
  • Build scripts: utils/build-ios.sh, utils/build-android.sh

​
Phase 2: Audio Engine Integration

  • cpal audio output on iOS (CoreAudio via AVAudioSession)
  • cpal audio output on Android (AAudio/Oboe)
  • Background audio playback (iOS UIBackgroundMode, Android ForegroundService)
  • Audio interruption handling (phone calls, Siri, alarms)
  • MIDI device support (CoreMIDI on iOS, android.media.midi on Android)

​
Phase 3: Touch UI Adaptations

  • Responsive layout: phone vs tablet detection
  • Touch-optimized piano roll (larger hit targets, pinch-to-zoom)
  • Touch-optimized mixer (swipe between tracks, gesture faders)
  • Safe area insets for notch/home indicator/nav bar
  • Haptic feedback integration for note entry and controls

​
Phase 4: Platform Features

  • AUv3 plugin hosting on iOS (load third-party Audio Unit plugins)
  • Inter-app audio on iOS
  • Files app integration (iOS) / SAF integration (Android)
  • .bird file type association (open Songbird files from Files/file manager)
  • Share sheet support (export stems, share projects)

​
Phase 5: Release

  • App Store submission (iOS)
  • Play Store submission (Android)
  • App icon and launch screen assets
  • In-app purchase integration (if applicable)