​

WebGPU Rendering — Invariants

Governs the GPU rendering layer: the renderer entry point in react_ui/src/lib/gpu/GpuCanvas.tsx and the surfaces it hosts (arrangement/meshes/, midi-editor/meshes/, visualizer/scenes/, node-graph/GpuLayer.tsx). Read this before adding a new GPU surface or shader.

The rendering pipeline runs on WebGPU (primary) and WebGL2 (fallback). Three.js + TSL bridge the two; this spec defines the rules that keep that bridge intact.

---

​

Invariants

These are load-bearing. Violating one is a critical bug even if the WebGPU path looks fine.

​

1. Single renderer entry point

react_ui/src/lib/gpu/GpuCanvas.tsx is the only place that constructs a renderer. The renderer is always WebGPURenderer from three/webgpu, which transparently picks a WebGPUBackend or a WebGLBackend under the hood. All GPU scenes mount inside <GpuCanvas>.

• Why. The renderer factory enforces backend selection, async init, fallback logic, DPR setup, and color-space configuration in one place. Bypassing it means each surface re-derives those choices and they drift. Using a single renderer class (rather than the legacy WebGLRenderer for WebGL and WebGPURenderer for WebGPU) is what makes NodeMaterial and the auto-translation of legacy materials (MeshBasicMaterial → MeshBasicNodeMaterial) work uniformly on both paths.
• How to apply. New GPU surface → mount <GpuCanvas> and put the scene inside. Do not import WebGPURenderer or the legacy WebGLRenderer anywhere else.

​

2. Stay in the dual-backend TSL envelope

The features below have no WebGL2 equivalent. They cause silent failures on the fallback path.

• Compute shaders (Fn with compute(), .computeAsync())
• Storage buffers, storage textures
• Indirect draws
• Timestamp queries
• Why. Three.js will throw at material/pipeline compile, or the feature will no-op silently. Users on the fallback see broken UI with no clear error path.
• How to apply. If you need one of these, opt-in explicitly per surface (Pattern: WebGPU-only features). Keep a JS/CPU fallback alive for the WebGL2 path. Enforced by the lint guard.

​

3. All custom shaders use TSL, not GLSL/WGSL strings

NodeMaterial + TSL only. No ShaderMaterial, no RawShaderMaterial, no inline vertexShader: \…“ strings in new code.

• Why. TSL compiles to both WGSL and GLSL automatically. Raw shader source is one-backend-only and forces dual codepaths.
• How to apply. New shader → write it as TSL nodes from three/tsl. ClipInstances.tsx is the reference for SDF + per-clip attribute patterns; copy from there.

​

4. Pixel parity between backends

Both backends must produce visually-equivalent output: same colors, same anti-aliasing, same alpha compositing, same DPR crispness.

• Why. The fallback isn’t a “degraded mode” — it’s a real path some users live on (Linux WebKitGTK, older macOS, driver-blocklisted GPUs). Quality drift on either backend is a bug.
• How to apply. Every PR that touches a GPU surface file must be tested on both VITE_GPU_BACKEND=webgpu and VITE_GPU_BACKEND=webgl. CI has no GPU; this is a human-eyeball check.

​

5. Per-frame work stays on the GPU

Don’t rebuild BufferGeometry or upload large buffers per frame. Per-frame state goes through uniforms, instance attributes, or the camera/projection matrix — not full geometry rebuilds.

• Why. Geometry rebuilds are the WebGL2 stutter source and waste WebGPU performance. The point of GPU-side rendering is that the data lives there between frames.
• How to apply. Geometry useMemo deps change only on scene- shape changes (clip add/remove, edit). Scroll/zoom drives the camera. Playhead position is a uniform, not a vertex.

​

6. Frame loop is demand by default

Use frameloop="demand" on <GpuCanvas> and call invalidate() only when the scene changes. Use "always" only for genuinely continuous animation (visualizer, scrolling).

• Why. “Always” wastes battery and CPU for static scenes. The demand-driven loop is half the reason the GL port doesn’t melt laptops.
• How to apply. Default to demand. If you reach for always, ask whether you can drive the redraw from input events instead.

---

​

Patterns

​

Writing a new GPU surface

1. Mount under <GpuCanvas frameloop="demand|always" transparent>.
2. Build geometry inside useMemo, keyed on scene-shape inputs.
3. Use built-in <meshBasicMaterial> etc. when you can; reach for NodeMaterial + TSL only when you need custom shading.
4. Per-frame state goes through useFrame updates to uniforms or the camera, not geometry rebuilds.
5. Verify on both backends before merging.

​

Writing a custom shader

1. Build the material with NodeMaterial from three/webgpu.
2. Express vertex/fragment logic as TSL nodes:
   • Attributes: attribute('aFoo', 'vec2')
   • Uniforms: uniform(value) or uniform(value, 'float')
   • Math: add, sub, mul, length, dot, smoothstep, etc.
3. Set material.colorNode for the fragment color output.
4. Apply transparency / depth-test / side flags directly on the material.
5. Test side-by-side on both backends — colors and alpha compositing are the most common drift points.

​

WebGPU-only features (the opt-in path)

If you genuinely need compute, storage, etc.:

1. Detect the active backend via renderer.backend.type (or the env override).
2. On WebGPU, use the compute/storage path.
3. On WebGL, fall through to a pre-existing CPU/JS path that produces the same visual result. The CPU path is mandatory, not optional.
4. Document the opt-in here as a new Decision entry, dated, with the surface name and the perf justification.

​

Forbidden patterns (lint-guarded)

The following are blocked at validation time:

• new WebGLRenderer( or new WebGPURenderer( outside GpuCanvas.tsx
• new ShaderMaterial( — use NodeMaterial instead
• new RawShaderMaterial( — never
• compute(, .computeAsync(, storage(, storageObject( — opt-in required
• Inline vertexShader: / fragmentShader: strings in new code

Single-line opt-out: // WEBGPU-OPT-IN: <reason> allows the next line through the guard. Force the conversation; don’t bypass silently.

---

​

Decisions

​

D1 — Default backend is WebGPU with WebGL2 fallback (2026-05-06)

GpuCanvas always constructs a WebGPURenderer and lets it pick its backend: WebGPUBackend by default, WebGLBackend when forceWebGL: true is passed (set when the user picks webgl) or when our explicit fallback fires (see D5). Why. WebGPU has lower per-frame CPU overhead and unblocks compute-shader use cases (waveform mip generation, real-time spectral analysis). The WebGL2 fallback covers Linux WebKitGTK, older macOS, and driver-blocklisted GPUs. Using a single renderer class with a swappable backend (rather than two separate renderer classes) is what keeps NodeMaterial and TSL working on both paths. How to apply. Don’t assume one backend; respect the override layers (localStorage.GPU_BACKEND runtime, VITE_GPU_BACKEND build). Don’t construct the legacy WebGLRenderer — it doesn’t share the node pipeline.

​

D2 — CPU-side waveform polys, not compute mip generation (2026-05-06)

Waveform peak data is decimated in JS (react_ui/src/components/panels/arrangement/renderer/peakMipmap.ts, a power-of-2 mipmap of Float32Array peaks) and emitted per-frame as filled polygon triangles in WaveformMesh.tsx, rendered with a built-in material (no custom shader, no texture sampler). Why. Polygon-tris render identically on both backends with zero backend-specific code. GPU-side mip generation via compute would unlock denser-than-pixel detail, but needs a CPU fallback anyway and the current path isn’t a perf bottleneck. How to apply. Don’t migrate waveform mip generation to compute, and don’t move the peak data into a DataTexture + fragment-sampler pipeline, without an explicit opt-in (see Pattern: WebGPU-only features) and a real perf justification.

​

D3 — DOM overlay + CanvasTexture for text, not MSDF (2026-05-06)

Chrome text (panel headers, tooltips) lives in the DOM. In-scene text (ruler labels, badges) uses CanvasTexture per RulerLabelsMesh.tsx / BadgeMesh.tsx. MSDF is not adopted. Why. MSDF adds a font-atlas pipeline and a custom shader for a problem that doesn’t exist at current zoom levels. CanvasTexture works on both backends and matches CSS-rendered text. How to apply. New labels follow the existing pattern. Reach for MSDF only if a specific surface needs sharp scaling beyond what CanvasTexture delivers, and document the exception here.

​

D4 — Hit-testing stays CPU-side (2026-05-06)

Clip/note picking uses bounding-box math in react_ui/src/components/panels/arrangement/interaction/hitTest.ts, not GPU framebuffer readback. Why. Framebuffer readback stalls the GPU pipeline on both backends, and the math is cheap. WebGPU’s compute path could change this analysis later, but not yet. How to apply. New hit-test paths use the same CPU-side approach.

​

D5 — Explicit WebGL fallback, not just three’s getFallback (2026-05-08)

GpuCanvas.makeRenderer runs an explicit two-stage fallback before trusting WebGPURenderer’s built-in fallback path:

1. Adapter probe. Before construction, in auto/webgpu mode we call navigator.gpu?.requestAdapter(). A null result, a thrown exception, or missing navigator.gpu flips us to forceWebGL: true without ever attempting WebGPUBackend.
2. Init retry. Even with a valid adapter, renderer.init() is wrapped in try/catch. On failure we dispose the half-initialised renderer and construct a fresh WebGPURenderer({ forceWebGL: true }).

Why. Three’s internal getFallback does not catch every Linux WebKitGTK failure mode. The two states that bit users in testing:

• navigator.gpu exposed but requestAdapter() resolves null (WebKitGTK with WebGPU experimental flag off but the API binding shipped). Without the probe, this manifests as an unhandled rejection inside init() and a black canvas — the user has no recovery path.
• requestAdapter() succeeds, requestDevice() rejects (driver reports adapter, kernel module refuses device creation). Without the init-retry, same black canvas. With the retry, the user gets a working WebGL2 surface and a banner explaining the fallback.

The dev console banner reports the fallback reason (fellback: <reason>) so support diagnostics are self-serve — the string is the first thing to ask for when “the timeline is blank on Linux.” How to apply. If you change the renderer construction sequence, preserve both stages (probe + init-retry) and the banner reason string. The glCanvasFallback.test.ts smoke test exercises the probe’s four states (no API, null, throw, success); add cases there when you discover a new Linux failure mode. Per-platform init status. Filled in as platform builds are run against localStorage.GPU_BACKEND=webgpu: Anyone running on a fresh platform: replace the row with the observed result and the console banner string. Don’t leave untested in place once a platform has shipped.