Development
The primary workflow uses Docker for C++/CUDA builds and Node UI checks. Host dependencies are Docker/Buildx, NVIDIA GPU support, Python 3.11+ and Task. Python helpers use the standard library; Python is not added to the inference image.
Repository layout
| Directory | Purpose |
|---|---|
src/ |
Native inference engine; src/cli/ contains executable entry points |
web/ |
Svelte 5/TypeScript studio and embedded UI asset |
scripts/ |
Cross-platform Task helpers and advanced model conversion tools |
cmake/ |
Build identity, asset embedding and reusable GGML package helpers |
docker/ |
Pinned model-download manifest and compiled dependency configuration |
tests/benchmarks/ |
Repeatable generation runner |
benchmarks/ |
Resident-model speed, memory and ASR lyric suite; corpus and public metric reports |
tests/unit/ |
CPU-only tooling and lifecycle tests |
tests/planning/ |
Model-free generated-score checks, retry policy and request contracts |
tests/integration/ |
Read-only checks against a running server |
tests/fixtures/ |
Baseline prompts |
tests/parity/ |
Optional model-dependent upstream numerical harnesses and historical logs |
vendor/ |
Retained upstream license notices |
docs/ |
Documentation index, user/developer guides and the Docker Hub overview |
assets/ |
Artwork conversion notes and documentation screenshots |
site/ |
Independent Pages builder and shared Apache-2.0 documentation theme |
examples/ |
Public song gallery, generated recordings, original prompts and provenance |
Model files and generated data/results/ stay outside version control. Native
dependencies are fetched from checksum-pinned upstream archives or imported from
verified build artifacts; no recursive clone or nested Git checkout is needed.
src/ contains application code. cmake/ and docker/ describe how that code
is built and packaged, so they stay beside it. The root CMakeLists.txt,
Dockerfile and Taskfile.yml remain the build entry points; .github/workflows/
is GitHub’s workflow location. LICENSE, THIRD_PARTY_NOTICES.md, VERSION
and the README remain easy to find at the repository root.
build/ holds disposable compiler output, dependency downloads and publication
reports. User files belong in data/, separately from those build caches.
Both directories are ignored by Git and excluded from Docker’s build context.
Product artwork belongs with the UI assets or documentation media, never in
the local model-data directories.
Preview the project website with task pages, validate with
task pages-test, and stop its container with task pages-stop. Public gallery
recordings are intentional versioned examples; personal generations still belong
in the browser library or ignored data/results/. Gallery regeneration is an
explicit task examples-generate action and requires the local GPU.
The studio, website, README and documentation introductions support six interface languages. See localization when changing visible text, accessible labels or settings help. User lyrics and API values keep their original content in every locale.
task examples-convert verifies or refreshes the gallery’s single Ogg Opus export
per song without a GPU. Verification rejects duplicate public audio exports.
task audio-convert -- input.wav output.flac --format flac uses the shared
FFmpeg profiles for local exports. New demo PCM masters stay in ignored
data/results/gallery-masters/; public compressed copies and provenance live
in examples/. See the format and generation guide.
Local data
| Host location | Purpose | Container location |
|---|---|---|
data/models/ |
Required and optional GGUF files | /models, read-only |
data/adapters/ |
Optional LoRA adapters | /adapters, read-only |
data/checkpoints/ |
Original checkpoints for advanced conversion/reference checks | Not mounted by normal startup |
data/results/ |
Benchmark evidence, build-dependency exports and server results in studio/ |
Studio /output bind and per-run output mounts |
Tasks create these directories when needed. The tracked download manifest is
docker/models.json; model files are never tracked or
baked into the image. Browser songs/settings still live in browser storage,
and completed server results live on /output with bounded retention. Only
active/queued work and small metadata stay in server memory. See queue/storage.
For an existing checkout with root-level models/, adapters/, checkpoints/
or results/, finish active work before migrating:
task stop
task data:migrate
task start
task data:status shows paths and pending moves. Migration renames local
directories without re-downloading files, refuses destination collisions and
live bind mounts, and never merges or overwrites existing data. If both old
and new locations contain files, reconcile them manually before retrying.
The downloader refuses legacy storage rather than downloading a second copy.
Existing external Docker deployments can keep their own host directories;
the in-container paths and server defaults have not changed.
Task interface
| Command | Behavior |
|---|---|
| task doctor | Inspect Docker, Buildx and host GPU visibility |
| task models | Verify/download pinned external models |
| task data:status / data:migrate | Inspect storage or migrate an old checkout |
| task init | Models, then CUDA image build |
| task build | Rebuild current native engine and UI |
| task deps:build | Prebuild a reusable GGML artifact with all distribution CUDA targets |
| task deps:plan | Show the dependency fingerprint and pinned release artifact |
| task deps:prepare | Download and verify the public dependency archive |
| task deps:export | Export a local dependency archive and checksum for release upload |
| task start | Start the default server or replace an older project image |
| task stop / status / logs | Operate this project’s container |
| task test | Tooling/benchmark tests and native dependency pin verification |
| task test:api | Read-only capabilities and GPU telemetry checks against the running server |
| task test:mcp-agent – –model-url URL | Inspect full-song MCP calls without generation; see tests/agents/README.md |
| task check:cpu | Compile all native targets and check executable loading in Docker |
| task web:check | Check UI types, request behavior and formatting in Node Docker |
| task web:build | Regenerate the embedded UI asset in Node Docker |
| task assets:build | Convert the original PNG artwork to WebP with host FFmpeg |
| task validate | Tooling, vendor, UI and CPU checks |
| task test1 | Full GPU generation baseline |
| task benchmark-plan | Offline resident-model workload and fixed settings |
| task benchmark-baseline | Cold/warm speed, sampled VRAM and four-language lyric scoring |
| task benchmark-long | Five English long songs, warm inference/VRAM and no ASR |
| task benchmark-compare | Matched before/after regression gates |
| task test1:prepare | Prepare the request without Docker inference |
Task configuration is in Taskfile.yml; implementation is in scripts/dev.py. Defaults match .env.example. CLI variables override local settings. Only the named project container is replaced. Start refuses replacement while a known job is running and does not stop another project’s container with that name. Changing images restarts the process and cancels active/queued work. Completed results persist on the output volume until their original expiry; models and browser data persist. Queue and retention environment variables are preserved across Task image updates.
Local builds reuse the pinned public release dependency when targeting is automatic. Otherwise they detect host NVIDIA compute capability and compile for that hardware. Set CUDA_ARCHITECTURES explicitly for a different build. Docker builds have no GPU access, so do not pass the CMake value native to Docker builds. The Dockerfile’s broader architecture default is for distribution builds and has not been qualified as a support matrix.
Build stages and caching
Dockerfile stages:
- web: Node 24 checks and builds the current Svelte source.
- source: selects source files; excludes models and local state.
- cpu-checks: compiles all native targets, checks startup, and exercises score validation/retries, codecs, APIs and queue/storage without weights.
- cuda-toolchain: pinned CUDA 12.9.1 development base and build tools.
- ggml-deps-build / ggml-deps: isolated GGML compilation and portable dependency artifact.
- selected-ggml: verified release named context, source artifact or selected prebuilt image.
- app-deps: cached upstream HTTP/JSON libraries and MP3 decoder headers.
- build: C++20 app compilation linked against the verified dependency installations.
- weights-check: verifies the assembled image contains no common model-weight files.
- runtime: CUDA runtime, native executables, notices and embedded UI.
Compiled GGML libraries live in a normal Docker layer, separate from app source, UI and revision inputs. This layer survives external cache export; BuildKit cache mounts alone are not preserved by GitHub Actions caching. Local cache mounts also retain app objects and compiler results. No cache or artifact contains model weights. Normal Task/CI builds use the pinned public release archive as a named context. See build dependencies for downloads, source builds and invalidation rules.
VERSION owns the application version. Task builds inject the Git revision; CI injects the validated commit. The same version/revision is embedded in UI and native binaries. Direct Docker builds without a SOURCE_REVISION argument show an unknown revision, but retain the application version.
Application tools and parity harnesses require C++20. Source-built dependencies are configured with C++17 to match the precompiled GGML and HTTP/JSON packages. This uses the existing GCC 13/CUDA 12.9.1 toolchain and GGML artifact; changing the application language standard does not rebuild CUDA kernels.
GGML dependency
docker/ggml-source.json pins official GGML 0.28.0 by exact commit and archive
SHA-256. cmake/ggml-source.cmake uses CMake FetchContent to verify and unpack
that archive inside the build directory. The repository contains no GGML source
tree or submodule. One checksum-verified CUDA shared-mask patch is applied during
source builds; see the dependency recipe. Other local settings
are in cmake/ggml-abi.cmake and cmake/ggml-deps/.
Normal CUDA builds use installed headers and libraries from the precompiled
artifact and do not download the source archive. See build dependencies
for publication, caching and rollback. The MIT notice remains in
vendor/licenses/ggml-MIT.txt and the runtime image.
Run task deps:verify after pin changes. task vendor:verify remains a compatibility
alias. Dependency upgrades require CPU/CUDA validation and test1 generation;
the source URL never follows a moving branch or latest-release endpoint.
Other native dependencies
FFmpeg is installed as a runtime executable in Docker, with no linking into the inference engine or changes to the GGML artifact. Native users need FFmpeg on PATH. See audio integration and codec checks.
HTTP (cpp-httplib), JSON (yyjson) and MP3 decoding (minimp3) use the pinned upstream
sources in docker/app-deps.json. CMake fetches them into build storage; Docker
caches their installation in the app-deps stage. No external library sources
remain checked in. The inherited encoder remains project code in src/codecs/mp3/.
See dependency configuration and caching.
Native builds (advanced)
The Docker runtime is the supported development baseline. The original engine also contains native CPU, CUDA, Vulkan, Metal and SYCL paths. They are not all qualified as product distributions.
For CPU development with CMake 3.19+, Python 3, Ninja and a C++20 toolchain:
cmake -S . -B build/native-cpu -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=OFF -DGGML_VULKAN=OFF
cmake --build build/native-cpu --parallel 4
python scripts/smoke_binaries.py build/native-cpu
Use a Visual Studio x64 developer environment on Windows. For native CUDA, use a separate build/native-cuda directory with -DGGML_CUDA=ON and a suitable CUDA architecture. For Vulkan, install its SDK, glslc and SPIRV-Headers. The first source configure requires HTTPS access to the pinned archive; subsequent configures reuse the downloaded source in that build directory. No build task recursively deletes build/.
Reference parity checks
tests/parity contains the inherited Torch/NumPy/model-based comparisons.
Builds compile their native harnesses, but startup checks are not numerical parity.
The Python scripts default to build/native-cpu, data/models/ and
data/checkpoints/.
Some require a sibling YuE source checkout. Run from tests/parity on Linux.
Historical logs are in reference-logs; they are not evidence for the current image.
Model tooling
scripts/models retains advanced checkpoint download, conversion and quantization. These are secondary to inference, not training support. See MODELS.md. tests/benchmarks/test1.py and tests/fixtures/test1.txt define the repeatable baseline. Generated artifacts stay under ignored data/results/.