From 2cf58aaa5cb102b7630f3dbee809d2d8f880d6c7 Mon Sep 17 00:00:00 2001 From: Michele Rossi Date: Mon, 6 Jul 2026 20:21:56 +0200 Subject: [PATCH] Move TECH_SPEC.md into project root --- TECH_SPEC.md | 257 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 257 insertions(+) create mode 100644 TECH_SPEC.md diff --git a/TECH_SPEC.md b/TECH_SPEC.md new file mode 100644 index 0000000..c8c0265 --- /dev/null +++ b/TECH_SPEC.md @@ -0,0 +1,257 @@ +# VNAsset — Visual Novel Asset Pipeline CLI + +## Problem + +ComfyUI's node-graph execution is too slow for batch visual novel asset production: +- Each run pays Python dispatch overhead across ~15 nodes +- Models may unload/reload between runs due to VRAM management +- The two-step flow (generate base sprite → edit variants) requires manual intervention and separate workflow runs +- No batch orientation: each sprite variant is a full cold or warm restart + +## Hardware Target + +**AMD Ryzen AI Max 395 Pro** (Strix Halo) +- Integrated RDNA 3.5 GPU, unified memory architecture +- 128 GB shared RAM → VRAM is effectively unlimited (up to ~96 GB allocatable to GPU) +- ROCm/HIP backend +- Bottleneck is **GPU compute throughput**, not memory capacity +- Implications: model offloading is pointless, everything stays resident; optimizations should target **fewer FLOPs**, not fewer bytes + +## Core Design Principles + +1. **Models stay hot.** All models loaded once at startup, held in VRAM for the session lifetime. +2. **No node graph.** Direct PyTorch forward calls. No serialization, no dispatch loop, no intermediate tensor copies between "nodes." +3. **Generate-then-edit as first-class pipeline.** The tool knows that output of generation feeds into editing. Optional human-review gate between stages. +4. **Batch-native.** Accept lists of prompts/variants and process them in one warm session. +5. **CLI-first.** Single binary, YAML/JSON config files, stdin/stdout where useful. +6. **AMD-first.** ROCm is the primary backend. No CUDA-only paths. `torch.compile` where ROCm supports it. + +## Pipeline Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ VNAsset Session │ +│ │ +│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ +│ │ SDXL │ │ Qwen │ │ Qwen VL │ │ VAE(s) │ │ +│ │ UNet │ │ Edit UNet│ │ 7B TE │ │ │ │ +│ │ (~3.5GB) │ │ (~5GB) │ │ (~14GB) │ │ (~1GB) │ │ +│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ +│ │ │ │ │ │ +│ ▼ │ │ │ │ +│ ┌─────────┐ │ │ │ │ +│ │ Generate│ │ │ │ │ +│ │ Phase │──────────┼───────────────┼──────────────┤ │ +│ │ │ base.png │ │ │ │ +│ └─────────┘ │ │ │ │ +│ │ ▼ ▼ ▼ │ +│ │ ┌──────────────────────────────────────┐ │ +│ └──────────► Edit Phase │ │ +│ │ base.png + prompts[] → variants[] │ │ +│ └──────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌──────────────────┐ │ +│ │ Output Manager │ │ +│ │ base.png │ │ +│ │ happy.png │ │ +│ │ sad.png │ │ +│ │ angry.png │ │ +│ └──────────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +## CLI Interface + +### Generate (SDXL text-to-image) + +``` +vnasset generate \ + --checkpoint ./models/novaAnimeXL_ilV190.safetensors \ + --prompt "1girl, solo, red hair, glasses, blue eyes..." \ + --negative-prompt "deformed, ugly, bad quality..." \ + --width 1024 --height 1024 \ + --steps 20 --cfg 4.5 \ + --sampler euler --scheduler simple \ + --seed 573523050 \ + --output ./output/character_base.png +``` + +### Edit (Qwen image-to-image) + +``` +vnasset edit \ + --model ./models/qwen_image_edit_2509_fp8_e4m3fn.safetensors \ + --clip ./models/qwen_2.5_vl_7b_fp8_scaled.safetensors \ + --vae ./models/qwen_image_vae.safetensors \ + --lora ./models/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors \ + --input ./output/character_base.png \ + --prompt "make her smile happily" \ + --turbo \ + --output ./output/character_happy.png +``` + +### Batch Pipeline (generate + multiple edits) + +``` +vnasset pipeline --config pipeline.yaml +``` + +Where `pipeline.yaml`: + +```yaml +models: + sdxl_checkpoint: ./models/novaAnimeXL_ilV190.safetensors + edit_unet: ./models/qwen_image_edit_2509_fp8_e4m3fn.safetensors + edit_clip: ./models/qwen_2.5_vl_7b_fp8_scaled.safetensors + edit_vae: ./models/qwen_image_vae.safetensors + edit_lora: ./models/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors + +generate: + prompt: "1girl, solo, red hair, glasses, blue eyes, standing..." + negative_prompt: "deformed, ugly, bad quality, lowres..." + width: 1024 + height: 1024 + steps: 20 + cfg: 4.5 + sampler: euler + scheduler: simple + seed: 573523050 + output: "character_base" + +edit: + turbo: true # use Lightning 4-step LoRA, CFG=1 + variants: + - name: happy + prompt: "make her smile happily" + - name: sad + prompt: "make her look sad, tears in her eyes" + - name: angry + prompt: "make her look angry, furrowed brows" + - name: surprised + prompt: "make her look surprised, eyes wide open" + - name: blush + prompt: "make her blush, embarrassed expression" +``` + +Output: `character_base.png`, `character_happy.png`, `character_sad.png`, etc. + +### Subcommands Summary + +| Command | Purpose | +|---------|---------| +| `vnasset generate` | SDXL text-to-image, one shot | +| `vnasset edit` | Qwen image edit, one shot | +| `vnasset pipeline` | Generate + batch edit from config file | +| `vnasset serve` | (future) lightweight HTTP API for integration | + +## Data Flow + +### Generate Phase +``` +prompt text ──► SDXL CLIP ──► conditioning ──┐ + ├──► SDXL UNet (euler, N steps) ──► latent ──► SDXL VAE ──► image +noise + empty_latent ─────────────────────────┘ +``` + +### Edit Phase +``` +input image ──► kontext_scale ──► Qwen VAE encode ──► latent ──┐ + │ +prompt text ──► Qwen VL 7B TE ──► conditioning (pos/neg) ──────┤ + ├──► Qwen Edit UNet (N steps, CFG) ──► latent ──► Qwen VAE decode ──► image +input image ──► kontext_scale ──► Qwen VL 7B TE (visual tokens)┘ +``` + +### Turbo vs Normal Mode +| Parameter | Normal | Turbo (Lightning LoRA) | +|-----------|--------|------------------------| +| Steps | 20 | 4 | +| CFG | 4.0 | 1.0 | +| LoRA | none | Lightning-4steps | +| Sampler | euler | euler | +| Scheduler | simple | simple | + +## Technology Stack + +| Layer | Choice | Rationale | +|-------|--------|-----------| +| Language | Python 3.12+ | Model ecosystem (transformers, diffusers, safetensors) is Python-only | +| ML Framework | PyTorch (ROCm) | Required for AMD GPU; HIP backend | +| Diffusion Backend | `diffusers` + custom pipelines | SDXL is standard; Qwen Image Edit needs a custom diffusers pipeline | +| Model Format | safetensors | Direct load from ComfyUI model files; no conversion | +| CLI Framework | `click` or `argparse` | Lightweight, no async needed | +| Config Format | YAML (PyYAML) | Readable; pipeline definitions are human-authored | +| Image I/O | Pillow + torchvision | Standard; PNG output at minimum | +| LoRA Loading | `peft` or manual merge | Weight-merging Lightning LoRA into UNet at load time | + +## Key Optimizations (AMD-specific) + +1. **Persistent model registry.** All models loaded at session init into a dict; references kept alive. On 128 GB unified memory this is free. + +2. **VAE overlap.** SDXL VAE decode can run on a separate CUDA/HIP stream while the Qwen pipeline begins encoding (if generating then editing). Marginal on iGPU but worth structuring for. + +3. **torch.compile on UNet.** The Qwen Edit UNet forward is called identically per variant (same spatial dims). Compile once, reuse. ROCm `torch.compile` support is improving; falls back gracefully if unavailable. + +4. **FP8 native.** Both the Qwen UNet and CLIP are already FP8 (`fp8_e4m3fn`). Load them as FP8, compute in FP8 where possible on RDNA 3.5 (WMMA instructions support FP8). No upcast overhead. + +5. **Lightning LoRA merge.** At model load time, merge the 4-step LoRA weights into the UNet base weights (simple linear addition). Avoids the extra `LoraLoaderModelOnly` forward overhead per step. The turbo/normal switch then becomes a steps+CFG change only. + +6. **Batch edit loop.** All edit variants share the same input latent. The VAE encode and VL text encode (visual branch) run once, not per variant. Only the text prompt encoding and UNet denoising repeat. + +## Shared State Between Variants (Edit Phase) + +For a batch of N variants from one input image: +- **Run once:** VAE encode (image → latent), VL visual token encoding +- **Run N times:** Text prompt encoding (short text → text tokens, cheap), UNet denoising (4 or 20 steps), VAE decode (latent → image) + +This means the 7B model is invoked once for visual encoding + N times for text encoding (but text encoding is a small fraction of the 7B model, essentially a CLIP-like forward). + +## Output Conventions + +- `{output_dir}/{base_name}.png` for the generated base sprite +- `{output_dir}/{base_name}_{variant}.png` for each variant +- Metadata saved alongside: `{output_dir}/{base_name}.json` with prompt, seed, model hashes, timings +- Flat green background left intact; post-processing (trimming, transparency) is out of scope for v1 + +## Phased Roadmap + +### Phase 1 — Core CLI (MVP) +- [ ] `vnasset generate` with SDXL checkpoint +- [ ] `vnasset edit` with Qwen Image Edit (normal mode, no LoRA) +- [ ] Model loading from ComfyUI safetensors paths +- [ ] Single-image output +- [ ] ROCm PyTorch verified working + +### Phase 2 — Batch & Turbo +- [ ] `vnasset pipeline` YAML config +- [ ] Lightning 4-step LoRA merge-at-load +- [ ] Turbo mode (steps=4, CFG=1) +- [ ] Shared encode optimization (one VAE encode, N UNet runs) +- [ ] `--seed` and seed randomization +- [ ] Output metadata JSON + +### Phase 3 — Polish +- [ ] `torch.compile` UNet forward +- [ ] Progress bars (tqdm) +- [ ] Timing reports (wall time per phase) +- [ ] Checkpoint compatibility: any SDXL safetensors, any Qwen Image Edit safetensors +- [ ] Resolution from CLI/config (preserving aspect ratio through the pipeline) + +### Phase 4 — Future +- [ ] `vnasset serve` lightweight HTTP API +- [ ] Background removal integration (green-screen trim) +- [ ] Multi-character generation (multiple SDXL prompts in one session) +- [ ] Upscaling pass (optional) + +## Open Questions + +1. **SDXL CLIP encoding:** The current workflow uses `BNK_CLIPTextEncodeAdvanced` with `token_normalization=none`, `weight_interpretation=comfy`. Is this identical to standard SDXL CLIP encoding, or does the BNK node do something special? Needs verification. + +2. **Qwen VAE vs SDXL VAE:** The SDXL checkpoint bundles its own VAE. The Qwen pipeline uses `qwen_image_vae`. These are distinct VAEs with different latent spaces. The pipeline must use the correct VAE per phase — no sharing. + +3. **Qwen 2.5 VL 7B text encoding:** The `TextEncodeQwenImageEditPlus` node is ComfyUI-specific. Porting this to raw HuggingFace `transformers` + `qwen-vl-utils` requires understanding exactly what tokenization/masking it does for edit-mode conditioning (positive + negative with image context). This is the riskiest port. + +4. **ROCm `torch.compile` status:** Needs testing on the user's specific ROCm version. May need `TORCH_COMPILE_DISABLE=1` initially. + +5. **FP8 compute path:** RDNA 3.5 supports FP8 via WMMA. PyTorch ROCm FP8 support is maturing. If FP8 compute isn't available, the FP8 weights should be upcast to BF16 at load time — still fits in 128 GB.