jmp-games/vnassets
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge TECH_SPEC.md into README.md, fix all inconsistencies

Browse Source 
- Consolidate TECH_SPEC into README as single source of truth
- Remove TECH_SPEC.md (375-line README covers everything)
- Fix architecture diagram: Qwen transformer ~5GB → ~20GB (20B params)
- Remove unimplemented CLI flags from docs: --sampler, --scheduler, --clip,
  --vae, --turbo (these don't exist in the actual code)
- Replace 'FP8 native compute' claim with actual FP8→BF16 conversion
- Replace 'peft' with actual diffusers native LoRA loading
- Move aspirational optimizations (VAE overlap, shared encode) to Future
- Resolve stale Open Questions; keep torch.compile as Future item
- Move 'Persistent model session' and 'Qwen Image Edit' from Future to
  implemented status with documented API
- Update performance numbers with actual measured session+flash numbers
- Document flash attention path alongside matmul fallback
- Simplify data flow diagram, remove ComfyUI-only kontext_scale concept
- Remove dead doc links (sdxl-generation.md)
This commit is contained in:
Michele Rossi
2026-07-08 10:35:28 +02:00
parent 97ac841518
commit d33f3a4e34
2 changed files with 213 additions and 305 deletions
Download Patch File Download Diff File Expand all files Collapse all files

261
README.md
View File

@@ -12,6 +12,10 @@ Built for **AMD Strix Halo** (Ryzen AI Max 395 Pro, Radeon 8060S, 128 GB unified
memory). Also works on discrete AMD GPUs with ROCm. NVIDIA support is untested
but should work if you swap the torch backend.
The 128 GB unified memory means VRAM is effectively unlimited (up to ~96 GB
allocatable to GPU). The bottleneck is **GPU compute throughput**, not memory
capacity — model offloading is pointless, everything stays resident.
## Install
```bash
@@ -37,11 +41,13 @@ Symlink your ComfyUI models into `models/`:
cd models
ln -s /path/to/ComfyUI/models/checkpoints/novaAnimeXL_ilV190.safetensors .
ln -s /path/to/ComfyUI/models/diffusion_models/qwen_image_edit_2509_fp8_e4m3fn.safetensors .
ln -s /path/to/ComfyUI/models/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors .
ln -s /path/to/ComfyUI/models/vae/qwen_image_vae.safetensors .
ln -s /path/to/ComfyUI/models/loras/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors .
```
The Qwen VAE and text encoder are downloaded automatically from HuggingFace Hub
on first use (`Qwen/Qwen-Image` and `Qwen/Qwen2.5-VL-7B-Instruct`). No symlinks
needed for those.
Or place the actual files there — the tool just reads whatever safetensors you
point it at.
@@ -68,21 +74,10 @@ vnasset generate \
| `--height` | `1024` | Image height |
| `--steps` | `20` | Inference steps |
| `--cfg` | `4.5` | CFG scale |
| `--seed` | `0` | RNG seed (use `random` for random) |
| `--seed` | `random` | RNG seed (integer or `random`) |
| `--output` | `output.png` | Output path |
| `--raw` | `false` | Disable Compel prompt weighting (fall back to plain diffusers encoding) |
When `--seed` is `random`, a random seed is generated and recorded in the
metadata file.
### Output
Each generation produces:
- `{output}.png` — the image
- `{output}.json` — metadata (prompt, seed, model path, timing, resolution)
Directories in `--output` are created automatically.
### Edit (Qwen Image Edit)
```bash
@@ -97,7 +92,7 @@ vnasset edit \
| Option | Default | Description |
|--------|---------|-------------|
| `--model` | (required) | Path to Qwen Image Edit `.safetensors` |
| `--model` | (required) | Path to Qwen Image Edit `.safetensors` (FP8) |
| `--input` | (required) | Input image to edit |
| `--prompt` | (required) | Edit instruction |
| `--steps` | `20` | Inference steps (`4` with Lightning LoRA) |
@@ -106,36 +101,165 @@ vnasset edit \
| `--lora` | (none) | Path to LoRA `.safetensors` |
| `--output` | `output.png` | Output path |
## Current State
**Turbo mode:** Use the Lightning 4-step LoRA with `--steps 4 --cfg 1.0` to cut
inference time proportionally. The LoRA is fused into the transformer at load
time, so there is no per-step LoRA overhead.
| Command | Status |
|---------|--------|
| `vnasset generate` | ✅ Working |
| `vnasset edit` | ✅ Working |
| `vnasset pipeline` | 🚧 Planned |
### Output
Each generation produces:
- `{output}.png` — the image
- `{output}.json` — metadata (prompt, seed, model path, timing, resolution)
Directories in `--output` are created automatically.
### Session (persistent models)
For multi-call workflows, use `VnAssetsSession` to keep models loaded in GPU
memory between operations. Models are loaded eagerly at construction:
```python
from vnassets import VnAssetsSession
with VnAssetsSession(
sdxl_checkpoint="models/novaAnimeXL.safetensors",
edit_model="models/qwen_image_edit.safetensors",
edit_lora="models/lightning-4steps.safetensors",
) as vna:
vna.generate("1girl, red hair", output="base.png")
vna.edit("base.png", "make her smile", output="happy.png")
vna.edit("base.png", "make her sad", output="sad.png")
```
Either model can be omitted (`None`) for single-model sessions. Properties:
- `vna.has_sdxl` / `vna.has_qwen` — check which models are loaded
- `vna.load_time_s` — total session construction time
- `vna.close()` — manual cleanup (automatic with `with`)
The standalone `vnasset generate` and `vnasset edit` CLI commands are thin
wrappers around a one-shot session — same API, backwards compatible.
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ VnAssetsSession │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ SDXL │ │ Qwen │ │ Qwen VL │ │ Qwen │ │
│ │ UNet │ │ Transf. │ │ 7B TE │ │ VAE │ │
│ │ (~3.5GB) │ │ (~20GB) │ │ (~14GB) │ │ (~1GB) │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │ │ │ │ │
│ ▼ │ │ │ │
│ ┌─────────┐ │ │ │ │
│ │ Generate│ │ │ │ │
│ │ │──────────┼───────────────┼──────────────┤ │
│ │ │ base.png │ │ │ │
│ └─────────┘ │ │ │ │
│ │ ▼ ▼ ▼ │
│ │ ┌──────────────────────────────────────┐ │
│ └──────────► Edit Phase │ │
│ │ base.png + prompts[] → variants[] │ │
│ └──────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ base.png │ │
│ │ happy.png │ │
│ │ sad.png │ │
│ │ angry.png │ │
│ └──────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
The Qwen transformer is loaded FP8 → BF16 at construction using
`init_empty_weights` + incremental conversion to keep peak memory manageable
(20B parameters: 20 GB FP8 on disk, ~40 GB BF16 at runtime). All models fit
comfortably in 128 GB unified memory — no offloading, no swapping.
SDXL and Qwen use **separate VAEs** with different latent spaces. The SDXL
checkpoint bundles its own VAE; Qwen uses `Qwen/Qwen-Image` VAE from
HuggingFace. Both coexist in the same session without conflict.
## Data Flow
### Generate Phase
```
prompt text ──► Compel (SDXL CLIPs) ──► conditioning ──┐
├──► SDXL UNet (N steps) ──► latent ──► SDXL VAE decode ──► image
noise + latent ─────────────────────────────────────────┘
```
### Edit Phase
```
input image ──► Qwen VAE encode ──► latent ──┐
│
prompt text ──► Qwen VL 7B TE ──► conditioning┤
├──► Qwen Transformer (N steps) ──► latent ──► Qwen VAE decode ──► image
input image ──► Qwen VL 7B TE ──► visual tok.┘
```
The text encoder handles both text conditioning and visual token encoding from
the input image.
### Turbo vs Normal Mode
| Parameter | Normal | Turbo (Lightning LoRA) |
|-----------|--------|------------------------|
| Steps | 20 | 4 |
| CFG | 4.0 | 1.0 |
| LoRA | none | Lightning-4steps (fused at load) |
| Sampler | Flow Match Euler | Flow Match Euler |
## Performance
Hardware: **AMD Strix Halo** (Ryzen AI Max 395 Pro, Radeon 8060S, 128 GB),
bfloat16, `novaAnimeXL_ilV190.safetensors`.
### Generate (SDXL)
Radeon 8060S (Strix Halo iGPU), bfloat16, 1024×1024:
- ~1s per step
- ~20s for 20-step generation
| Resolution | Steps | Load (s) | Inference (s) | Total (s) |
|-----------|-------|----------|---------------|-----------|
| 1024×1024 | 20 | ~2.5 | ~29 | ~31 |
Model loading adds ~5s cold-start overhead.
Per-step breakdown (1024×1024):
- Step 1 (warmup): ~1.3–1.6s
- Steps 2–20 (steady): ~1.0–1.3s each
- VAE decode included in final step
### Edit (Qwen Image Edit)
Compel prompt weighting adds ~0.4s encoding overhead — negligible.
Radeon 8060S, bfloat16:
- ~120s per step at 512×512
- ~23s model loading
### Edit (Qwen Image Edit, 4-step Lightning LoRA)
Larger resolutions scale proportionally. The SDPA math fallback in the text
encoder's visual branch and the transformer's attention blocks is the main
bottleneck.
| Attention | Steps | Load (s) | Inference (s) | Total (s) |
|-----------|-------|----------|---------------|-----------|
| Matmul fallback | 4 | ~28 | ~87 | ~115 |
| Flash attention | 4 | ~31 | ~53 | ~84 |
**Turbo mode:** Use the Lightning 4-step LoRA with `--steps 4 --cfg 1.0` to
cut per-step time proportionally (4× fewer steps).
Per-step breakdown (1024×1024, flash attention):
- Step 1: ~0.3s (prefill/encoding)
- Steps 2–4: ~1.6 → ~5.1 → ~7.0s (transformer + VAE)
Flash attention (`TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1`) cuts edit
inference by **1.63×**. SDXL generate is unaffected (uses its own attention
processor, not SDPA).
### Session (persistent models)
| Phase | Wall (s) | Notes |
|-------|----------|-------|
| Session load | ~28 | SDXL + Qwen transformer + VAE + TE + LoRA fuse |
| Generate | ~31 | SDXL 20-step, 1024×1024 |
| Edit (turbo) | ~87 / ~54 | Matmul / flash attention |
| **Total (2 ops)** | **~146 / ~118** | One session load amortized |
With a session, each additional edit saves one model-load round trip (~28s).
5 edits save 112s. Flash attention adds a further 1.6× multiplier on inference.
See [`docs/stats.md`](docs/stats.md) for detailed per-run breakdowns.
## Technical Notes
@@ -144,21 +268,43 @@ cut per-step time proportionally (4× fewer steps).
`float16` causes GPU kernel crashes (segfault) on the Radeon 8060S. The tool
uses `bfloat16` internally. This is transparent to the user.
### Custom attention
### FP8 → BF16 conversion
The default PyTorch SDPA backends (flash attention, mem-efficient attention) are
unstable on this AMD GPU. VNAsset uses a simple matmul-based attention
implementation that avoids the SDPA dispatch entirely.
The Qwen Image Edit model ships as FP8 (`fp8_e4m3fn`). It is converted to BF16
at load time using `init_empty_weights` + incremental tensor conversion to keep
peak memory manageable. RDNA 3.5 WMMA supports FP8 compute, but PyTorch ROCm FP8
support is not yet mature enough to compute in FP8 — upcast to BF16 is the safe
path and still fits in 128 GB.
### Attention backends
Two attention paths, selected automatically:
| Path | When | Performance |
|------|------|-------------|
| **Flash attention** | `TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1` set | 1.63× faster edits |
| **Matmul fallback** | Default (env var not set) | Stable, slower |
SDXL always uses a custom `AttnProcessor` (direct QKV matmul) regardless of the
env var — its attention path is separate from the Qwen SDPA dispatch. The flash
attention toggle only affects the Qwen transformer and text encoder.
```bash
# Enable flash attention for the session
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 vnasset edit ...
```
### ROCm torch version
Tested with `torch 2.11.0+rocm7.2`. Newer ROCm nightlies (2.13+, 2.14+) may
cause GPU crashes. If you encounter segfaults, try matching this version.
## Documentation
### LoRA loading
- **[SDXL Generation](docs/sdxl-generation.md)** — checkpoint loading, attention patches, prompt encoding, and generation pipeline details
- **[ComfyUI Prompt Style Support](docs/comfyui-prompt-style.md)** — prompt weighting and BREAK syntax specification
Lightning LoRA is loaded via `diffusers` native `load_lora_weights()` and fused
into the transformer with `fuse_lora()` at session construction. The fusion
takes ~1.5s and happens once — the turbo/normal switch is then just a
steps+CFG change with no per-step LoRA overhead.
## Prompt Syntax
@@ -194,17 +340,36 @@ vnasset generate \
Use `--raw` to bypass weighting and fall back to plain diffusers encoding.
## Current State
| Feature | Status |
|---------|--------|
| `vnasset generate` | ✅ Working |
| `vnasset edit` | ✅ Working |
| `VnAssetsSession` (persistent models) | ✅ Working |
| Compel prompt weighting + BREAK | ✅ Working |
| Lightning LoRA fuse-at-load | ✅ Working |
| Flash attention (experimental) | ✅ Working |
| Metadata JSON output | ✅ Working |
| `vnasset pipeline` (batch YAML config) | 🚧 Planned |
| `vnasset serve` (daemon/HTTP API) | 🚧 Planned |
| `torch.compile` on UNet | 🚧 Planned |
| Batch edit loop (shared VAE encode) | 🚧 Planned |
## Future Improvements
- **Persistent model session** — keep models loaded between commands to
eliminate the ~5s cold-start overhead per generation. A `vnasset serve`
daemon or `vnasset batch` command.
- **Pipeline batch mode** — `vnasset pipeline --config pipeline.yaml` for
generate + multiple edits in one session from a YAML config file.
- **`torch.compile` on UNet** — the UNet forward is identical each step; ROCm's
`torch.compile` support is maturing and could cut per-step time in half.
`torch.compile` support is maturing and could cut per-step time significantly.
- **Shared encode optimization** — for N edit variants of the same input image,
run VAE encode and VL visual token encoding once, then only text-encode and
denoise per variant.
- **Self-contained torch wheel** — bundle the known-working torch wheel file in
the project (`wheels/torch-2.11.0+rocm7.2-cp312-cp312-linux_x86_64.whl`) so
the install is reproducible without depending on PyTorch's nightly index
availability or a ComfyUI installation.
- **Qwen Image Edit support** — `vnasset edit` and `vnasset pipeline` for
batch expression/outfit variant editing.
- **`vnasset serve`** — lightweight daemon with Unix socket or HTTP API for
integrating VNAsset into external tools.
- **Background removal** — green-screen trim and transparency pass for
post-processing sprites.

257
TECH_SPEC.md
View File

@@ -1,257 +0,0 @@
# VNAsset — Visual Novel Asset Pipeline CLI
## Problem
ComfyUI's node-graph execution is too slow for batch visual novel asset production:
- Each run pays Python dispatch overhead across ~15 nodes
- Models may unload/reload between runs due to VRAM management
- The two-step flow (generate base sprite → edit variants) requires manual intervention and separate workflow runs
- No batch orientation: each sprite variant is a full cold or warm restart
## Hardware Target
**AMD Ryzen AI Max 395 Pro** (Strix Halo)
- Integrated RDNA 3.5 GPU, unified memory architecture
- 128 GB shared RAM → VRAM is effectively unlimited (up to ~96 GB allocatable to GPU)
- ROCm/HIP backend
- Bottleneck is **GPU compute throughput**, not memory capacity
- Implications: model offloading is pointless, everything stays resident; optimizations should target **fewer FLOPs**, not fewer bytes
## Core Design Principles
1. **Models stay hot.** All models loaded once at startup, held in VRAM for the session lifetime.
2. **No node graph.** Direct PyTorch forward calls. No serialization, no dispatch loop, no intermediate tensor copies between "nodes."
3. **Generate-then-edit as first-class pipeline.** The tool knows that output of generation feeds into editing. Optional human-review gate between stages.
4. **Batch-native.** Accept lists of prompts/variants and process them in one warm session.
5. **CLI-first.** Single binary, YAML/JSON config files, stdin/stdout where useful.
6. **AMD-first.** ROCm is the primary backend. No CUDA-only paths. `torch.compile` where ROCm supports it.
## Pipeline Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ VNAsset Session │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ SDXL │ │ Qwen │ │ Qwen VL │ │ VAE(s) │ │
│ │ UNet │ │ Edit UNet│ │ 7B TE │ │ │ │
│ │ (~3.5GB) │ │ (~5GB) │ │ (~14GB) │ │ (~1GB) │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │ │ │ │ │
│ ▼ │ │ │ │
│ ┌─────────┐ │ │ │ │
│ │ Generate│ │ │ │ │
│ │ Phase │──────────┼───────────────┼──────────────┤ │
│ │ │ base.png │ │ │ │
│ └─────────┘ │ │ │ │
│ │ ▼ ▼ ▼ │
│ │ ┌──────────────────────────────────────┐ │
│ └──────────► Edit Phase │ │
│ │ base.png + prompts[] → variants[] │ │
│ └──────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ Output Manager │ │
│ │ base.png │ │
│ │ happy.png │ │
│ │ sad.png │ │
│ │ angry.png │ │
│ └──────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
## CLI Interface
### Generate (SDXL text-to-image)
```
vnasset generate \
--checkpoint ./models/novaAnimeXL_ilV190.safetensors \
--prompt "1girl, solo, red hair, glasses, blue eyes..." \
--negative-prompt "deformed, ugly, bad quality..." \
--width 1024 --height 1024 \
--steps 20 --cfg 4.5 \
--sampler euler --scheduler simple \
--seed 573523050 \
--output ./output/character_base.png
```
### Edit (Qwen image-to-image)
```
vnasset edit \
--model ./models/qwen_image_edit_2509_fp8_e4m3fn.safetensors \
--clip ./models/qwen_2.5_vl_7b_fp8_scaled.safetensors \
--vae ./models/qwen_image_vae.safetensors \
--lora ./models/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors \
--input ./output/character_base.png \
--prompt "make her smile happily" \
--turbo \
--output ./output/character_happy.png
```
### Batch Pipeline (generate + multiple edits)
```
vnasset pipeline --config pipeline.yaml
```
Where `pipeline.yaml`:
```yaml
models:
sdxl_checkpoint: ./models/novaAnimeXL_ilV190.safetensors
edit_unet: ./models/qwen_image_edit_2509_fp8_e4m3fn.safetensors
edit_clip: ./models/qwen_2.5_vl_7b_fp8_scaled.safetensors
edit_vae: ./models/qwen_image_vae.safetensors
edit_lora: ./models/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors
generate:
prompt: "1girl, solo, red hair, glasses, blue eyes, standing..."
negative_prompt: "deformed, ugly, bad quality, lowres..."
width: 1024
height: 1024
steps: 20
cfg: 4.5
sampler: euler
scheduler: simple
seed: 573523050
output: "character_base"
edit:
turbo: true # use Lightning 4-step LoRA, CFG=1
variants:
- name: happy
prompt: "make her smile happily"
- name: sad
prompt: "make her look sad, tears in her eyes"
- name: angry
prompt: "make her look angry, furrowed brows"
- name: surprised
prompt: "make her look surprised, eyes wide open"
- name: blush
prompt: "make her blush, embarrassed expression"
```
Output: `character_base.png`, `character_happy.png`, `character_sad.png`, etc.
### Subcommands Summary
| Command | Purpose |
|---------|---------|
| `vnasset generate` | SDXL text-to-image, one shot |
| `vnasset edit` | Qwen image edit, one shot |
| `vnasset pipeline` | Generate + batch edit from config file |
| `vnasset serve` | (future) lightweight HTTP API for integration |
## Data Flow
### Generate Phase
```
prompt text ──► SDXL CLIP ──► conditioning ──┐
├──► SDXL UNet (euler, N steps) ──► latent ──► SDXL VAE ──► image
noise + empty_latent ─────────────────────────┘
```
### Edit Phase
```
input image ──► kontext_scale ──► Qwen VAE encode ──► latent ──┐
│
prompt text ──► Qwen VL 7B TE ──► conditioning (pos/neg) ──────┤
├──► Qwen Edit UNet (N steps, CFG) ──► latent ──► Qwen VAE decode ──► image
input image ──► kontext_scale ──► Qwen VL 7B TE (visual tokens)┘
```
### Turbo vs Normal Mode
| Parameter | Normal | Turbo (Lightning LoRA) |
|-----------|--------|------------------------|
| Steps | 20 | 4 |
| CFG | 4.0 | 1.0 |
| LoRA | none | Lightning-4steps |
| Sampler | euler | euler |
| Scheduler | simple | simple |
## Technology Stack
| Layer | Choice | Rationale |
|-------|--------|-----------|
| Language | Python 3.12+ | Model ecosystem (transformers, diffusers, safetensors) is Python-only |
| ML Framework | PyTorch (ROCm) | Required for AMD GPU; HIP backend |
| Diffusion Backend | `diffusers` + custom pipelines | SDXL is standard; Qwen Image Edit needs a custom diffusers pipeline |
| Model Format | safetensors | Direct load from ComfyUI model files; no conversion |
| CLI Framework | `click` or `argparse` | Lightweight, no async needed |
| Config Format | YAML (PyYAML) | Readable; pipeline definitions are human-authored |
| Image I/O | Pillow + torchvision | Standard; PNG output at minimum |
| LoRA Loading | `peft` or manual merge | Weight-merging Lightning LoRA into UNet at load time |
## Key Optimizations (AMD-specific)
1. **Persistent model registry.** All models loaded at session init into a dict; references kept alive. On 128 GB unified memory this is free.
2. **VAE overlap.** SDXL VAE decode can run on a separate CUDA/HIP stream while the Qwen pipeline begins encoding (if generating then editing). Marginal on iGPU but worth structuring for.
3. **torch.compile on UNet.** The Qwen Edit UNet forward is called identically per variant (same spatial dims). Compile once, reuse. ROCm `torch.compile` support is improving; falls back gracefully if unavailable.
4. **FP8 native.** Both the Qwen UNet and CLIP are already FP8 (`fp8_e4m3fn`). Load them as FP8, compute in FP8 where possible on RDNA 3.5 (WMMA instructions support FP8). No upcast overhead.
5. **Lightning LoRA merge.** At model load time, merge the 4-step LoRA weights into the UNet base weights (simple linear addition). Avoids the extra `LoraLoaderModelOnly` forward overhead per step. The turbo/normal switch then becomes a steps+CFG change only.
6. **Batch edit loop.** All edit variants share the same input latent. The VAE encode and VL text encode (visual branch) run once, not per variant. Only the text prompt encoding and UNet denoising repeat.
## Shared State Between Variants (Edit Phase)
For a batch of N variants from one input image:
- **Run once:** VAE encode (image → latent), VL visual token encoding
- **Run N times:** Text prompt encoding (short text → text tokens, cheap), UNet denoising (4 or 20 steps), VAE decode (latent → image)
This means the 7B model is invoked once for visual encoding + N times for text encoding (but text encoding is a small fraction of the 7B model, essentially a CLIP-like forward).
## Output Conventions
- `{output_dir}/{base_name}.png` for the generated base sprite
- `{output_dir}/{base_name}_{variant}.png` for each variant
- Metadata saved alongside: `{output_dir}/{base_name}.json` with prompt, seed, model hashes, timings
- Flat green background left intact; post-processing (trimming, transparency) is out of scope for v1
## Phased Roadmap
### Phase 1 — Core CLI (MVP)
- [ ] `vnasset generate` with SDXL checkpoint
- [ ] `vnasset edit` with Qwen Image Edit (normal mode, no LoRA)
- [ ] Model loading from ComfyUI safetensors paths
- [ ] Single-image output
- [ ] ROCm PyTorch verified working
### Phase 2 — Batch & Turbo
- [ ] `vnasset pipeline` YAML config
- [ ] Lightning 4-step LoRA merge-at-load
- [ ] Turbo mode (steps=4, CFG=1)
- [ ] Shared encode optimization (one VAE encode, N UNet runs)
- [ ] `--seed` and seed randomization
- [ ] Output metadata JSON
### Phase 3 — Polish
- [ ] `torch.compile` UNet forward
- [ ] Progress bars (tqdm)
- [ ] Timing reports (wall time per phase)
- [ ] Checkpoint compatibility: any SDXL safetensors, any Qwen Image Edit safetensors
- [ ] Resolution from CLI/config (preserving aspect ratio through the pipeline)
### Phase 4 — Future
- [ ] `vnasset serve` lightweight HTTP API
- [ ] Background removal integration (green-screen trim)
- [ ] Multi-character generation (multiple SDXL prompts in one session)
- [ ] Upscaling pass (optional)
## Open Questions
1. **SDXL CLIP encoding:** The current workflow uses `BNK_CLIPTextEncodeAdvanced` with `token_normalization=none`, `weight_interpretation=comfy`. Is this identical to standard SDXL CLIP encoding, or does the BNK node do something special? Needs verification.
2. **Qwen VAE vs SDXL VAE:** The SDXL checkpoint bundles its own VAE. The Qwen pipeline uses `qwen_image_vae`. These are distinct VAEs with different latent spaces. The pipeline must use the correct VAE per phase — no sharing.
3. **Qwen 2.5 VL 7B text encoding:** The `TextEncodeQwenImageEditPlus` node is ComfyUI-specific. Porting this to raw HuggingFace `transformers` + `qwen-vl-utils` requires understanding exactly what tokenization/masking it does for edit-mode conditioning (positive + negative with image context). This is the riskiest port.
4. **ROCm `torch.compile` status:** Needs testing on the user's specific ROCm version. May need `TORCH_COMPILE_DISABLE=1` initially.
5. **FP8 compute path:** RDNA 3.5 supports FP8 via WMMA. PyTorch ROCm FP8 support is maturing. If FP8 compute isn't available, the FP8 weights should be upcast to BF16 at load time — still fits in 128 GB.