vnassets/README.md

# VNAsset

Fast CLI pipeline for visual novel image asset generation.

Drop-in replacement for the ComfyUI workflow loop: generate base character sprites
with SDXL, then batch-edit variants (expressions, outfits) with Qwen Image Edit —
all in one warm session, no node-graph overhead.

## Hardware

Built for **AMD Strix Halo** (Ryzen AI Max 395 Pro, Radeon 8060S, 128 GB unified
memory). Also works on discrete AMD GPUs with ROCm. NVIDIA support is untested
but should work if you swap the torch backend.

The 128 GB unified memory means VRAM is effectively unlimited (up to ~96 GB
allocatable to GPU). The bottleneck is **GPU compute throughput**, not memory
capacity — model offloading is pointless, everything stays resident.

## Install

```bash
git clone <repo> vnassets
cd vnassets

# Create venv (Python 3.12 required for ROCm torch compatibility)
python3.12 -m venv .venv
source .venv/bin/activate

# Install ROCm PyTorch (adjust index URL for your ROCm version)
pip install torch --index-url https://download.pytorch.org/whl/rocm7.2

# Install the rest
pip install -e .
```

### Models

Symlink your ComfyUI models into `models/`:

```bash
cd models
ln -s /path/to/ComfyUI/models/checkpoints/novaAnimeXL_ilV190.safetensors .
ln -s /path/to/ComfyUI/models/diffusion_models/qwen_image_edit_2509_fp8_e4m3fn.safetensors .
ln -s /path/to/ComfyUI/models/loras/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors .
```

The Qwen VAE and text encoder are downloaded automatically from HuggingFace Hub
on first use (`Qwen/Qwen-Image` and `Qwen/Qwen2.5-VL-7B-Instruct`). No symlinks
needed for those.

Or place the actual files there — the tool just reads whatever safetensors you
point it at.

## Usage

### Generate (SDXL text-to-image)

```bash
vnasset generate \
  --checkpoint models/novaAnimeXL_ilV190.safetensors \
  --prompt "1girl, solo, red hair, glasses, blue eyes, white crop top, standing, portrait" \
  --negative-prompt "deformed, ugly, bad quality, lowres" \
  --steps 20 \
  --seed 42 \
  --output output/character_base.png
```

| Option | Default | Description |
|--------|---------|-------------|
| `--checkpoint` | (required) | Path to SDXL `.safetensors` |
| `--prompt` | (required) | Positive prompt |
| `--negative-prompt` | `""` | Negative prompt |
| `--width` | `1024` | Image width |
| `--height` | `1024` | Image height |
| `--steps` | `20` | Inference steps |
| `--cfg` | `4.5` | CFG scale |
| `--seed` | `random` | RNG seed (integer or `random`) |
| `--output` | `output.png` | Output path |
| `--raw` | `false` | Disable Compel prompt weighting (fall back to plain diffusers encoding) |

### Edit (Qwen Image Edit)

```bash
vnasset edit \
  --model models/qwen_image_edit_2509_fp8_e4m3fn.safetensors \
  --input character_base.png \
  --prompt "make her smile happily" \
  --steps 4 --cfg 1.0 \
  --lora models/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors \
  --output character_happy.png
```

| Option | Default | Description |
|--------|---------|-------------|
| `--model` | (required) | Path to Qwen Image Edit `.safetensors` (FP8) |
| `--input` | (required) | Input image to edit |
| `--prompt` | (required) | Edit instruction |
| `--steps` | `20` | Inference steps (`4` with Lightning LoRA) |
| `--cfg` | `4.0` | CFG scale (`1.0` with Lightning LoRA) |
| `--seed` | `random` | RNG seed |
| `--lora` | (none) | Path to LoRA `.safetensors` |
| `--output` | `output.png` | Output path |

**Turbo mode:** Use the Lightning 4-step LoRA with `--steps 4 --cfg 1.0` to cut
inference time proportionally. The LoRA is fused into the transformer at load
time, so there is no per-step LoRA overhead.

### Output

Each generation produces:
- `{output}.png` — the image
- `{output}.json` — metadata (prompt, seed, model path, timing, resolution)

Directories in `--output` are created automatically.

### Session (persistent models)

For multi-call workflows, use `VnAssetsSession` to keep models loaded in GPU
memory between operations. Models are loaded eagerly at construction:

```python
from vnassets import VnAssetsSession

with VnAssetsSession(
    sdxl_checkpoint="models/novaAnimeXL.safetensors",
    edit_model="models/qwen_image_edit.safetensors",
    edit_lora="models/lightning-4steps.safetensors",
) as vna:
    vna.generate("1girl, red hair", output="base.png")
    vna.edit("base.png", "make her smile", output="happy.png")
    vna.edit("base.png", "make her sad", output="sad.png")
```

Either model can be omitted (`None`) for single-model sessions. Properties:
- `vna.has_sdxl` / `vna.has_qwen` — check which models are loaded
- `vna.load_time_s` — total session construction time
- `vna.close()` — manual cleanup (automatic with `with`)

The standalone `vnasset generate` and `vnasset edit` CLI commands are thin
wrappers around a one-shot session — same API, backwards compatible.

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                      VnAssetsSession                         │
│                                                              │
│  ┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐ │
│  │ SDXL     │   │ Qwen     │   │ Qwen VL  │   │ Qwen     │ │
│  │ UNet     │   │ Transf.  │   │ 7B TE    │   │ VAE      │ │
│  │ (~3.5GB) │   │ (~20GB)  │   │ (~14GB)  │   │ (~1GB)   │ │
│  └──────────┘   └──────────┘   └──────────┘   └──────────┘ │
│       │               │               │              │       │
│       ▼               │               │              │       │
│  ┌─────────┐          │               │              │       │
│  │ Generate│          │               │              │       │
│  │         │──────────┼───────────────┼──────────────┤       │
│  │         │  base.png │              │              │       │
│  └─────────┘          │               │              │       │
│       │               ▼               ▼              ▼       │
│       │          ┌──────────────────────────────────────┐    │
│       └──────────►         Edit Phase                    │    │
│                  │  base.png + prompts[] → variants[]    │    │
│                  └──────────────────────────────────────┘    │
│                                      │                       │
│                                      ▼                       │
│                           ┌──────────────────┐               │
│                           │   base.png       │               │
│                           │   happy.png      │               │
│                           │   sad.png        │               │
│                           │   angry.png      │               │
│                           └──────────────────┘               │
└─────────────────────────────────────────────────────────────┘
```

The Qwen transformer is loaded FP8 → BF16 at construction using
`init_empty_weights` + incremental conversion to keep peak memory manageable
(20B parameters: 20 GB FP8 on disk, ~40 GB BF16 at runtime). All models fit
comfortably in 128 GB unified memory — no offloading, no swapping.

SDXL and Qwen use **separate VAEs** with different latent spaces. The SDXL
checkpoint bundles its own VAE; Qwen uses `Qwen/Qwen-Image` VAE from
HuggingFace. Both coexist in the same session without conflict.

## Data Flow

### Generate Phase

```
prompt text ──► Compel (SDXL CLIPs) ──► conditioning ──┐
                                                        ├──► SDXL UNet (N steps) ──► latent ──► SDXL VAE decode ──► image
noise + latent ─────────────────────────────────────────┘
```

### Edit Phase

```
input image ──► Qwen VAE encode ──► latent ──┐
                                              │
prompt text ──► Qwen VL 7B TE ──► conditioning┤
                                              ├──► Qwen Transformer (N steps) ──► latent ──► Qwen VAE decode ──► image
input image ──► Qwen VL 7B TE ──► visual tok.┘
```

The text encoder handles both text conditioning and visual token encoding from
the input image.

### Turbo vs Normal Mode

| Parameter | Normal | Turbo (Lightning LoRA) |
|-----------|--------|------------------------|
| Steps | 20 | 4 |
| CFG | 4.0 | 1.0 |
| LoRA | none | Lightning-4steps (fused at load) |
| Sampler | Flow Match Euler | Flow Match Euler |

## Performance

Hardware: **AMD Strix Halo** (Ryzen AI Max 395 Pro, Radeon 8060S, 128 GB),
bfloat16, `novaAnimeXL_ilV190.safetensors`.

### Generate (SDXL)

| Resolution | Steps | Load (s) | Inference (s) | Total (s) |
|-----------|-------|----------|---------------|-----------|
| 1024×1024 | 20 | ~2.5 | ~29 | ~31 |

Per-step breakdown (1024×1024):
- Step 1 (warmup): ~1.3–1.6s
- Steps 2–20 (steady): ~1.0–1.3s each
- VAE decode included in final step

Compel prompt weighting adds ~0.4s encoding overhead — negligible.

### Edit (Qwen Image Edit, 4-step Lightning LoRA)

| Attention | Steps | Load (s) | Inference (s) | Total (s) |
|-----------|-------|----------|---------------|-----------|
| Matmul fallback | 4 | ~28 | ~87 | ~115 |
| Flash attention | 4 | ~31 | ~53 | ~84 |

Per-step breakdown (1024×1024, flash attention):
- Step 1: ~0.3s (prefill/encoding)
- Steps 2–4: ~1.6 → ~5.1 → ~7.0s (transformer + VAE)

Flash attention (`TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1`) cuts edit
inference by **1.63×**. SDXL generate is unaffected (uses its own attention
processor, not SDPA).

### Session (persistent models)

| Phase | Wall (s) | Notes |
|-------|----------|-------|
| Session load | ~28 | SDXL + Qwen transformer + VAE + TE + LoRA fuse |
| Generate | ~31 | SDXL 20-step, 1024×1024 |
| Edit (turbo) | ~87 / ~54 | Matmul / flash attention |
| **Total (2 ops)** | **~146 / ~118** | One session load amortized |

With a session, each additional edit saves one model-load round trip (~28s).
5 edits save 112s. Flash attention adds a further 1.6× multiplier on inference.

See [`docs/stats.md`](docs/stats.md) for detailed per-run breakdowns.

## Technical Notes

### bfloat16 required on RDNA 3.5

`float16` causes GPU kernel crashes (segfault) on the Radeon 8060S. The tool
uses `bfloat16` internally. This is transparent to the user.

### FP8 → BF16 conversion

The Qwen Image Edit model ships as FP8 (`fp8_e4m3fn`). It is converted to BF16
at load time using `init_empty_weights` + incremental tensor conversion to keep
peak memory manageable. RDNA 3.5 WMMA supports FP8 compute, but PyTorch ROCm FP8
support is not yet mature enough to compute in FP8 — upcast to BF16 is the safe
path and still fits in 128 GB.

### Attention backends

Two attention paths, selected automatically:

| Path | When | Performance |
|------|------|-------------|
| **Flash attention** | `TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1` set | 1.63× faster edits |
| **Matmul fallback** | Default (env var not set) | Stable, slower |

SDXL always uses a custom `AttnProcessor` (direct QKV matmul) regardless of the
env var — its attention path is separate from the Qwen SDPA dispatch. The flash
attention toggle only affects the Qwen transformer and text encoder.

```bash
# Enable flash attention for the session
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 vnasset edit ...
```

### ROCm torch version

Tested with `torch 2.11.0+rocm7.2`. Newer ROCm nightlies (2.13+, 2.14+) may
cause GPU crashes. If you encounter segfaults, try matching this version.

### LoRA loading

Lightning LoRA is loaded via `diffusers` native `load_lora_weights()` and fused
into the transformer with `fuse_lora()` at session construction. The fusion
takes ~1.5s and happens once — the turbo/normal switch is then just a
steps+CFG change with no per-step LoRA overhead.

## Prompt Syntax

VNAsset supports **ComfyUI-style prompt weighting** via the `compel` library.

### Weighting

| Syntax | Effect |
|--------|--------|
| `(word)` | Boost ×1.1 |
| `(word:1.5)` | Boost ×1.5 |
| `(word:0.6)` | De-emphasize ×0.6 |
| `[word]` | De-emphasize ×0.9 (shorthand) |
| `\(word\)` | Literal parentheses (escaped) |

```bash
vnasset generate \
  --checkpoint models/novaAnimeXL_ilV190.safetensors \
  --prompt "(masterpiece:1.2), 1girl, (red hair:1.3), blue eyes, [glasses]" \
  --negative-prompt "(bad quality, worst quality:1.4)" \
  --steps 20 --seed 42
```

### BREAK (condition chunking)

Split the prompt into independent conditioning chunks with `BREAK`:

```bash
vnasset generate \
  --prompt "1girl, red hair, standing BREAK blue sky, cherry blossoms" \
  --steps 20 --seed 42
```

Use `--raw` to bypass weighting and fall back to plain diffusers encoding.

## Current State

| Feature | Status |
|---------|--------|
| `vnasset generate` | ✅ Working |
| `vnasset edit` | ✅ Working |
| `VnAssetsSession` (persistent models) | ✅ Working |
| Compel prompt weighting + BREAK | ✅ Working |
| Lightning LoRA fuse-at-load | ✅ Working |
| Flash attention (experimental) | ✅ Working |
| Metadata JSON output | ✅ Working |
| `vnasset pipeline` (batch YAML config) | 🚧 Planned |
| `vnasset serve` (daemon/HTTP API) | 🚧 Planned |
| `torch.compile` on UNet | 🚧 Planned |
| Batch edit loop (shared VAE encode) | 🚧 Planned |

## Future Improvements

- **Pipeline batch mode** — `vnasset pipeline --config pipeline.yaml` for
  generate + multiple edits in one session from a YAML config file.
- **`torch.compile` on UNet** — the UNet forward is identical each step; ROCm's
  `torch.compile` support is maturing and could cut per-step time significantly.
- **Shared encode optimization** — for N edit variants of the same input image,
  run VAE encode and VL visual token encoding once, then only text-encode and
  denoise per variant.
- **Self-contained torch wheel** — bundle the known-working torch wheel file in
  the project (`wheels/torch-2.11.0+rocm7.2-cp312-cp312-linux_x86_64.whl`) so
  the install is reproducible without depending on PyTorch's nightly index
  availability or a ComfyUI installation.
- **`vnasset serve`** — lightweight daemon with Unix socket or HTTP API for
  integrating VNAsset into external tools.
- **Background removal** — green-screen trim and transparency pass for
  post-processing sprites.