23135e62cb
Adds `vnasset upscale` CLI command, `VnAssetsSession.upscale()` / `.upscales()` session methods, and a standalone `vnassets.upscale` module following the existing remove-bg pattern. Uses Real-ESRGAN RRDBNet with RealESRGAN_x4plus_anime_6B (~17 MB, auto-downloaded). A 256x256 warmup tile at load time eliminates ~80s of first-run CUDA JIT compilation on ROCm. Steady-state: ~1.8s per 1024->2048 upscale on Strix Halo. The upsampler is lazy-loaded on first call and coexists with SDXL/Qwen in the same session. Works around a basicsr/torchvision API incompatibility (rgb_to_grayscale moved in torchvision 0.20+) with a 3-line module shim in upscale.py.
502 lines
20 KiB
Markdown
502 lines
20 KiB
Markdown
# VNAsset
|
||
|
||
Fast CLI pipeline for visual novel image asset generation.
|
||
|
||
Drop-in replacement for the ComfyUI workflow loop: generate base character sprites
|
||
with SDXL, then batch-edit variants (expressions, outfits) with Qwen Image Edit —
|
||
all in one warm session, no node-graph overhead.
|
||
|
||
## Hardware
|
||
|
||
Built for **AMD Strix Halo** (Ryzen AI Max 395 Pro, Radeon 8060S, 128 GB unified
|
||
memory). Also works on discrete AMD GPUs with ROCm. NVIDIA support is untested
|
||
but should work if you swap the torch backend.
|
||
|
||
The 128 GB unified memory means VRAM is effectively unlimited (up to ~96 GB
|
||
allocatable to GPU). The bottleneck is **GPU compute throughput**, not memory
|
||
capacity — model offloading is pointless, everything stays resident.
|
||
|
||
## Install
|
||
|
||
```bash
|
||
git clone <repo> vnassets
|
||
cd vnassets
|
||
|
||
# Create venv (Python 3.12 required for ROCm torch compatibility)
|
||
python3.12 -m venv .venv
|
||
source .venv/bin/activate
|
||
|
||
# Install ROCm PyTorch (adjust index URL for your ROCm version)
|
||
pip install torch --index-url https://download.pytorch.org/whl/rocm7.2
|
||
|
||
# Install the rest
|
||
pip install -e .
|
||
```
|
||
|
||
### Models
|
||
|
||
Symlink your ComfyUI models into `models/`:
|
||
|
||
```bash
|
||
cd models
|
||
ln -s /path/to/ComfyUI/models/checkpoints/novaAnimeXL_ilV190.safetensors .
|
||
ln -s /path/to/ComfyUI/models/diffusion_models/qwen_image_edit_2509_fp8_e4m3fn.safetensors .
|
||
ln -s /path/to/ComfyUI/models/loras/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors .
|
||
```
|
||
|
||
The Qwen VAE and text encoder are downloaded automatically from HuggingFace Hub
|
||
on first use (`Qwen/Qwen-Image` and `Qwen/Qwen2.5-VL-7B-Instruct`). No symlinks
|
||
needed for those.
|
||
|
||
Or place the actual files there — the tool just reads whatever safetensors you
|
||
point it at.
|
||
|
||
## Usage
|
||
|
||
### Generate (SDXL text-to-image)
|
||
|
||
```bash
|
||
vnasset generate \
|
||
--checkpoint models/novaAnimeXL_ilV190.safetensors \
|
||
--prompt "1girl, solo, red hair, glasses, blue eyes, white crop top, standing, portrait" \
|
||
--negative-prompt "deformed, ugly, bad quality, lowres" \
|
||
--steps 20 \
|
||
--seed 42 \
|
||
--output output/character_base.png
|
||
```
|
||
|
||
| Option | Default | Description |
|
||
|--------|---------|-------------|
|
||
| `--checkpoint` | (required) | Path to SDXL `.safetensors` |
|
||
| `--prompt` | (required) | Positive prompt |
|
||
| `--negative-prompt` | `""` | Negative prompt |
|
||
| `--width` | `1024` | Image width |
|
||
| `--height` | `1024` | Image height |
|
||
| `--steps` | `20` | Inference steps |
|
||
| `--cfg` | `4.5` | CFG scale |
|
||
| `--seed` | `random` | RNG seed (integer or `random`) |
|
||
| `--output` | `output.png` | Output path |
|
||
| `--raw` | `false` | Disable Compel prompt weighting (fall back to plain diffusers encoding) |
|
||
|
||
### Edit (Qwen Image Edit)
|
||
|
||
```bash
|
||
vnasset edit \
|
||
--model models/qwen_image_edit_2509_fp8_e4m3fn.safetensors \
|
||
--input character_base.png \
|
||
--prompt "make her smile happily" \
|
||
--steps 4 --cfg 1.0 \
|
||
--lora models/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors \
|
||
--output character_happy.png
|
||
```
|
||
|
||
| Option | Default | Description |
|
||
|--------|---------|-------------|
|
||
| `--model` | (required) | Path to Qwen Image Edit `.safetensors` (FP8) |
|
||
| `--input` | (required) | Input image to edit |
|
||
| `--prompt` | (required) | Edit instruction |
|
||
| `--steps` | `20` | Inference steps (`4` with Lightning LoRA) |
|
||
| `--cfg` | `4.0` | CFG scale (`1.0` with Lightning LoRA) |
|
||
| `--seed` | `random` | RNG seed |
|
||
| `--lora` | (none) | Path to LoRA `.safetensors` |
|
||
| `--output` | `output.png` | Output path |
|
||
|
||
**Turbo mode:** Use the Lightning 4-step LoRA with `--steps 4 --cfg 1.0` to cut
|
||
inference time proportionally. The LoRA is fused into the transformer at load
|
||
time, so there is no per-step LoRA overhead.
|
||
|
||
### Output
|
||
|
||
Each generation produces:
|
||
- `{output}.png` — the image
|
||
- `{output}.json` — metadata (prompt, seed, model path, timing, resolution)
|
||
|
||
Directories in `--output` are created automatically.
|
||
|
||
### Session (persistent models)
|
||
|
||
For multi-call workflows, use `VnAssetsSession` to keep models loaded in GPU
|
||
memory between operations. Models are loaded eagerly at construction:
|
||
|
||
```python
|
||
from vnassets import VnAssetsSession
|
||
|
||
with VnAssetsSession(
|
||
sdxl_checkpoint="models/novaAnimeXL.safetensors",
|
||
edit_model="models/qwen_image_edit.safetensors",
|
||
edit_lora="models/lightning-4steps.safetensors",
|
||
) as vna:
|
||
vna.generate("1girl, red hair", output="base.png")
|
||
vna.edit("base.png", "make her smile", output="happy.png")
|
||
vna.edit("base.png", "make her sad", output="sad.png")
|
||
```
|
||
|
||
Either model can be omitted (`None`) for single-model sessions. Properties:
|
||
- `vna.has_sdxl` / `vna.has_qwen` — check which models are loaded
|
||
- `vna.load_time_s` — total session construction time
|
||
- `vna.close()` — manual cleanup (automatic with `with`)
|
||
|
||
The standalone `vnasset generate` and `vnasset edit` CLI commands are thin
|
||
wrappers around a one-shot session — same API, backwards compatible.
|
||
|
||
### Background Removal
|
||
|
||
Remove backgrounds from character sprites (output is RGBA PNG):
|
||
|
||
```bash
|
||
# Single file
|
||
vnasset remove-bg --input character_base.png --output character_transparent.png
|
||
|
||
# Batch (reuses model across files)
|
||
vnasset remove-bg --input base.png --input happy.png --input sad.png --output-dir transparent/
|
||
```
|
||
|
||
Or via the session API:
|
||
|
||
```python
|
||
with VnAssetsSession() as vna:
|
||
vna.remove_background("base.png", output="base_transparent.png")
|
||
|
||
# Batch
|
||
vna.remove_backgrounds(
|
||
["base.png", "happy.png", "sad.png"],
|
||
output_dir="transparent/",
|
||
)
|
||
```
|
||
|
||
| Option | Default | Description |
|
||
|--------|---------|-------------|
|
||
| `--input` | (required, repeatable) | Input image path(s) |
|
||
| `--output` | (auto) | Output path (single mode) |
|
||
| `--output-dir` | (none) | Output directory (batch mode) |
|
||
| `--model` | `isnet-anime` | Model: `isnet-anime`, `u2net`, `u2netp`, `u2net_human_seg`, `isnet-general-use`, `sam` |
|
||
|
||
The default `isnet-anime` model is trained on anime images — ideal for the
|
||
novaAnimeXL art style. Inference runs on CPU via onnxruntime (~0.25s per
|
||
1024×1024 image on Strix Halo). The model is ~176 MB, downloaded on first use
|
||
to `~/.u2net/`.
|
||
|
||
### Upscaling (Real-ESRGAN)
|
||
|
||
Upscale images 2×, 3×, or 4× using Real-ESRGAN with the anime-optimized
|
||
`RealESRGAN_x4plus_anime_6B` model:
|
||
|
||
```bash
|
||
# Single file
|
||
vnasset upscale --input character_base.png --output character_2x.png --scale 2
|
||
|
||
# Batch (reuses model across files)
|
||
vnasset upscale --input base.png --input happy.png --input sad.png --output-dir upscaled/ --scale 2
|
||
```
|
||
|
||
Or via the session API:
|
||
|
||
```python
|
||
with VnAssetsSession() as vna:
|
||
vna.upscale("base.png", output="base_2x.png", scale=2)
|
||
|
||
# Batch
|
||
vna.upscales(
|
||
["base.png", "happy.png", "sad.png"],
|
||
output_dir="upscaled/",
|
||
scale=2,
|
||
)
|
||
```
|
||
|
||
| Option | Default | Description |
|
||
|--------|---------|-------------|
|
||
| `--input` | (required, repeatable) | Input image path(s) |
|
||
| `--output` | (auto) | Output path (single mode) |
|
||
| `--output-dir` | (none) | Output directory (batch mode; `{stem}_x{scale}.png`) |
|
||
| `--scale` | `2` | Upscale factor: `2`, `3`, or `4` |
|
||
|
||
The model is ~17 MB (auto-downloaded from GitHub on first use). The upsampler
|
||
runs in FP32 on GPU and is lazy-loaded on first call within a session — it
|
||
coexists with SDXL and Qwen without memory pressure. A 256×256 warmup tile is
|
||
run at load time to compile CUDA kernels, so first-user upscale doesn't incur
|
||
a compilation penalty.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────┐
|
||
│ VnAssetsSession │
|
||
│ │
|
||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||
│ │ SDXL │ │ Qwen │ │ Qwen VL │ │ Qwen │ │
|
||
│ │ UNet │ │ Transf. │ │ 7B TE │ │ VAE │ │
|
||
│ │ (~3.5GB) │ │ (~20GB) │ │ (~14GB) │ │ (~1GB) │ │
|
||
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
|
||
│ │ │ │ │ │
|
||
│ ▼ │ │ │ │
|
||
│ ┌─────────┐ │ │ │ │
|
||
│ │ Generate│ │ │ │ │
|
||
│ │ │──────────┼───────────────┼──────────────┤ │
|
||
│ │ │ base.png │ │ │ │
|
||
│ └─────────┘ │ │ │ │
|
||
│ │ ▼ ▼ ▼ │
|
||
│ │ ┌──────────────────────────────────────┐ │
|
||
│ └──────────► Edit Phase │ │
|
||
│ │ base.png + prompts[] → variants[] │ │
|
||
│ └──────────────────────────────────────┘ │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ ┌──────────────────┐ │
|
||
│ │ base.png │ │
|
||
│ │ happy.png │ │
|
||
│ │ sad.png │ │
|
||
│ │ angry.png │ │
|
||
│ └────────┬─────────┘ │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ ┌──────────────────┐ │
|
||
│ │ Upscale │ │
|
||
│ │ (Real-ESRGAN) │ │
|
||
│ │ ~1.8s each │ │
|
||
│ └────────┬─────────┘ │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ ┌──────────────────┐ │
|
||
│ │ Remove BG │ │
|
||
│ │ (isnet-anime) │ │
|
||
│ └──────────────────┘ │
|
||
└─────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
The Qwen transformer is loaded FP8 → BF16 at construction using
|
||
`init_empty_weights` + incremental conversion to keep peak memory manageable
|
||
(20B parameters: 20 GB FP8 on disk, ~40 GB BF16 at runtime). All models fit
|
||
comfortably in 128 GB unified memory — no offloading, no swapping.
|
||
|
||
SDXL and Qwen use **separate VAEs** with different latent spaces. The SDXL
|
||
checkpoint bundles its own VAE; Qwen uses `Qwen/Qwen-Image` VAE from
|
||
HuggingFace. Both coexist in the same session without conflict.
|
||
|
||
The upsampler (Real-ESRGAN RRDBNet, ~17 MB, FP32) runs on GPU via PyTorch
|
||
and is lazy-loaded on first use. It uses tiled processing (512×512 tiles)
|
||
to keep memory modest even at 4× output (4096×4096).
|
||
|
||
## Data Flow
|
||
|
||
### Generate Phase
|
||
|
||
```
|
||
prompt text ──► Compel (SDXL CLIPs) ──► conditioning ──┐
|
||
├──► SDXL UNet (N steps) ──► latent ──► SDXL VAE decode ──► image
|
||
noise + latent ─────────────────────────────────────────┘
|
||
```
|
||
|
||
### Edit Phase
|
||
|
||
```
|
||
input image ──► Qwen VAE encode ──► latent ──┐
|
||
│
|
||
prompt text ──► Qwen VL 7B TE ──► conditioning┤
|
||
├──► Qwen Transformer (N steps) ──► latent ──► Qwen VAE decode ──► image
|
||
input image ──► Qwen VL 7B TE ──► visual tok.┘
|
||
```
|
||
|
||
The text encoder handles both text conditioning and visual token encoding from
|
||
the input image.
|
||
|
||
### Upscale Phase
|
||
|
||
```
|
||
input image (RGB) ──► tile split (512×512) ──┐
|
||
├──► RRDBNet (FP32, 4×) ──► tiles ──► blend ──► upscaled image
|
||
tile overlap padding ─────────────────────────┘
|
||
```
|
||
|
||
If the requested output scale is less than the model's native scale (e.g. 2×
|
||
from a 4× model), the 4× result is Lanczos-downsampled to the target size.
|
||
Tiled processing keeps GPU memory constant regardless of output resolution.
|
||
|
||
### Turbo vs Normal Mode
|
||
|
||
| Parameter | Normal | Turbo (Lightning LoRA) |
|
||
|-----------|--------|------------------------|
|
||
| Steps | 20 | 4 |
|
||
| CFG | 4.0 | 1.0 |
|
||
| LoRA | none | Lightning-4steps (fused at load) |
|
||
| Sampler | Flow Match Euler | Flow Match Euler |
|
||
|
||
## Performance
|
||
|
||
Hardware: **AMD Strix Halo** (Ryzen AI Max 395 Pro, Radeon 8060S, 128 GB),
|
||
bfloat16, `novaAnimeXL_ilV190.safetensors`.
|
||
|
||
### Generate (SDXL)
|
||
|
||
| Resolution | Steps | Load (s) | Inference (s) | Total (s) |
|
||
|-----------|-------|----------|---------------|-----------|
|
||
| 1024×1024 | 20 | ~2.5 | ~29 | ~31 |
|
||
|
||
Per-step breakdown (1024×1024):
|
||
- Step 1 (warmup): ~1.3–1.6s
|
||
- Steps 2–20 (steady): ~1.0–1.3s each
|
||
- VAE decode included in final step
|
||
|
||
Compel prompt weighting adds ~0.4s encoding overhead — negligible.
|
||
|
||
### Edit (Qwen Image Edit, 4-step Lightning LoRA)
|
||
|
||
| Attention | Steps | Load (s) | Inference (s) | Total (s) |
|
||
|-----------|-------|----------|---------------|-----------|
|
||
| Matmul fallback | 4 | ~28 | ~87 | ~115 |
|
||
| Flash attention | 4 | ~31 | ~53 | ~84 |
|
||
|
||
Per-step breakdown (1024×1024, flash attention):
|
||
- Step 1: ~0.3s (prefill/encoding)
|
||
- Steps 2–4: ~1.6 → ~5.1 → ~7.0s (transformer + VAE)
|
||
|
||
Flash attention (`TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1`) cuts edit
|
||
inference by **1.63×**. SDXL generate is unaffected (uses its own attention
|
||
processor, not SDPA).
|
||
|
||
### Session (persistent models)
|
||
|
||
| Phase | Wall (s) | Notes |
|
||
|-------|----------|-------|
|
||
| Session load | ~28 | SDXL + Qwen transformer + VAE + TE + LoRA fuse |
|
||
| Generate | ~31 | SDXL 20-step, 1024×1024 |
|
||
| Edit (turbo) | ~87 / ~54 | Matmul / flash attention |
|
||
| Upscale (2×) | ~1.8 | Real-ESRGAN, 1024→2048 |
|
||
| **Total (3 ops)** | **~147 / ~119** | One session load amortized |
|
||
|
||
With a session, each additional edit saves one model-load round trip (~28s).
|
||
5 edits save 112s. Flash attention adds a further 1.6× multiplier on inference.
|
||
|
||
See [`docs/stats.md`](docs/stats.md) for detailed per-run breakdowns.
|
||
|
||
### Upscale (Real-ESRGAN, anime model)
|
||
|
||
| Scale | Input | Tiles | Load (s) | Inference (s) |
|
||
|-------|-------|-------|----------|---------------|
|
||
| 2× | 1024×1024 | 4 | ~0.2¹ | ~1.8 |
|
||
| 4× | 1024×1024 | 4 | ~0.2¹ | ~1.8 |
|
||
|
||
¹ Warmup tile only; model download is ~17 MB (first use, cached thereafter).
|
||
|
||
Upscaling is tiled (512×512 input tiles) to keep GPU memory modest — the
|
||
RRDBNet forward pass processes each tile independently, and the results are
|
||
blended at tile boundaries. The 2× and 4× paths have near-identical latency
|
||
because the model is natively 4×; 2× output is achieved by upscaling to 4×
|
||
then downsampling.
|
||
|
||
A 256×256 warmup tile is run at model load time to compile CUDA kernels.
|
||
Without it, the first real upscale incurs ~80s of JIT compilation.
|
||
|
||
## Technical Notes
|
||
|
||
### bfloat16 required on RDNA 3.5
|
||
|
||
`float16` causes GPU kernel crashes (segfault) on the Radeon 8060S. The tool
|
||
uses `bfloat16` internally. This is transparent to the user.
|
||
|
||
### FP8 → BF16 conversion
|
||
|
||
The Qwen Image Edit model ships as FP8 (`fp8_e4m3fn`). It is converted to BF16
|
||
at load time using `init_empty_weights` + incremental tensor conversion to keep
|
||
peak memory manageable. RDNA 3.5 WMMA supports FP8 compute, but PyTorch ROCm FP8
|
||
support is not yet mature enough to compute in FP8 — upcast to BF16 is the safe
|
||
path and still fits in 128 GB.
|
||
|
||
### Attention backends
|
||
|
||
Two attention paths, selected automatically:
|
||
|
||
| Path | When | Performance |
|
||
|------|------|-------------|
|
||
| **Flash attention** | `TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1` set | 1.63× faster edits |
|
||
| **Matmul fallback** | Default (env var not set) | Stable, slower |
|
||
|
||
SDXL always uses a custom `AttnProcessor` (direct QKV matmul) regardless of the
|
||
env var — its attention path is separate from the Qwen SDPA dispatch. The flash
|
||
attention toggle only affects the Qwen transformer and text encoder.
|
||
|
||
```bash
|
||
# Enable flash attention for the session
|
||
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 vnasset edit ...
|
||
```
|
||
|
||
### ROCm torch version
|
||
|
||
Tested with `torch 2.11.0+rocm7.2`. Newer ROCm nightlies (2.13+, 2.14+) may
|
||
cause GPU crashes. If you encounter segfaults, try matching this version.
|
||
|
||
### LoRA loading
|
||
|
||
Lightning LoRA is loaded via `diffusers` native `load_lora_weights()` and fused
|
||
into the transformer with `fuse_lora()` at session construction. The fusion
|
||
takes ~1.5s and happens once — the turbo/normal switch is then just a
|
||
steps+CFG change with no per-step LoRA overhead.
|
||
|
||
## Prompt Syntax
|
||
|
||
VNAsset supports **ComfyUI-style prompt weighting** via the `compel` library.
|
||
|
||
### Weighting
|
||
|
||
| Syntax | Effect |
|
||
|--------|--------|
|
||
| `(word)` | Boost ×1.1 |
|
||
| `(word:1.5)` | Boost ×1.5 |
|
||
| `(word:0.6)` | De-emphasize ×0.6 |
|
||
| `[word]` | De-emphasize ×0.9 (shorthand) |
|
||
| `\(word\)` | Literal parentheses (escaped) |
|
||
|
||
```bash
|
||
vnasset generate \
|
||
--checkpoint models/novaAnimeXL_ilV190.safetensors \
|
||
--prompt "(masterpiece:1.2), 1girl, (red hair:1.3), blue eyes, [glasses]" \
|
||
--negative-prompt "(bad quality, worst quality:1.4)" \
|
||
--steps 20 --seed 42
|
||
```
|
||
|
||
### BREAK (condition chunking)
|
||
|
||
Split the prompt into independent conditioning chunks with `BREAK`:
|
||
|
||
```bash
|
||
vnasset generate \
|
||
--prompt "1girl, red hair, standing BREAK blue sky, cherry blossoms" \
|
||
--steps 20 --seed 42
|
||
```
|
||
|
||
Use `--raw` to bypass weighting and fall back to plain diffusers encoding.
|
||
|
||
## Current State
|
||
|
||
| Feature | Status |
|
||
|---------|--------|
|
||
| `vnasset generate` | ✅ Working |
|
||
| `vnasset edit` | ✅ Working |
|
||
| `VnAssetsSession` (persistent models) | ✅ Working |
|
||
| Compel prompt weighting + BREAK | ✅ Working |
|
||
| Lightning LoRA fuse-at-load | ✅ Working |
|
||
| Flash attention (experimental) | ✅ Working |
|
||
| `vnasset remove-bg` | ✅ Working |
|
||
| Session background removal | ✅ Working |
|
||
| `vnasset upscale` | ✅ Working |
|
||
| Session upscaling | ✅ Working |
|
||
| `vnasset pipeline` (batch YAML config) | 🚧 Planned |
|
||
| `vnasset serve` (daemon/HTTP API) | 🚧 Planned |
|
||
| `torch.compile` on UNet | 🚧 Planned |
|
||
| Batch edit loop (shared VAE encode) | 🚧 Planned |
|
||
|
||
## Future Improvements
|
||
|
||
- **Pipeline batch mode** — `vnasset pipeline --config pipeline.yaml` for
|
||
generate + multiple edits in one session from a YAML config file.
|
||
- **`torch.compile` on UNet** — the UNet forward is identical each step; ROCm's
|
||
`torch.compile` support is maturing and could cut per-step time significantly.
|
||
- **Shared encode optimization** — for N edit variants of the same input image,
|
||
run VAE encode and VL visual token encoding once, then only text-encode and
|
||
denoise per variant.
|
||
- **Self-contained torch wheel** — bundle the known-working torch wheel file in
|
||
the project (`wheels/torch-2.11.0+rocm7.2-cp312-cp312-linux_x86_64.whl`) so
|
||
the install is reproducible without depending on PyTorch's nightly index
|
||
availability or a ComfyUI installation.
|
||
- **`vnasset serve`** — lightweight daemon with Unix socket or HTTP API for
|
||
integrating VNAsset into external tools.
|