VNAsset

Fast CLI pipeline for visual novel image asset generation.

Drop-in replacement for the ComfyUI workflow loop: generate base character sprites with SDXL, then batch-edit variants (expressions, outfits) with Qwen Image Edit — all in one warm session, no node-graph overhead.

Hardware

Built for AMD Strix Halo (Ryzen AI Max 395 Pro, Radeon 8060S, 128 GB unified memory). Also works on discrete AMD GPUs with ROCm. NVIDIA support is untested but should work if you swap the torch backend.

The 128 GB unified memory means VRAM is effectively unlimited (up to ~96 GB allocatable to GPU). The bottleneck is GPU compute throughput, not memory capacity — model offloading is pointless, everything stays resident.

Install

git clone <repo> vnassets
cd vnassets

# Create venv (Python 3.12 required for ROCm torch compatibility)
python3.12 -m venv .venv
source .venv/bin/activate

# Install ROCm PyTorch (adjust index URL for your ROCm version)
pip install torch --index-url https://download.pytorch.org/whl/rocm7.2

# Install the rest
pip install -e .

Models

Symlink your ComfyUI models into models/:

cd models
ln -s /path/to/ComfyUI/models/checkpoints/novaAnimeXL_ilV190.safetensors .
ln -s /path/to/ComfyUI/models/diffusion_models/qwen_image_edit_2509_fp8_e4m3fn.safetensors .
ln -s /path/to/ComfyUI/models/loras/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors .

The Qwen VAE and text encoder are downloaded automatically from HuggingFace Hub on first use (Qwen/Qwen-Image and Qwen/Qwen2.5-VL-7B-Instruct). No symlinks needed for those.

Or place the actual files there — the tool just reads whatever safetensors you point it at.

Usage

Generate (SDXL text-to-image)

vnasset generate \
  --checkpoint models/novaAnimeXL_ilV190.safetensors \
  --prompt "1girl, solo, red hair, glasses, blue eyes, white crop top, standing, portrait" \
  --negative-prompt "deformed, ugly, bad quality, lowres" \
  --steps 20 \
  --seed 42 \
  --output output/character_base.png

Option	Default	Description
--checkpoint	(required)	Path to SDXL .safetensors
--prompt	(required)	Positive prompt
--negative-prompt	""	Negative prompt
--width	1024	Image width
--height	1024	Image height
--steps	20	Inference steps
--cfg	4.5	CFG scale
--seed	random	RNG seed (integer or random)
--output	output.png	Output path
--raw	false	Disable Compel prompt weighting (fall back to plain diffusers encoding)

Edit (Qwen Image Edit)

vnasset edit \
  --model models/qwen_image_edit_2509_fp8_e4m3fn.safetensors \
  --input character_base.png \
  --prompt "make her smile happily" \
  --steps 4 --cfg 1.0 \
  --lora models/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors \
  --output character_happy.png

Option	Default	Description
--model	(required)	Path to Qwen Image Edit .safetensors (FP8)
--input	(required)	Input image to edit
--prompt	(required)	Edit instruction
--steps	20	Inference steps (4 with Lightning LoRA)
--cfg	4.0	CFG scale (1.0 with Lightning LoRA)
--seed	random	RNG seed
--lora	(none)	Path to LoRA .safetensors
--output	output.png	Output path

Turbo mode: Use the Lightning 4-step LoRA with --steps 4 --cfg 1.0 to cut inference time proportionally. The LoRA is fused into the transformer at load time, so there is no per-step LoRA overhead.

Output

Each generation produces:

• {output}.png — the image
• {output}.json — metadata (prompt, seed, model path, timing, resolution)

Directories in --output are created automatically.

Session (persistent models)

For multi-call workflows, use VnAssetsSession to keep models loaded in GPU memory between operations. Models are loaded eagerly at construction:

from vnassets import VnAssetsSession

with VnAssetsSession(
    sdxl_checkpoint="models/novaAnimeXL.safetensors",
    edit_model="models/qwen_image_edit.safetensors",
    edit_lora="models/lightning-4steps.safetensors",
) as vna:
    vna.generate("1girl, red hair", output="base.png")
    vna.edit("base.png", "make her smile", output="happy.png")
    vna.edit("base.png", "make her sad", output="sad.png")

Either model can be omitted (None) for single-model sessions. Properties:

• vna.has_sdxl / vna.has_qwen — check which models are loaded
• vna.load_time_s — total session construction time
• vna.close() — manual cleanup (automatic with with)

The standalone vnasset generate and vnasset edit CLI commands are thin wrappers around a one-shot session — same API, backwards compatible.

Background Removal

Remove backgrounds from character sprites (output is RGBA PNG):

# Single file
vnasset remove-bg --input character_base.png --output character_transparent.png

# Batch (reuses model across files)
vnasset remove-bg --input base.png --input happy.png --input sad.png --output-dir transparent/

Or via the session API:

with VnAssetsSession() as vna:
    vna.remove_background("base.png", output="base_transparent.png")

    # Batch
    vna.remove_backgrounds(
        ["base.png", "happy.png", "sad.png"],
        output_dir="transparent/",
    )

Option	Default	Description
--input	(required, repeatable)	Input image path(s)
--output	(auto)	Output path (single mode)
--output-dir	(none)	Output directory (batch mode)
--model	isnet-anime	Model: isnet-anime, u2net, u2netp, u2net_human_seg, isnet-general-use, sam

The default isnet-anime model is trained on anime images — ideal for the novaAnimeXL art style. Inference runs on CPU via onnxruntime (~0.25s per 1024×1024 image on Strix Halo). The model is ~176 MB, downloaded on first use to ~/.u2net/.

Upscaling (Real-ESRGAN)

Upscale images 2×, 3×, or 4× using Real-ESRGAN with the anime-optimized RealESRGAN_x4plus_anime_6B model:

# Single file
vnasset upscale --input character_base.png --output character_2x.png --scale 2

# Batch (reuses model across files)
vnasset upscale --input base.png --input happy.png --input sad.png --output-dir upscaled/ --scale 2

Or via the session API:

with VnAssetsSession() as vna:
    vna.upscale("base.png", output="base_2x.png", scale=2)

    # Batch
    vna.upscales(
        ["base.png", "happy.png", "sad.png"],
        output_dir="upscaled/",
        scale=2,
    )

Option	Default	Description
--input	(required, repeatable)	Input image path(s)
--output	(auto)	Output path (single mode)
--output-dir	(none)	Output directory (batch mode; {stem}_x{scale}.png)
--scale	2	Upscale factor: 2, 3, or 4

The model is ~17 MB (auto-downloaded from GitHub on first use). The upsampler runs in FP32 on GPU and is lazy-loaded on first call within a session — it coexists with SDXL and Qwen without memory pressure. A 256×256 warmup tile is run at load time to compile CUDA kernels, so first-user upscale doesn't incur a compilation penalty.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                      VnAssetsSession                         │
│                                                              │
│  ┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐ │
│  │ SDXL     │   │ Qwen     │   │ Qwen VL  │   │ Qwen     │ │
│  │ UNet     │   │ Transf.  │   │ 7B TE    │   │ VAE      │ │
│  │ (~3.5GB) │   │ (~20GB)  │   │ (~14GB)  │   │ (~1GB)   │ │
│  └──────────┘   └──────────┘   └──────────┘   └──────────┘ │
│       │               │               │              │       │
│       ▼               │               │              │       │
│  ┌─────────┐          │               │              │       │
│  │ Generate│          │               │              │       │
│  │         │──────────┼───────────────┼──────────────┤       │
│  │         │  base.png │              │              │       │
│  └─────────┘          │               │              │       │
│       │               ▼               ▼              ▼       │
│       │          ┌──────────────────────────────────────┐    │
│       └──────────►         Edit Phase                    │    │
│                  │  base.png + prompts[] → variants[]    │    │
│                  └──────────────────────────────────────┘    │
│                                      │                       │
│                                      ▼                       │
│                           ┌──────────────────┐               │
│                           │   base.png       │               │
│                           │   happy.png      │               │
│                           │   sad.png        │               │
│                           │   angry.png      │               │
│                           └────────┬─────────┘               │
│                                    │                         │
│                                    ▼                         │
│                           ┌──────────────────┐               │
│                           │   Upscale        │               │
│                           │ (Real-ESRGAN)    │               │
│                           │   ~1.8s each     │               │
│                           └────────┬─────────┘               │
│                                    │                         │
│                                    ▼                         │
│                           ┌──────────────────┐               │
│                           │   Remove BG      │               │
│                           │ (isnet-anime)    │               │
│                           └──────────────────┘               │
└─────────────────────────────────────────────────────────────┘

The Qwen transformer is loaded FP8 → BF16 at construction using init_empty_weights + incremental conversion to keep peak memory manageable (20B parameters: 20 GB FP8 on disk, ~40 GB BF16 at runtime). All models fit comfortably in 128 GB unified memory — no offloading, no swapping.

SDXL and Qwen use separate VAEs with different latent spaces. The SDXL checkpoint bundles its own VAE; Qwen uses Qwen/Qwen-Image VAE from HuggingFace. Both coexist in the same session without conflict.

The upsampler (Real-ESRGAN RRDBNet, ~17 MB, FP32) runs on GPU via PyTorch and is lazy-loaded on first use. It uses tiled processing (512×512 tiles) to keep memory modest even at 4× output (4096×4096).

Data Flow

Generate Phase

prompt text ──► Compel (SDXL CLIPs) ──► conditioning ──┐
                                                        ├──► SDXL UNet (N steps) ──► latent ──► SDXL VAE decode ──► image
noise + latent ─────────────────────────────────────────┘

Edit Phase

input image ──► Qwen VAE encode ──► latent ──┐
                                              │
prompt text ──► Qwen VL 7B TE ──► conditioning┤
                                              ├──► Qwen Transformer (N steps) ──► latent ──► Qwen VAE decode ──► image
input image ──► Qwen VL 7B TE ──► visual tok.┘

The text encoder handles both text conditioning and visual token encoding from the input image.

Upscale Phase

input image (RGB) ──► tile split (512×512) ──┐
                                              ├──► RRDBNet (FP32, 4×) ──► tiles ──► blend ──► upscaled image
tile overlap padding ─────────────────────────┘

If the requested output scale is less than the model's native scale (e.g. 2× from a 4× model), the 4× result is Lanczos-downsampled to the target size. Tiled processing keeps GPU memory constant regardless of output resolution.

Turbo vs Normal Mode

Parameter	Normal	Turbo (Lightning LoRA)
Steps	20	4
CFG	4.0	1.0
LoRA	none	Lightning-4steps (fused at load)
Sampler	Flow Match Euler	Flow Match Euler

Performance

Hardware: AMD Strix Halo (Ryzen AI Max 395 Pro, Radeon 8060S, 128 GB), bfloat16, novaAnimeXL_ilV190.safetensors.

Generate (SDXL)

Resolution	Steps	Load (s)	Inference (s)	Total (s)
1024×1024	20	~2.5	~29	~31

Per-step breakdown (1024×1024):

• Step 1 (warmup): ~1.3–1.6s
• Steps 2–20 (steady): ~1.0–1.3s each
• VAE decode included in final step

Compel prompt weighting adds ~0.4s encoding overhead — negligible.

Edit (Qwen Image Edit, 4-step Lightning LoRA)

Attention	Steps	Load (s)	Inference (s)	Total (s)
Matmul fallback	4	~28	~87	~115
Flash attention	4	~31	~53	~84

Per-step breakdown (1024×1024, flash attention):

• Step 1: ~0.3s (prefill/encoding)
• Steps 2–4: ~1.6 → ~5.1 → ~7.0s (transformer + VAE)

Flash attention (TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1) cuts edit inference by 1.63×. SDXL generate is unaffected (uses its own attention processor, not SDPA).

Session (persistent models)

Phase	Wall (s)	Notes
Session load	~28	SDXL + Qwen transformer + VAE + TE + LoRA fuse
Generate	~31	SDXL 20-step, 1024×1024
Edit (turbo)	~87 / ~54	Matmul / flash attention
Upscale (2×)	~1.8	Real-ESRGAN, 1024→2048
Total (3 ops)	~147 / ~119	One session load amortized

With a session, each additional edit saves one model-load round trip (~28s). 5 edits save 112s. Flash attention adds a further 1.6× multiplier on inference.

See docs/stats.md for detailed per-run breakdowns.

Upscale (Real-ESRGAN, anime model)

Scale	Input	Tiles	Load (s)	Inference (s)
2×	1024×1024	4	~0.2¹	~1.8
4×	1024×1024	4	~0.2¹	~1.8

¹ Warmup tile only; model download is ~17 MB (first use, cached thereafter).

Upscaling is tiled (512×512 input tiles) to keep GPU memory modest — the RRDBNet forward pass processes each tile independently, and the results are blended at tile boundaries. The 2× and 4× paths have near-identical latency because the model is natively 4×; 2× output is achieved by upscaling to 4× then downsampling.

A 256×256 warmup tile is run at model load time to compile CUDA kernels. Without it, the first real upscale incurs ~80s of JIT compilation.

Technical Notes

bfloat16 required on RDNA 3.5

float16 causes GPU kernel crashes (segfault) on the Radeon 8060S. The tool uses bfloat16 internally. This is transparent to the user.

FP8 → BF16 conversion

The Qwen Image Edit model ships as FP8 (fp8_e4m3fn). It is converted to BF16 at load time using init_empty_weights + incremental tensor conversion to keep peak memory manageable. RDNA 3.5 WMMA supports FP8 compute, but PyTorch ROCm FP8 support is not yet mature enough to compute in FP8 — upcast to BF16 is the safe path and still fits in 128 GB.

Attention backends

Two attention paths, selected automatically:

Path	When	Performance
Flash attention	TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 set	1.63× faster edits
Matmul fallback	Default (env var not set)	Stable, slower

SDXL always uses a custom AttnProcessor (direct QKV matmul) regardless of the env var — its attention path is separate from the Qwen SDPA dispatch. The flash attention toggle only affects the Qwen transformer and text encoder.

# Enable flash attention for the session
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 vnasset edit ...

ROCm torch version

Tested with torch 2.11.0+rocm7.2. Newer ROCm nightlies (2.13+, 2.14+) may cause GPU crashes. If you encounter segfaults, try matching this version.

LoRA loading

Lightning LoRA is loaded via diffusers native load_lora_weights() and fused into the transformer with fuse_lora() at session construction. The fusion takes ~1.5s and happens once — the turbo/normal switch is then just a steps+CFG change with no per-step LoRA overhead.

Prompt Syntax

VNAsset supports ComfyUI-style prompt weighting via the compel library.

Weighting

Syntax	Effect
(word)	Boost ×1.1
(word:1.5)	Boost ×1.5
(word:0.6)	De-emphasize ×0.6
[word]	De-emphasize ×0.9 (shorthand)
\(word\)	Literal parentheses (escaped)

vnasset generate \
  --checkpoint models/novaAnimeXL_ilV190.safetensors \
  --prompt "(masterpiece:1.2), 1girl, (red hair:1.3), blue eyes, [glasses]" \
  --negative-prompt "(bad quality, worst quality:1.4)" \
  --steps 20 --seed 42

BREAK (condition chunking)

Split the prompt into independent conditioning chunks with BREAK:

vnasset generate \
  --prompt "1girl, red hair, standing BREAK blue sky, cherry blossoms" \
  --steps 20 --seed 42

Use --raw to bypass weighting and fall back to plain diffusers encoding.

Current State

Feature	Status
vnasset generate	✅ Working
vnasset edit	✅ Working
VnAssetsSession (persistent models)	✅ Working
Compel prompt weighting + BREAK	✅ Working
Lightning LoRA fuse-at-load	✅ Working
Flash attention (experimental)	✅ Working
vnasset remove-bg	✅ Working
Session background removal	✅ Working
vnasset upscale	✅ Working
Session upscaling	✅ Working
vnasset pipeline (batch YAML config)	🚧 Planned
vnasset serve (daemon/HTTP API)	🚧 Planned
torch.compile on UNet	🚧 Planned
Batch edit loop (shared VAE encode)	🚧 Planned

Future Improvements

• Pipeline batch mode — vnasset pipeline --config pipeline.yaml for generate + multiple edits in one session from a YAML config file.
• torch.compile on UNet — the UNet forward is identical each step; ROCm's torch.compile support is maturing and could cut per-step time significantly.
• Shared encode optimization — for N edit variants of the same input image, run VAE encode and VL visual token encoding once, then only text-encode and denoise per variant.
• Self-contained torch wheel — bundle the known-working torch wheel file in the project (wheels/torch-2.11.0+rocm7.2-cp312-cp312-linux_x86_64.whl) so the install is reproducible without depending on PyTorch's nightly index availability or a ComfyUI installation.
• vnasset serve — lightweight daemon with Unix socket or HTTP API for integrating VNAsset into external tools.