06fba9c234
ROCm-safe bfloat16 inference with custom matmul attention. Automatic output directories, random seed, timing metadata.
147 lines
4.8 KiB
Markdown
147 lines
4.8 KiB
Markdown
# VNAsset
|
||
|
||
Fast CLI pipeline for visual novel image asset generation.
|
||
|
||
Drop-in replacement for the ComfyUI workflow loop: generate base character sprites
|
||
with SDXL, then batch-edit variants (expressions, outfits) with Qwen Image Edit —
|
||
all in one warm session, no node-graph overhead.
|
||
|
||
## Hardware
|
||
|
||
Built for **AMD Strix Halo** (Ryzen AI Max 395 Pro, Radeon 8060S, 128 GB unified
|
||
memory). Also works on discrete AMD GPUs with ROCm. NVIDIA support is untested
|
||
but should work if you swap the torch backend.
|
||
|
||
## Install
|
||
|
||
```bash
|
||
git clone <repo> vnassets
|
||
cd vnassets
|
||
|
||
# Create venv (Python 3.12 required for ROCm torch compatibility)
|
||
python3.12 -m venv .venv
|
||
source .venv/bin/activate
|
||
|
||
# Install ROCm PyTorch (adjust index URL for your ROCm version)
|
||
pip install torch --index-url https://download.pytorch.org/whl/rocm7.2
|
||
|
||
# Install the rest
|
||
pip install -e .
|
||
```
|
||
|
||
### Models
|
||
|
||
Symlink your ComfyUI models into `models/`:
|
||
|
||
```bash
|
||
cd models
|
||
ln -s /path/to/ComfyUI/models/checkpoints/novaAnimeXL_ilV190.safetensors .
|
||
ln -s /path/to/ComfyUI/models/diffusion_models/qwen_image_edit_2509_fp8_e4m3fn.safetensors .
|
||
ln -s /path/to/ComfyUI/models/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors .
|
||
ln -s /path/to/ComfyUI/models/vae/qwen_image_vae.safetensors .
|
||
ln -s /path/to/ComfyUI/models/loras/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors .
|
||
```
|
||
|
||
Or place the actual files there — the tool just reads whatever safetensors you
|
||
point it at.
|
||
|
||
## Usage
|
||
|
||
### Generate (SDXL text-to-image)
|
||
|
||
```bash
|
||
vnasset generate \
|
||
--checkpoint models/novaAnimeXL_ilV190.safetensors \
|
||
--prompt "1girl, solo, red hair, glasses, blue eyes, white crop top, standing, portrait" \
|
||
--negative-prompt "deformed, ugly, bad quality, lowres" \
|
||
--steps 20 \
|
||
--seed 42 \
|
||
--output output/character_base.png
|
||
```
|
||
|
||
| Option | Default | Description |
|
||
|--------|---------|-------------|
|
||
| `--checkpoint` | (required) | Path to SDXL `.safetensors` |
|
||
| `--prompt` | (required) | Positive prompt |
|
||
| `--negative-prompt` | `""` | Negative prompt |
|
||
| `--width` | `1024` | Image width |
|
||
| `--height` | `1024` | Image height |
|
||
| `--steps` | `20` | Inference steps |
|
||
| `--cfg` | `4.5` | CFG scale |
|
||
| `--seed` | `0` | RNG seed (use `random` for random) |
|
||
| `--output` | `output.png` | Output path |
|
||
|
||
When `--seed` is `random`, a random seed is generated and recorded in the
|
||
metadata file.
|
||
|
||
### Output
|
||
|
||
Each generation produces:
|
||
- `{output}.png` — the image
|
||
- `{output}.json` — metadata (prompt, seed, model path, timing, resolution)
|
||
|
||
Directories in `--output` are created automatically.
|
||
|
||
## Current State
|
||
|
||
| Command | Status |
|
||
|---------|--------|
|
||
| `vnasset generate` | ✅ Working |
|
||
| `vnasset edit` | 🚧 Planned |
|
||
| `vnasset pipeline` | 🚧 Planned |
|
||
|
||
## Performance
|
||
|
||
Radeon 8060S (Strix Halo iGPU), bfloat16, 1024×1024:
|
||
- ~1s per step
|
||
- ~20s for 20-step generation
|
||
|
||
Model loading adds ~5s cold-start overhead.
|
||
|
||
## Technical Notes
|
||
|
||
### bfloat16 required on RDNA 3.5
|
||
|
||
`float16` causes GPU kernel crashes (segfault) on the Radeon 8060S. The tool
|
||
uses `bfloat16` internally. This is transparent to the user.
|
||
|
||
### Custom attention
|
||
|
||
The default PyTorch SDPA backends (flash attention, mem-efficient attention) are
|
||
unstable on this AMD GPU. VNAsset uses a simple matmul-based attention
|
||
implementation that avoids the SDPA dispatch entirely.
|
||
|
||
### ROCm torch version
|
||
|
||
Tested with `torch 2.11.0+rocm7.2`. Newer ROCm nightlies (2.13+, 2.14+) may
|
||
cause GPU crashes. If you encounter segfaults, try matching this version.
|
||
|
||
## Prompt Compatibility
|
||
|
||
VNAsset uses standard diffusers SDXL encoding, which is equivalent to ComfyUI's
|
||
`BNK_CLIPTextEncodeAdvanced` with `token_normalization=none` and
|
||
`weight_interpretation=comfy` for plain comma-separated prompts.
|
||
|
||
ComfyUI-specific syntax is **not currently supported**:
|
||
- `(word:1.2)` — prompt weighting
|
||
- `BREAK` — conditioning chunking
|
||
|
||
If your prompts rely on these, you'll get different output than the ComfyUI
|
||
workflow. compel integration is planned for later.
|
||
|
||
## Future Improvements
|
||
|
||
- **Persistent model session** — keep models loaded between commands to
|
||
eliminate the ~5s cold-start overhead per generation. A `vnasset serve`
|
||
daemon or `vnasset batch` command.
|
||
- **`torch.compile` on UNet** — the UNet forward is identical each step; ROCm's
|
||
`torch.compile` support is maturing and could cut per-step time in half.
|
||
- **Self-contained torch wheel** — bundle the known-working torch wheel file in
|
||
the project (`wheels/torch-2.11.0+rocm7.2-cp312-cp312-linux_x86_64.whl`) so
|
||
the install is reproducible without depending on PyTorch's nightly index
|
||
availability or a ComfyUI installation.
|
||
- **Qwen Image Edit support** — `vnasset edit` and `vnasset pipeline` for
|
||
batch expression/outfit variant editing.
|
||
- **compel prompt weighting** — support `(word:weight)` and `BREAK` syntax for
|
||
parity with ComfyUI prompt encoding.
|