Add vnasset SDXL generate command
ROCm-safe bfloat16 inference with custom matmul attention. Automatic output directories, random seed, timing metadata.
This commit is contained in:
Michele Rossi
146
README.md
Normal file
146
README.md
Normal file
@@ -0,0 +1,146 @@
|
||||
# VNAsset
|
||||
|
||||
Fast CLI pipeline for visual novel image asset generation.
|
||||
|
||||
Drop-in replacement for the ComfyUI workflow loop: generate base character sprites
|
||||
with SDXL, then batch-edit variants (expressions, outfits) with Qwen Image Edit —
|
||||
all in one warm session, no node-graph overhead.
|
||||
|
||||
## Hardware
|
||||
|
||||
Built for **AMD Strix Halo** (Ryzen AI Max 395 Pro, Radeon 8060S, 128 GB unified
|
||||
memory). Also works on discrete AMD GPUs with ROCm. NVIDIA support is untested
|
||||
but should work if you swap the torch backend.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
git clone <repo> vnassets
|
||||
cd vnassets
|
||||
|
||||
# Create venv (Python 3.12 required for ROCm torch compatibility)
|
||||
python3.12 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
|
||||
# Install ROCm PyTorch (adjust index URL for your ROCm version)
|
||||
pip install torch --index-url https://download.pytorch.org/whl/rocm7.2
|
||||
|
||||
# Install the rest
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
### Models
|
||||
|
||||
Symlink your ComfyUI models into `models/`:
|
||||
|
||||
```bash
|
||||
cd models
|
||||
ln -s /path/to/ComfyUI/models/checkpoints/novaAnimeXL_ilV190.safetensors .
|
||||
ln -s /path/to/ComfyUI/models/diffusion_models/qwen_image_edit_2509_fp8_e4m3fn.safetensors .
|
||||
ln -s /path/to/ComfyUI/models/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors .
|
||||
ln -s /path/to/ComfyUI/models/vae/qwen_image_vae.safetensors .
|
||||
ln -s /path/to/ComfyUI/models/loras/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16.safetensors .
|
||||
```
|
||||
|
||||
Or place the actual files there — the tool just reads whatever safetensors you
|
||||
point it at.
|
||||
|
||||
## Usage
|
||||
|
||||
### Generate (SDXL text-to-image)
|
||||
|
||||
```bash
|
||||
vnasset generate \
|
||||
--checkpoint models/novaAnimeXL_ilV190.safetensors \
|
||||
--prompt "1girl, solo, red hair, glasses, blue eyes, white crop top, standing, portrait" \
|
||||
--negative-prompt "deformed, ugly, bad quality, lowres" \
|
||||
--steps 20 \
|
||||
--seed 42 \
|
||||
--output output/character_base.png
|
||||
```
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `--checkpoint` | (required) | Path to SDXL `.safetensors` |
|
||||
| `--prompt` | (required) | Positive prompt |
|
||||
| `--negative-prompt` | `""` | Negative prompt |
|
||||
| `--width` | `1024` | Image width |
|
||||
| `--height` | `1024` | Image height |
|
||||
| `--steps` | `20` | Inference steps |
|
||||
| `--cfg` | `4.5` | CFG scale |
|
||||
| `--seed` | `0` | RNG seed (use `random` for random) |
|
||||
| `--output` | `output.png` | Output path |
|
||||
|
||||
When `--seed` is `random`, a random seed is generated and recorded in the
|
||||
metadata file.
|
||||
|
||||
### Output
|
||||
|
||||
Each generation produces:
|
||||
- `{output}.png` — the image
|
||||
- `{output}.json` — metadata (prompt, seed, model path, timing, resolution)
|
||||
|
||||
Directories in `--output` are created automatically.
|
||||
|
||||
## Current State
|
||||
|
||||
| Command | Status |
|
||||
|---------|--------|
|
||||
| `vnasset generate` | ✅ Working |
|
||||
| `vnasset edit` | 🚧 Planned |
|
||||
| `vnasset pipeline` | 🚧 Planned |
|
||||
|
||||
## Performance
|
||||
|
||||
Radeon 8060S (Strix Halo iGPU), bfloat16, 1024×1024:
|
||||
- ~1s per step
|
||||
- ~20s for 20-step generation
|
||||
|
||||
Model loading adds ~5s cold-start overhead.
|
||||
|
||||
## Technical Notes
|
||||
|
||||
### bfloat16 required on RDNA 3.5
|
||||
|
||||
`float16` causes GPU kernel crashes (segfault) on the Radeon 8060S. The tool
|
||||
uses `bfloat16` internally. This is transparent to the user.
|
||||
|
||||
### Custom attention
|
||||
|
||||
The default PyTorch SDPA backends (flash attention, mem-efficient attention) are
|
||||
unstable on this AMD GPU. VNAsset uses a simple matmul-based attention
|
||||
implementation that avoids the SDPA dispatch entirely.
|
||||
|
||||
### ROCm torch version
|
||||
|
||||
Tested with `torch 2.11.0+rocm7.2`. Newer ROCm nightlies (2.13+, 2.14+) may
|
||||
cause GPU crashes. If you encounter segfaults, try matching this version.
|
||||
|
||||
## Prompt Compatibility
|
||||
|
||||
VNAsset uses standard diffusers SDXL encoding, which is equivalent to ComfyUI's
|
||||
`BNK_CLIPTextEncodeAdvanced` with `token_normalization=none` and
|
||||
`weight_interpretation=comfy` for plain comma-separated prompts.
|
||||
|
||||
ComfyUI-specific syntax is **not currently supported**:
|
||||
- `(word:1.2)` — prompt weighting
|
||||
- `BREAK` — conditioning chunking
|
||||
|
||||
If your prompts rely on these, you'll get different output than the ComfyUI
|
||||
workflow. compel integration is planned for later.
|
||||
|
||||
## Future Improvements
|
||||
|
||||
- **Persistent model session** — keep models loaded between commands to
|
||||
eliminate the ~5s cold-start overhead per generation. A `vnasset serve`
|
||||
daemon or `vnasset batch` command.
|
||||
- **`torch.compile` on UNet** — the UNet forward is identical each step; ROCm's
|
||||
`torch.compile` support is maturing and could cut per-step time in half.
|
||||
- **Self-contained torch wheel** — bundle the known-working torch wheel file in
|
||||
the project (`wheels/torch-2.11.0+rocm7.2-cp312-cp312-linux_x86_64.whl`) so
|
||||
the install is reproducible without depending on PyTorch's nightly index
|
||||
availability or a ComfyUI installation.
|
||||
- **Qwen Image Edit support** — `vnasset edit` and `vnasset pipeline` for
|
||||
batch expression/outfit variant editing.
|
||||
- **compel prompt weighting** — support `(word:weight)` and `BREAK` syntax for
|
||||
parity with ComfyUI prompt encoding.
|
||||
Reference in New Issue
Block a user