Bundles the known-working ROCm torch, triton-rocm, and torchvision wheels
locally so install no longer depends on the PyTorch nightly index being
available. Also fixes model name mismatches between docs and code, and
adds missing dependencies.
What:
• wheels/ — bundled torch 2.11.0+rocm7.2, triton-rocm 3.6.0, and
torchvision 0.26.0+rocm7.2 (wheels/ is in .gitignore; they are
downloaded once and reused)
• README install section — replaced 'pip install torch --index-url ...'
with 'pip install wheels/*.whl' for all three ROCm wheels; added
download instructions for the initial fetch
• README Current State — marked self-contained torch wheel as done,
removed from Future Improvements
• Fixed model name defaults across README examples and session.py
docstring — they used shortened filenames (novaAnimeXL.safetensors,
qwen_image_edit.safetensors) that don't match the actual symlinks
created by the Models section
• pyproject.toml — added torchvision and realesrgan (previously
undeclared; upscale.py imported torchvision.transforms at module
level without the dependency being declared)
Why:
The ROCm nightly index is volatile — versions rotate frequently and the
index itself may not be reachable. Bundling the three ROCm-only wheels
(torch, triton-rocm, torchvision) makes the install reproducible. All
other dependencies resolve from standard PyPI.
Verified by creating a clean Python 3.12 venv, installing all three
wheels from the local files, running 'pip install -e .', and confirming
'vnasset --help' and GPU tensor allocation work without any remote
ROCm index access.
Multi-stage pipelines declared in YAML: generate, edit (cross-product),
remove_bg, and upscale stages executed sequentially in a single GPU
session. Every intermediate artifact (image + metadata JSON) is saved
to {output_dir}/{stage_id}/ for full traceability.
Data routing:
- generate: list of prompts → list of images (1:1 per item)
- edit: fan-out cross-product (each input × each prompt)
- remove_bg / upscale: 1:1 passthrough
Resume: outputs that already exist are skipped. Use --force to re-run
everything — lets you add items to a stage without regenerating.
Examples: examples/portrait.yaml, examples/backgrounds.yaml
Adds `vnasset upscale` CLI command, `VnAssetsSession.upscale()` / `.upscales()`
session methods, and a standalone `vnassets.upscale` module following the
existing remove-bg pattern.
Uses Real-ESRGAN RRDBNet with RealESRGAN_x4plus_anime_6B (~17 MB,
auto-downloaded). A 256x256 warmup tile at load time eliminates ~80s of
first-run CUDA JIT compilation on ROCm. Steady-state: ~1.8s per 1024->2048
upscale on Strix Halo. The upsampler is lazy-loaded on first call and coexists
with SDXL/Qwen in the same session.
Works around a basicsr/torchvision API incompatibility (rgb_to_grayscale moved
in torchvision 0.20+) with a 3-line module shim in upscale.py.
rembg with ONNX Runtime on CPU was chosen over BiRefNet: the latter
requires deform_conv2d which crashes on ROCm bfloat16 and runs at 40s
in float32. rembg delivers 0.23s per 1024x1024 image, no GPU deps,
and isnet-anime is trained specifically on anime images — exactly the
target domain.
CLI and session API both support single-image and batch (plural-first)
modes, reusing one model session across files.
- Consolidate TECH_SPEC into README as single source of truth
- Remove TECH_SPEC.md (375-line README covers everything)
- Fix architecture diagram: Qwen transformer ~5GB → ~20GB (20B params)
- Remove unimplemented CLI flags from docs: --sampler, --scheduler, --clip,
--vae, --turbo (these don't exist in the actual code)
- Replace 'FP8 native compute' claim with actual FP8→BF16 conversion
- Replace 'peft' with actual diffusers native LoRA loading
- Move aspirational optimizations (VAE overlap, shared encode) to Future
- Resolve stale Open Questions; keep torch.compile as Future item
- Move 'Persistent model session' and 'Qwen Image Edit' from Future to
implemented status with documented API
- Update performance numbers with actual measured session+flash numbers
- Document flash attention path alongside matmul fallback
- Simplify data flow diagram, remove ComfyUI-only kontext_scale concept
- Remove dead doc links (sdxl-generation.md)
- Extract model loading from generate()/edit() into VnAssetsSession class
- Session eagerly loads SDXL + Qwen Image Edit at construction (28s, once)
- Both models held in GPU memory across calls; generate()/edit() reuse them
- generate.py and edit.py become thin wrappers (backwards compatible CLI)
- Context manager (with VnAssetsSession(...) as vna:) for library use
- Metadata backwards-compatible: all fields preserved including lora_load_s
- load_time_s now reflects total session construction, amortized across calls
- Add performance stats for edit path (Qwen Image Edit + Lightning LoRA)
- Benchmark matmul fallback (86.8s) vs flash attention (53.3s, 1.63x speedup)
- Session vs cold start comparison: 2 ops save one 28s load, 5 edits save 112s
- Flash attention via TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 documented
- Support (word:weight), [word], ((nested)) syntax
- Support BREAK for conditioning chunking (.and() translation)
- Use CompelForSDXL (modern API, avoids deprecation)
- Add --raw flag to bypass weighting and fall back to plain encoding
- Update README with Prompt Syntax section and examples
- Add docs/comfyui-prompt-style.md with design doc
Monkey-patches dispatch_attention_fn to use simple Q@K^T@V matmul,
bypassing SDPA dispatch entirely. Enables editing at full 1024x1024
(was limited to 512x512 due to SDPA crashes) and is ~3x faster.
LoRA loading via PEFT crashes on ROCm; removed since user doesn't need
it. Added automatic downscaling of inputs >512px to avoid O(N^2)
attention explosion in the Qwen transformer (262K tokens at 1024^2
exceeds GPU capability). Added explicit GPU memory cleanup after each
command to prevent OOM when chaining generate + edit.
Qwen Image Edit pipeline with FP8 Safetensors loading.
Uses init_empty_weights for memory-efficient 40GB model loading.
bf16 dtype to avoid ROCm crashes; falls back to math SDPA.
