add YAML-driven pipeline batch mode (vnasset pipeline)
Multi-stage pipelines declared in YAML: generate, edit (cross-product),
remove_bg, and upscale stages executed sequentially in a single GPU
session. Every intermediate artifact (image + metadata JSON) is saved
to {output_dir}/{stage_id}/ for full traceability.
Data routing:
- generate: list of prompts → list of images (1:1 per item)
- edit: fan-out cross-product (each input × each prompt)
- remove_bg / upscale: 1:1 passthrough
Resume: outputs that already exist are skipped. Use --force to re-run
everything — lets you add items to a stage without regenerating.
Examples: examples/portrait.yaml, examples/backgrounds.yaml
This commit is contained in:
Michele Rossi
186
README.md
186
README.md
@@ -53,6 +53,105 @@ point it at.
|
||||
|
||||
## Usage
|
||||
|
||||
### Pipeline (batch YAML config)
|
||||
|
||||
Declare multi-stage pipelines in YAML — the primary workflow for bulk asset
|
||||
generation. All intermediate outputs are saved: every stage gets its own
|
||||
subdirectory with images and metadata JSON files.
|
||||
|
||||
```bash
|
||||
# Portrait pipeline: generate → emotion variants → remove bg → upscale
|
||||
vnasset pipeline --config examples/portrait.yaml
|
||||
|
||||
# Background batch: generate many images from independent prompts
|
||||
vnasset pipeline --config examples/backgrounds.yaml
|
||||
|
||||
# Force re-run all stages (default: skip items whose output files exist)
|
||||
vnasset pipeline --config examples/portrait.yaml --force
|
||||
```
|
||||
|
||||
Config structure:
|
||||
|
||||
```yaml
|
||||
session:
|
||||
sdxl_checkpoint: models/novaAnimeXL.safetensors
|
||||
edit_model: models/qwen_image_edit.safetensors
|
||||
edit_lora: models/lightning-4steps.safetensors # optional
|
||||
|
||||
output_dir: output/my_pipeline
|
||||
|
||||
defaults:
|
||||
generate:
|
||||
steps: 20
|
||||
cfg: 4.5
|
||||
negative_prompt: "deformed, ugly, bad quality, lowres"
|
||||
edit:
|
||||
steps: 4
|
||||
cfg: 1.0
|
||||
|
||||
stages:
|
||||
# Independent batch: N prompts → N images
|
||||
- id: characters
|
||||
generate:
|
||||
- id: heroine
|
||||
prompt: "1girl, red hair, school uniform, portrait"
|
||||
seed: 42
|
||||
|
||||
# Fan-out cross-product: each input × each prompt
|
||||
- id: expressions
|
||||
edit:
|
||||
input: characters
|
||||
prompts:
|
||||
- id: smile
|
||||
text: "make her smile happily"
|
||||
- id: angry
|
||||
text: "make her look angry"
|
||||
|
||||
# 1:1 passthrough: each input → one output
|
||||
- id: nobg
|
||||
remove_bg:
|
||||
input: expressions
|
||||
|
||||
- id: final
|
||||
upscale:
|
||||
input: nobg
|
||||
scale: 2
|
||||
```
|
||||
|
||||
Output structure (every stage saved):
|
||||
|
||||
```
|
||||
output/my_pipeline/
|
||||
pipeline.json ← summary (stages, timings, skip/done counts)
|
||||
characters/ ← stage 1
|
||||
heroine.png
|
||||
heroine.json
|
||||
expressions/ ← stage 2 (cross-product: {input}_{prompt})
|
||||
heroine_smile.png
|
||||
heroine_angry.png
|
||||
...
|
||||
nobg/ ← stage 3
|
||||
heroine_smile_nobg.png
|
||||
...
|
||||
final/ ← stage 4
|
||||
heroine_smile_nobg_x2.png
|
||||
...
|
||||
```
|
||||
|
||||
| Stage type | Input | Routing |
|
||||
|-----------|-------|---------|
|
||||
| `generate` | none | list of prompts → list of images (1:1 per item) |
|
||||
| `edit` | previous stage | cross-product: each input × each prompt |
|
||||
| `remove_bg` | previous stage | 1:1 passthrough |
|
||||
| `upscale` | previous stage | 1:1 passthrough |
|
||||
|
||||
Resume: if an output file already exists, that item is skipped. Use `--force`
|
||||
to re-run everything. This lets you add items to a stage and re-run without
|
||||
regenerating existing work.
|
||||
|
||||
See `examples/portrait.yaml` and `examples/backgrounds.yaml` for ready-to-use
|
||||
configs.
|
||||
|
||||
### Generate (SDXL text-to-image)
|
||||
|
||||
```bash
|
||||
@@ -137,7 +236,10 @@ Either model can be omitted (`None`) for single-model sessions. Properties:
|
||||
- `vna.close()` — manual cleanup (automatic with `with`)
|
||||
|
||||
The standalone `vnasset generate` and `vnasset edit` CLI commands are thin
|
||||
wrappers around a one-shot session — same API, backwards compatible.
|
||||
wrappers around a one-shot session.
|
||||
|
||||
For bulk workflows, use `vnasset pipeline` instead — it handles the
|
||||
session, outputs, and resume logic declaratively.
|
||||
|
||||
### Background Removal
|
||||
|
||||
@@ -219,48 +321,40 @@ a compilation penalty.
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ VnAssetsSession │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ SDXL │ │ Qwen │ │ Qwen VL │ │ Qwen │ │
|
||||
│ │ UNet │ │ Transf. │ │ 7B TE │ │ VAE │ │
|
||||
│ │ (~3.5GB) │ │ (~20GB) │ │ (~14GB) │ │ (~1GB) │ │
|
||||
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
|
||||
│ │ │ │ │ │
|
||||
│ ▼ │ │ │ │
|
||||
│ ┌─────────┐ │ │ │ │
|
||||
│ │ Generate│ │ │ │ │
|
||||
│ │ │──────────┼───────────────┼──────────────┤ │
|
||||
│ │ │ base.png │ │ │ │
|
||||
│ └─────────┘ │ │ │ │
|
||||
│ │ ▼ ▼ ▼ │
|
||||
│ │ ┌──────────────────────────────────────┐ │
|
||||
│ └──────────► Edit Phase │ │
|
||||
│ │ base.png + prompts[] → variants[] │ │
|
||||
│ └──────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌──────────────────┐ │
|
||||
│ │ base.png │ │
|
||||
│ │ happy.png │ │
|
||||
│ │ sad.png │ │
|
||||
│ │ angry.png │ │
|
||||
│ └────────┬─────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌──────────────────┐ │
|
||||
│ │ Upscale │ │
|
||||
│ │ (Real-ESRGAN) │ │
|
||||
│ │ ~1.8s each │ │
|
||||
│ └────────┬─────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌──────────────────┐ │
|
||||
│ │ Remove BG │ │
|
||||
│ │ (isnet-anime) │ │
|
||||
│ └──────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ VnAssetsSession │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ SDXL │ │ Qwen │ │ Qwen VL │ │ Qwen │ │
|
||||
│ │ UNet │ │ Transf. │ │ 7B TE │ │ VAE │ │
|
||||
│ │ (~3.5GB) │ │ (~20GB) │ │ (~14GB) │ │ (~1GB) │ │
|
||||
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
|
||||
│ │ │ │ │ │
|
||||
│ ▼ ▼ ▼ ▼ │
|
||||
│ ┌────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Pipeline Runner │ │
|
||||
│ │ │ │
|
||||
│ │ stage 1: generate ──► stage 2: edit ──► stage 3: remove_bg │
|
||||
│ │ │ │ │ │
|
||||
│ │ ▼ ▼ ▼ │
|
||||
│ │ characters/ expressions/ nobg/ │
|
||||
│ │ heroine.png heroine_smile.png ..._nobg.png │
|
||||
│ │ heroine.json ... ... │
|
||||
│ │ │ │
|
||||
│ │ ▼ │
|
||||
│ │ stage 4: upscale │
|
||||
│ │ │ │
|
||||
│ │ ▼ │
|
||||
│ │ final/ │
|
||||
│ │ ..._nobg_x2.png │
|
||||
│ └────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌──────────────┐ │
|
||||
│ │ Upscale │ │ Remove BG │ (lazy-loaded on first use) │
|
||||
│ │ Real-ESRGAN │ │ isnet-anime │ │
|
||||
│ │ (~17 MB) │ │ (~176 MB) │ │
|
||||
│ └─────────────┘ └──────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
The Qwen transformer is loaded FP8 → BF16 at construction using
|
||||
@@ -469,6 +563,7 @@ Use `--raw` to bypass weighting and fall back to plain diffusers encoding.
|
||||
|
||||
| Feature | Status |
|
||||
|---------|--------|
|
||||
| `vnasset pipeline` (batch YAML config) | ✅ Working |
|
||||
| `vnasset generate` | ✅ Working |
|
||||
| `vnasset edit` | ✅ Working |
|
||||
| `VnAssetsSession` (persistent models) | ✅ Working |
|
||||
@@ -476,18 +571,13 @@ Use `--raw` to bypass weighting and fall back to plain diffusers encoding.
|
||||
| Lightning LoRA fuse-at-load | ✅ Working |
|
||||
| Flash attention (experimental) | ✅ Working |
|
||||
| `vnasset remove-bg` | ✅ Working |
|
||||
| Session background removal | ✅ Working |
|
||||
| `vnasset upscale` | ✅ Working |
|
||||
| Session upscaling | ✅ Working |
|
||||
| `vnasset pipeline` (batch YAML config) | 🚧 Planned |
|
||||
| `vnasset serve` (daemon/HTTP API) | 🚧 Planned |
|
||||
| `torch.compile` on UNet | 🚧 Planned |
|
||||
| Batch edit loop (shared VAE encode) | 🚧 Planned |
|
||||
|
||||
## Future Improvements
|
||||
|
||||
- **Pipeline batch mode** — `vnasset pipeline --config pipeline.yaml` for
|
||||
generate + multiple edits in one session from a YAML config file.
|
||||
- **`torch.compile` on UNet** — the UNet forward is identical each step; ROCm's
|
||||
`torch.compile` support is maturing and could cut per-step time significantly.
|
||||
- **Shared encode optimization** — for N edit variants of the same input image,
|
||||
|
||||
Reference in New Issue
Block a user