docs: add optimization-decisions analysis, update README future plans

Detail the shared-encode and torch.compile tradeoffs with data-driven
estimates from the pipeline code. Shared encode saves ~1.2-2.3s per
variant (2-4% total) and is deferred; torch.compile needs a 30-minute
ROCm spike before deciding. Update Current State and Future Improvements
to reflect the analysis.
This commit is contained in:
Michele Rossi
2026-07-08 16:10:46 +02:00
parent 1ab1fee15d
commit 9c4474b93c
2 changed files with 183 additions and 7 deletions

View File

@@ -586,15 +586,22 @@ Use `--raw` to bypass weighting and fall back to plain diffusers encoding.
| `vnasset upscale` | ✅ Working |
| Self-contained torch wheel | ✅ Working |
| `vnasset serve` (daemon/HTTP API) | 🚧 Planned |
| `torch.compile` on UNet | 🚧 Planned |
| Batch edit loop (shared VAE encode) | 🚧 Planned |
| `torch.compile` on UNet | 🔬 Needs spike (see [decisions](docs/optimization-decisions.md)) |
| Shared encode optimization | 📋 Analyzed, deferred (see [decisions](docs/optimization-decisions.md)) |
## Future Improvements
- **`torch.compile` on UNet** — the UNet forward is identical each step; ROCm's
`torch.compile` support is maturing and could cut per-step time significantly.
- **Shared encode optimization** — for N edit variants of the same input image,
run VAE encode and VL visual token encoding once, then only text-encode and
denoise per variant.
- **`vnasset serve`** — lightweight daemon with Unix socket or HTTP API for
integrating VNAsset into external tools.
- **`torch.compile` on UNet** — the UNet forward is identical each step, but
ROCm `torch.compile` maturity is uncertain. Needs a 30-minute spike to measure
actual speedup and check for graph breaks / crashes on RDNA 3.5. If the spike
shows 15%+ and stable, implement with a warmup step at session load.
- **Shared encode optimization** — for N edit variants of the same input image,
saves ~1.22.3s per variant (VAE encode + vision encoder). The dominant cost
(transformer denoising, ~4550s) cannot be shared, so the overall gain is
24%. Deferred until workload shape demands it (100+ variants). When built,
the right place is the pipeline runner, not the session API.
See [`docs/optimization-decisions.md`](docs/optimization-decisions.md) for
full tradeoff analysis with data and cost estimates.