docs: add optimization-decisions analysis, update README future plans
Detail the shared-encode and torch.compile tradeoffs with data-driven estimates from the pipeline code. Shared encode saves ~1.2-2.3s per variant (2-4% total) and is deferred; torch.compile needs a 30-minute ROCm spike before deciding. Update Current State and Future Improvements to reflect the analysis.
This commit is contained in:
21
README.md
21
README.md
@@ -586,15 +586,22 @@ Use `--raw` to bypass weighting and fall back to plain diffusers encoding.
|
||||
| `vnasset upscale` | ✅ Working |
|
||||
| Self-contained torch wheel | ✅ Working |
|
||||
| `vnasset serve` (daemon/HTTP API) | 🚧 Planned |
|
||||
| `torch.compile` on UNet | 🚧 Planned |
|
||||
| Batch edit loop (shared VAE encode) | 🚧 Planned |
|
||||
| `torch.compile` on UNet | 🔬 Needs spike (see [decisions](docs/optimization-decisions.md)) |
|
||||
| Shared encode optimization | 📋 Analyzed, deferred (see [decisions](docs/optimization-decisions.md)) |
|
||||
|
||||
## Future Improvements
|
||||
|
||||
- **`torch.compile` on UNet** — the UNet forward is identical each step; ROCm's
|
||||
`torch.compile` support is maturing and could cut per-step time significantly.
|
||||
- **Shared encode optimization** — for N edit variants of the same input image,
|
||||
run VAE encode and VL visual token encoding once, then only text-encode and
|
||||
denoise per variant.
|
||||
- **`vnasset serve`** — lightweight daemon with Unix socket or HTTP API for
|
||||
integrating VNAsset into external tools.
|
||||
- **`torch.compile` on UNet** — the UNet forward is identical each step, but
|
||||
ROCm `torch.compile` maturity is uncertain. Needs a 30-minute spike to measure
|
||||
actual speedup and check for graph breaks / crashes on RDNA 3.5. If the spike
|
||||
shows 15%+ and stable, implement with a warmup step at session load.
|
||||
- **Shared encode optimization** — for N edit variants of the same input image,
|
||||
saves ~1.2–2.3s per variant (VAE encode + vision encoder). The dominant cost
|
||||
(transformer denoising, ~45–50s) cannot be shared, so the overall gain is
|
||||
2–4%. Deferred until workload shape demands it (100+ variants). When built,
|
||||
the right place is the pipeline runner, not the session API.
|
||||
|
||||
See [`docs/optimization-decisions.md`](docs/optimization-decisions.md) for
|
||||
full tradeoff analysis with data and cost estimates.
|
||||
|
||||
Reference in New Issue
Block a user