vnassets/AGENTS.md

# Data-Oriented Design — Operating Rules

These are operating rules, not philosophy:
every rule here tells you what to *do*. Approach every problem — code, plan,
pipeline, document — by understanding the real data first, then designing the
simplest machine that transforms the input you actually have into the output
you actually need, at a cost you can state. Decide from facts and measurement,
not habit, analogy, or dogma.

## Scope, tiers, and precedence

Scale the ceremony to the task. Decide the tier first; when unsure, pick the
higher tier and say which you picked.

- **Tier 0 — trivial.** Typo fixes, mechanical edits, one-line bugfixes,
  answering questions. Apply the defaults silently (naming, explicit error
  behavior, no speculative generality). No written plan or checklist.
- **Tier 1 — non-trivial change.** New function or feature, behavior change,
  anything that touches a data layout, contract, or interface. Required:
  answer the framing and data questions in a short written plan *before*
  implementing, run the simplification pass, and run the final self-check.
- **Tier 2 — subsystem-scale.** New or substantially reworked subsystem,
  pipeline, or tool. Everything in tier 1 plus the enforceable deliverables.

Precedence when rules conflict:

1. An explicit instruction from the user for the current task.
2. This document.
3. Existing codebase or workflow convention.

When this document conflicts with existing convention and complying would
mean a large refactor, do not silently rewrite and do not silently conform:
state the conflict, estimate the cost of each option, and propose the
smallest compliant change.

## Defaults to reject

These are the three default beliefs that produce bad solutions. Each comes
with the replacement behavior — do the replacement, every time:

1. **"The tools are the platform."** Reality is the platform: the actual
   hardware, organization, deadline, physics. *Do instead:* before designing,
   name the real platform and the 2–3 of its fixed properties that constrain
   this solution, and design within them.
2. **"Design around a model of the world."** World models (objects, metaphors,
   idealized categories) hide the actual data and the actual cost. *Do
   instead:* design around the data. Do not introduce an abstraction until
   you can describe, concretely, the data it organizes and the transform it
   serves — and what the abstraction costs.
3. **"The solution matters more than the data."** The only purpose of any
   solution is to transform data from one form to another. *Do instead:*
   start every task from the actual inputs and required outputs, never from
   the machinery you'd like to build.

## Core defaults (any problem)

- **The problem is the data.** Before proposing any solution, describe the
  input and output concretely. If you can't, getting that description *is*
  the first task — do it before anything else.
- **State the cost.** Every design recommendation you make must state its
  cost (time, memory, complexity, maintenance) and on what platform that
  cost is paid. A recommendation without a cost is a guess — don't deliver
  guesses unlabeled.
- **Solve only the problem you have.** Different data is a different problem.
  Concretely: do not add parameters, options, abstraction layers, or
  extension points for hypothetical future needs. If you're tempted, write
  the one-line note of what you *didn't* build and why, and move on.
- **Where there is one, there are many.** Anything that happens once almost
  always happens many times — across space or across the time axis. Default
  every design to the batch; treat the single case as a batch of size one.
- **The common case dominates.** Identify the most common case explicitly and
  design the straight-line path for it. Handle rare and error cases, but
  outside that path — a "maybe" checked everywhere is an "always."
- **Exploit every constraint you have.** List the known constraints (ranges,
  volumes, rates, invariants) and use them to remove work. Do not discard a
  constraint to make the solution "more general" — that generality is a cost
  paid forever for a benefit nobody asked for.
- **Simplicity is removing work.** Prefer fewer states, fewer steps, fewer
  special cases, fewer moving parts. Every added state or branch must be
  carried, tested, and explained — count them as cost.
- **"Can't be done" is a cost claim.** When something seems impossible, what
  is almost always true is that it costs more than it's worth. Say that, with
  the estimate, so the tradeoff can actually be decided.

## Get the real data (required before designing)

You cannot observe data you were not given — so observe what you *can*, and
label everything else:

- **Inspect before assuming.** Read representative input files, sample actual
  values, read the actual call sites, run the code on real input when a way
  to do so exists. Do not design from the type signatures or the docs alone.
- **Sample the data you already have — instrument the live solution.** The
  richest data is usually already flowing through the current code; go get it.
  Temporarily dump a representative sample of what actually moves through the
  system: the arguments reaching a function, the values a hot variable takes,
  what a function returns, which branch is taken, the real sizes/counts/lengths.
  Then *analyze the sample* — histogram it, sort it, count distinct values, look
  at the min/max/mode — and hunt for patterns. Real distributions expose what
  the types hide: a variable that is almost always one value, an "array" that is
  usually length 0 or 1, input that arrives already sorted or already unique, a
  "general" path that is one specific case 98% of the time, a result that is
  constant across a run. Each such pattern is a concrete opportunity — specialize
  the common case, skip the dead branch, hoist the invariant, precompute the
  constant, size the structure to what actually occurs. And the pattern can be
  bigger than a local tweak: the data's *shape* can show that a **different
  algorithm or representation is the better-fit machine** (sorted-enough → a
  different sort/merge; skewed → a different code; runny → a run/stream form;
  sparse → a different container), not just that the current machine needs
  filing. Sampling justifies *replacing* the machine, not only trimming it.
  Sampling is also how you find *new* opportunities mid-optimization, not just
  before starting: when a pass **stalls or plateaus**, that is the signal to
  re-sample the hottest stage's data and ask whether a different machine fits it
  better — not to keep filing the current one. Add the probes, run on real
  input, read the output, then remove them — never leave instrumentation on a
  timed/measured path.
- **Label every assumption.** For each fact you need but cannot observe,
  write an explicit line — `ASSUMPTION: <fact> — affects <decision>` — in
  your plan, and prefer designs that are cheap to revisit if the assumption
  is wrong. Ask the user only when the answer materially changes the design.
- **Never fabricate.** Do not invent plausible-looking values, distributions,
  or measurements and treat them as real.

Answer these about the data (in the tier 1+ plan):

1. What does the input actually look like — shape, volume, source?
2. What are the most common real values, and how are they distributed?
3. What are the acceptable ranges, and what happens when out-of-range data
   arrives?
4. What is the frequency of change — what is stable, what is volatile?
5. What does the solution read and where does it come from? What does it
   write and where is it used? What does it touch that it doesn't need?

## Method (tier 1+ — show this work as a short plan, a line or two per step)

1. **Frame it.** What is the problem, why is it worth solving, where is the
   limit beyond which it isn't, and what is plan B?
2. **Get the data** (section above).
3. **State the cost** of the dominant transform on the real platform.
4. **Design the transform**: a sequence or DAG of explicit transformations —
   what comes in, what goes out, what each step is responsible for, with
   explicit contracts (shape, meaning, ownership, lifetime, valid ranges) at
   each boundary.
5. **Run the simplification pass** (below); say which questions applied and
   what work they removed.
6. **Define done.** State the success criteria and what evidence would prove
   the approach wrong, before building.
7. **Verify.** Check the result against the real data and the stated
   criteria, and report what was and wasn't verified.

## Simplification pass (run recursively on every sub-problem)

1. Can we **not do this at all**?
2. Can we do this **only once** (precompute, cache, amortize)?
3. Can we do this **fewer times**?
4. Can we **approximate** the result so that no one notices the difference?
5. Can we use a **small lookup table**?
6. Can we use a **large lookup table**?
7. Can we use a **small buffer/FIFO** to decouple producer from consumer?
8. Can we **constrain the problem further** so a simpler machine suffices?
9. Is there a **different algorithm or representation that fits the data better**
   than the current machine? Subtraction has a floor; when filing the current
   approach stops paying (a plateau), the win is often a *different* machine the
   data's shape points to — reconsider the approach, don't only shrink it.

## Design rules

- **Minimize states and branches by design**, not by adding checks. Where the
  data genuinely varies, partition it by case and handle each partition
  straight-line, rather than re-deciding the case per element.
- **Out-of-range and error behavior is always explicit** — clamp, reject,
  drop, or fail loudly; chosen deliberately and written down. Never leave
  undefined behavior as an implicit policy, in any tier.
- **Complexity requires evidence.** Add complexity only against a real,
  observed need — never a hypothetical one.

## Performance claims

- **Never assert an unmeasured performance result.** Not "this should be
  faster," not invented numbers.
- If a way to measure exists (benchmark, profiler, test harness, counters),
  measure, and include before/after numbers with the change.
- If no way to measure exists here, label the change **unverified**, state
  the expected effect as a hypothesis, and specify the exact measurement
  that would verify it.
- If there is no measurable performance requirement, build the simplest
  correct design and skip speculative optimization entirely.

---

# Software specifics (systems, engine, embedded, game)

The rules above apply to any problem. These are their conclusions for
software, where the hardware is unforgiving and the data volumes are real.

## Batch-first transforms (plural by default)

- Write transforms to operate on **batches/arrays** by default, named in the
  **plural** (`update_things`, not `update_thing`).
- A singular call is a degenerate batch: the same batch path with
  `count = 1`. Do not maintain separate singular logic without a proven,
  measured need.
- Exception: true singletons (configuration state, a single shared resource).
  Taking the exception requires a written note: why the data is genuinely
  singular and batch semantics don't apply.

## Memory, layout, and access

- **Indices over pointers/references/handles by default** (index into a
  contiguous array or table). Any pointer-heavy hot path must include a
  short written justification for why indices are insufficient.
- Organize data by **access pattern, not conceptual ownership**. Split hot
  and cold fields when the cold fields aren't needed in the dominant loop.
- For each hot path, write down the expected **access pattern**
  (linear / strided / random), expected **branch behavior**
  (predictable / unpredictable), and the hardware assumptions.
- When branch entropy is high, prefer **partitioned passes** (bucket by
  state/tag, process each bucket straight-line) over per-element branching.
- Keep the common-case path branch-minimal; rare and error handling lives
  outside the hot loop.

## Data protocols between systems

Systems communicate through **explicit data protocols**, modeled after
network protocols and file formats — explicit layout, versioning, documented
meaning. The default is a **flat struct**: fixed layout, no hidden pointers,
no OO-style interfaces. Use tagged unions or header-plus-payload when the
flat struct genuinely can't express it. Do not model system boundaries as
objects, virtual calls, or opaque handles.

## Hardware is the platform

- Design with the actual hardware's properties — cache hierarchy, memory
  bandwidth, alignment, latency vs. throughput — and to its strengths.
- **Latency and throughput are only the same thing in a sequential system.**
  For every performance requirement, identify which one it actually is
  before designing for it.
- The compiler and language are tools, not magic: memory layout, access
  order, and the choice of what work to do at all are your job, not theirs —
  and they are roughly 90% of the problem. Know what the compiler can
  reasonably do with what you wrote, and don't delegate what it can't.

## Enforceable deliverables (tier 2)

For each new or substantially reworked subsystem:

- One explicit **batch transform contract**: input layout, output layout,
  owner, lifetime, valid value ranges.
- A **plural/batch path** for every transform; singular calls are thin
  wrappers over the batch implementation (`count = 1`) unless documented as
  a true singleton.
- A written **justification for any pointer/reference/handle-heavy hot path**
  explaining why index-based access is insufficient.
- Explicit **out-of-range behavior** (clamp/reject/drop/error) at every
  input boundary.
- Unresolved design questions filed as **local issue files under `issues/`**
  — not GitHub issues, not inline TODOs.

---

# Final self-check (run before delivering tier 1+ work)

Verify, and fix or flag anything that fails:

- [ ] The plan answered the framing, data, and cost questions — or every gap
      is labeled `ASSUMPTION` with what it affects.
- [ ] The most common case is identified and the design serves it
      straight-line; rare/error cases are out of the common path.
- [ ] The simplification pass ran; the work it removed (or why nothing could
      be removed) is stated.
- [ ] No speculative generality: no parameter, option, or abstraction exists
      for a need that isn't real yet.
- [ ] Out-of-range and error behavior is explicit at every boundary.
- [ ] Transforms are plural/batch, or the singleton exception is documented.
- [ ] Pointer-heavy hot paths carry their written justification; everything
      else uses indices.
- [ ] No unmeasured performance claim anywhere in code, comments, or
      summary; measurements included where possible, hypotheses labeled
      where not.
- [ ] Done-criteria from the plan were checked, and the summary reports what
      was verified and what wasn't.
- [ ] (Tier 2) Deliverables above are present; open questions are filed
      under `issues/`.

---

# Commit

When user ask to commit, use mitchellh style commit.