Skip to content

ltm: a repeated dimension in an owner routes to element-level Johnson and enumerates 34,904 circuits that are all declined (150x on an 8-element fixture) #1017

Description

@bpowers

Summary

An owner variable that repeats a dimension (cube[D1,D1,D2]) makes the variable-level cycle
classify as CrossElementOrMixed, which routes it to the element-level Johnson slow path. A repeated
dimension pairs every (i,j) with every (k,l), so a 16-element model produces a near-complete
16-node SCC and Johnson enumerates 34,904 elementary circuits — all of which are subsequently
declined.

Enumeration itself is only ~13 ms. ~93% of the cost is downstream, in model_ltm_variables
processing those 34,904 circuits (rotation dedup, assign_loop_ids sort, partition assignment)
before declining them.

Measured

Controlled A/B where the only difference is whether the owner repeats a dimension. Element counts
are identical in both fixtures:

fixture model_ltm_variables vars / arms slow-path circuits largest SCC
cube[D1,D1,D2] / pop[D1,D1] (square) 191–224 ms 93 / 93 34,904 16
cube[D1,D3,D2] / pop[D1,D3] (rectangular) 1.4 ms 109 / 109 0 0
square, discovery mode 1.1 ms 84 / 84

~150x apart, and the square variant emits fewer variables and arms than the rectangular one.
That rules out equation generation as the cause — the obvious first hypothesis — and is why perf
shows no hot loop: the work is 34,904 circuits × a few small allocations and salsa query-edge
inserts, diffuse by construction. The observed signature is diffuse string interning plus salsa
edge inserts with nothing above 2.6%.

Exhaustive mode only. Discovery mode never calls the tiered enumeration, which is why the same
fixture is 1.1 ms there.

Why the existing guard does not catch it

MAX_LTM_SCC_NODES gates on SCC node count. The blowup is in circuit count, and the two are
only loosely related — a dense n-node SCC has worst-case ~(n-1)! elementary circuits. 16 nodes sails
through the node gate, and 34,904 circuits evidently does not trip MAX_LTM_CIRCUITS either.

A node-count gate cannot bound this in general: the quantity that needs bounding is the one being
enumerated.

Impact

  • Test-suite: ltm_array_agg::square_owner_whole_rhs_declines_and_skips_loudly runs 1.98 s solo
    on an 8-element fixture, against 0.2–0.3 s for siblings using the same helper.
  • Models: a repeated dimension is an ordinary modelling shape, and the cost is superlinear in
    the dimension's element count, so a moderately larger square owner is far worse than 200 ms.

Why this is not a perf tweak

Any fix — a density-aware auto-flip, a circuit-count cap, or short-circuiting the decline before
per-circuit processing — changes which loops are found on some models. That is a behaviour
change requiring loop-discovery validation, not an optimization that can ride a perf branch.

One lead, unverified: the decline verdict may be reachable before the 34,904 circuits are
individually processed, which would recover ~93% of the cost without changing enumeration semantics
at all. Nobody has checked whether the decline is actually decidable that early — treat this as a
direction to investigate, not a finding.

Components

  • src/simlin-engine/src/ltm/graph.rs — Johnson enumeration, SCC construction
  • src/simlin-engine/src/db/analysis.rsmodel_loop_circuits_tiered, the CrossElementOrMixed
    classification and auto-flip gate
  • src/simlin-engine/src/db/ltm/mod.rsmodel_ltm_variables circuit post-processing (rotation
    dedup, assign_loop_ids, partition assignment) — where ~93% of the time goes

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    ltmLoops that Matter (LTM) analysis subsystem

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions