Nim/lib/system/yrc_proof.lean

/-
  YRC Safety Proof — lock-free SATB collector with parallel collections
  =====================================================================
  Self-contained, no Mathlib. Checked with Lean 4 (v4.32.0).

  Formal model of the safety arguments behind lib/system/yrc.nim in its
  current form: lock-free write barrier, optimistic capture / validate /
  commit, and up to `MaxPar` concurrent collections over disjoint
  CAS-claimed partitions.

  ## What the implementation does (the things we model)

  Write barrier `nimAsgnYrc(dest, src)`:
    1. direct ATOMIC incRef of src        (rc word mutation, visible to all)
    2. atomicExchange dest ← src          (graph is immediately current;
                                           old value read atomically)
    3. buffer dec(old) in a striped queue (deferred — this queue IS the
                                           snapshot-at-the-beginning log)

  A collection (any mutator thread can become a collector):
    1. merge queues into rc words, steal candidate roots  (under gMergeLock)
    2. CAPTURE: Tarjan SCC traversal; each visited cell is claimed by
       CAS-ing a collection tag into its spare header word (claimCell);
       cells claimed by another ACTIVE collection are not traversed
       (claimCell → -1, deferred via crossPend)
    3. compute deadness per SCC: ext(S) = sumRefs − internal − deadIn
    4. VALIDATE at commit: an SCC is freed only if no queue entry mentions
       a member (dirty check) and every member's rc word is unchanged
       since capture (recheck) — validateDead
    5. COMMIT: nil all slots of dead cells, trialDec edges to survivors,
       wait out concurrent captures (grace period), then free — commitDead

  ## Proof structure

  §1  Heap model, reachability, the core safety theorem.
  §2  Write barrier: no lost objects.
  §3  Mutator operational semantics and GARBAGE STABILITY: a closed
      (externally unreferenced) set stays closed under every mutator step,
      allocation, and foreign frees. This is why optimistic
      capture/validate/commit is sound and why aborts cost nothing.
  §4  Commit validation arithmetic: validated ext(D) = 0 implies D is
      closed; corollary CROSS-TARGET LIVENESS — a cell referenced from
      outside a collection's partition is never freed by that collection.
  §5  Tag uniqueness and partition disjointness for parallel collections.
  §6  Grace period: no capture ever dereferences a freed cell.
  §7  The asymmetric seq/GC fence (seqs_v2.nim): Dekker-style mutual
      exclusion between seq structure mutations and collections.
  §8  Deadlock freedom for the remaining locks (gMergeLock + stripes) and
      the spin-wait ordering argument.
-/

-- Objects and threads are just natural numbers for simplicity.
abbrev Obj := Nat
abbrev Thread := Nat

/-! ## §1 Heap model and reachability -/

/-- The state of the heap at a point in time. -/
structure State where
  /-- Physical heap edges: `edges x y` means object `x` has a ref field
      pointing to `y`. Always up-to-date (atomic stores/exchanges). -/
  edges : Obj → Obj → Prop
  /-- Stack roots per thread: local variables and the shared candidate
      roots buffer (both are "external" to any captured subgraph). -/
  roots : Thread → Obj → Prop
  /-- Live allocations. The allocator hands out only unallocated objects;
      captured cells stay allocated until their collection frees them. -/
  allocated : Obj → Prop

/-- An object is *reachable* if some thread can reach it via stack roots
    plus heap edges. -/
inductive Reachable (s : State) : Obj → Prop where
  | root (t : Thread) (x : Obj) : s.roots t x → Reachable s x
  | step (x y : Obj) : Reachable s x → s.edges x y → Reachable s y

/-- Directed reachability following physical heap edges only. -/
inductive HeapReachable (s : State) : Obj → Obj → Prop where
  | refl (x : Obj) : HeapReachable s x x
  | step (x y z : Obj) : HeapReachable s x y → s.edges y z → HeapReachable s x z

theorem heapReachable_of_reachable (s : State) (r x : Obj)
    (hr : Reachable s r) (hp : HeapReachable s r x) :
    Reachable s x := by
  induction hp with
  | refl => exact hr
  | step _ _ _ hedge ih => exact Reachable.step _ _ ih hedge

/-- An object has an *external reference* if some thread points to it. -/
def hasExternalRef (s : State) (x : Obj) : Prop :=
  ∃ t, s.roots t x

/-- Externally anchored: heap-reachable from an externally referenced
    object. This is what deadness computation + survivor rescue computes. -/
def anchored (s : State) (x : Obj) : Prop :=
  ∃ r, hasExternalRef s r ∧ HeapReachable s r x

/-- The collector frees `x` only if `x` is not anchored. -/
def collectorFrees (s : State) (x : Obj) : Prop :=
  ¬ anchored s x

/-- Every reachable object is anchored. -/
theorem reachable_is_anchored (s : State) (x : Obj)
    (h : Reachable s x) : anchored s x := by
  induction h with
  | root t x hroot =>
    exact ⟨x, ⟨t, hroot⟩, HeapReachable.refl x⟩
  | step a b _ h_edge ih =>
    obtain ⟨r, h_ext_r, h_path_r_a⟩ := ih
    exact ⟨r, h_ext_r, HeapReachable.step r a b h_path_r_a h_edge⟩

/-- **Core Safety Theorem**: freed objects are unreachable. -/
theorem yrc_safety (s : State) (x : Obj)
    (h_freed : collectorFrees s x) : ¬ Reachable s x := by
  intro h_reach
  exact h_freed (reachable_is_anchored s x h_reach)

/-! ## §2 The write barrier

  `nimAsgnYrc` performs the atomic inc of `src` BEFORE the exchange, so
  there is no instant at which the edge `a → src` exists without src's rc
  accounting for it; and the exchange reads `old` atomically, so two
  racing barriers on the same slot can never both dec the same old value.
  The dec of `old` is deferred: until the next merge, old's rc is merely
  inflated — always conservative. -/

/-- Model of `nimAsgnYrc(field a, src)`: `a`'s field pointed to `old`,
    now points to `src`. The graph update is immediate (atomicExchange). -/
def writeBarrier (s : State) (a old src : Obj) : State :=
  { s with
    edges := fun x y =>
      if x = a ∧ y = src then True
      else if x = a ∧ y = old then False
      else s.edges x y }

/-- Overwriting a slot with nil: only removes an edge. -/
def storeNil (s : State) (a old : Obj) : State :=
  { s with
    edges := fun x y =>
      if x = a ∧ y = old then False else s.edges x y }

/-- **No Lost Object**: if thread `t` holds `a` and stores `a.f = b`,
    then `b` is reachable afterwards — the exchange publishes the edge
    atomically, so a concurrent collection's survivor rescue traces it. -/
theorem no_lost_object (s : State) (t : Thread) (a old b : Obj)
    (h_root_a : s.roots t a) :
    Reachable (writeBarrier s a old b) b := by
  apply Reachable.step a b
  · exact Reachable.root t a h_root_a
  · simp [writeBarrier]

/-! ## §3 Mutator semantics and garbage stability

  The heart of optimistic capture/validate/commit is the *garbage
  stability theorem*: a set with no external references cannot acquire
  one later, because mutators can only copy references they can reach.
  Hence a dead set that VALIDATES at commit time stays dead through the
  grace window and until the actual `free` calls — no re-validation is
  needed, and an aborted (dirty) capture merely wasted its own work.

  Every constructor's precondition encodes the fundamental capability
  restriction: to use a reference you must hold it. `P` is the set of
  cells protected from foreign frees (in yrc: cells stamped with an
  active tag are never freed by another collection — §5). -/

def addRoot (s : State) (t : Thread) (x : Obj) : State :=
  { s with roots := fun t' y => (t' = t ∧ y = x) ∨ s.roots t' y }

def delRoot (s : State) (t : Thread) (x : Obj) : State :=
  { s with roots := fun t' y => if t' = t ∧ y = x then False else s.roots t' y }

def allocObj (s : State) (t : Thread) (x : Obj) : State :=
  { s with
    roots := fun t' y => (t' = t ∧ y = x) ∨ s.roots t' y
    allocated := fun y => y = x ∨ s.allocated y }

def freeObj (s : State) (x : Obj) : State :=
  { s with
    edges := fun u v => if u = x ∨ v = x then False else s.edges u v
    allocated := fun y => if y = x then False else s.allocated y }

/-- One step of the concurrent system, as seen by a fixed observer
    protecting the cell set `P`. -/
inductive MutStep (P : Obj → Prop) (s : State) : State → Prop where
  /-- `a.f = src`: the mutator must hold refs to `a` and `src`. -/
  | write (a old src : Obj)
      (ha : Reachable s a) (hsrc : Reachable s src) :
      MutStep P s (writeBarrier s a old src)
  /-- `a.f = nil`. -/
  | writeNil (a old : Obj) (ha : Reachable s a) :
      MutStep P s (storeNil s a old)
  /-- Copy a reachable ref into a local / the roots buffer. -/
  | rootCopy (t : Thread) (x : Obj) (hx : Reachable s x) :
      MutStep P s (addRoot s t x)
  /-- Drop a local ref (scope exit, roots-buffer unregistration). -/
  | rootDrop (t : Thread) (x : Obj) :
      MutStep P s (delRoot s t x)
  /-- Allocate: the allocator returns only unallocated addresses. -/
  | alloc (t : Thread) (x : Obj) (hfresh : ¬ s.allocated x) :
      MutStep P s (allocObj s t x)
  /-- A DIFFERENT collection frees one of its own dead cells: it is
      unreachable (its own §1 safety) and not protected (§5 partition
      disjointness: it carries the other collection's tag, not ours). -/
  | foreignFree (x : Obj) (hunreach : ¬ Reachable s x) (hprot : ¬ P x) :
      MutStep P s (freeObj s x)

/-- Reflexive-transitive closure: an arbitrary interleaving of steps by
    all mutators and all other collections. -/
inductive MutSteps (P : Obj → Prop) (s : State) : State → Prop where
  | refl : MutSteps P s s
  | tail {s' s'' : State} :
      MutSteps P s s' → MutStep P s' s'' → MutSteps P s s''

/-- `S` is *closed*: no thread points into it and no heap edge enters it
    from outside. This is exactly "validated dead set" (§4). -/
def closed (s : State) (S : Obj → Prop) : Prop :=
  (∀ t x, S x → ¬ s.roots t x) ∧
  (∀ u v, S v → s.edges u v → S u)

/-- Members of a closed set are unreachable. -/
theorem closed_unreachable (s : State) (S : Obj → Prop)
    (h : closed s S) : ∀ x, Reachable s x → ¬ S x := by
  intro x hr
  induction hr with
  | root t x hroot => exact fun hS => h.1 t x hS hroot
  | step a b _ hedge ih => exact fun hS => ih (h.2 a b hS hedge)

/-- The invariant carried through the grace window: `S` closed and all
    members still allocated (their memory has not been reused). -/
def DeadInv (s : State) (S : Obj → Prop) : Prop :=
  closed s S ∧ ∀ x, S x → s.allocated x

/-- **One-step stability**: no single action of any mutator, allocator or
    other collection can break the invariant of a closed set. -/
theorem step_preserves_deadInv (s s' : State) (S : Obj → Prop)
    (hinv : DeadInv s S) (hstep : MutStep S s s') : DeadInv s' S := by
  obtain ⟨hcl, halloc⟩ := hinv
  cases hstep with
  | write a old src ha hsrc =>
    refine ⟨⟨fun t x hS hroot => hcl.1 t x hS hroot, ?_⟩, fun x hS => halloc x hS⟩
    intro u v hSv hedge
    simp only [writeBarrier] at hedge
    by_cases h1 : u = a ∧ v = src
    · exact absurd (h1.2 ▸ hSv) (closed_unreachable s S hcl src hsrc)
    · by_cases h2 : u = a ∧ v = old
      · -- corner case old = src: the "remove old" branch is overridden
        -- by the "add src" branch, so the edge survives — but then
        -- v = old = src is reachable, hence not in S
        simp [h2] at hedge
        have hSsrc : S src := by rw [← hedge, ← h2.2]; exact hSv
        exact absurd hSsrc (closed_unreachable s S hcl src hsrc)
      · simp [h1, h2] at hedge
        exact hcl.2 u v hSv hedge
  | writeNil a old ha =>
    refine ⟨⟨fun t x hS hroot => hcl.1 t x hS hroot, ?_⟩, fun x hS => halloc x hS⟩
    intro u v hSv hedge
    simp only [storeNil] at hedge
    by_cases h2 : u = a ∧ v = old
    · simp [h2] at hedge
    · simp [h2] at hedge
      exact hcl.2 u v hSv hedge
  | rootCopy t x hx =>
    refine ⟨⟨?_, fun u v hSv hedge => hcl.2 u v hSv hedge⟩, fun y hS => halloc y hS⟩
    intro t' y hSy hroot
    simp only [addRoot] at hroot
    cases hroot with
    | inl h => exact absurd (h.2 ▸ hSy) (closed_unreachable s S hcl x hx)
    | inr h => exact hcl.1 t' y hSy h
  | rootDrop t x =>
    refine ⟨⟨?_, fun u v hSv hedge => hcl.2 u v hSv hedge⟩, fun y hS => halloc y hS⟩
    intro t' y hSy hroot
    simp only [delRoot] at hroot
    by_cases h : t' = t ∧ y = x
    · simp [h] at hroot
    · simp [h] at hroot
      exact hcl.1 t' y hSy hroot
  | alloc t x hfresh =>
    refine ⟨⟨?_, fun u v hSv hedge => hcl.2 u v hSv hedge⟩, ?_⟩
    · intro t' y hSy hroot
      simp only [allocObj] at hroot
      cases hroot with
      | inl h => exact hfresh (h.2 ▸ halloc y hSy)
      | inr h => exact hcl.1 t' y hSy h
    · intro y hS
      simp only [allocObj]
      exact Or.inr (halloc y hS)
  | foreignFree x hunreach hprot =>
    refine ⟨⟨fun t y hSy hroot => hcl.1 t y hSy hroot, ?_⟩, ?_⟩
    · intro u v hSv hedge
      simp only [freeObj] at hedge
      by_cases h : u = x ∨ v = x
      · simp [h] at hedge
      · simp [h] at hedge
        exact hcl.2 u v hSv hedge
    · intro y hSy
      simp only [freeObj]
      have hyx : ¬ y = x := fun he => hprot (he ▸ hSy)
      simp [hyx]
      exact halloc y hSy

/-- **Garbage Stability Theorem**: once a set is closed, it stays closed
    (and unreusable) under any interleaving of concurrent activity. -/
theorem deadInv_stable (s s' : State) (S : Obj → Prop)
    (hinv : DeadInv s S) (hsteps : MutSteps S s s') : DeadInv s' S := by
  induction hsteps with
  | refl => exact hinv
  | tail _ hstep ih => exact step_preserves_deadInv _ _ S ih hstep

/-- Snapshot garbage cannot be resurrected: members of a set that was
    closed at commit time are unreachable at every later point. -/
theorem garbage_stability (s s' : State) (S : Obj → Prop)
    (hinv : DeadInv s S) (hsteps : MutSteps S s s') :
    ∀ x, S x → ¬ Reachable s' x := by
  intro x hS hr
  exact closed_unreachable s' S (deadInv_stable s s' S hinv hsteps).1 x hr hS

/-- **Commit-then-free safety**: if the dead set validated (was closed)
    at commit time, then freeing its members after ANY amount of further
    concurrent activity (the grace window, other collections' frees,
    destructor-driven mutations) satisfies the §1 free condition. -/
theorem commit_free_safe (s s' : State) (S : Obj → Prop)
    (hinv : DeadInv s S) (hsteps : MutSteps S s s') :
    ∀ x, S x → collectorFrees s' x := by
  intro x hS hanch
  obtain ⟨r, ⟨t, hroot⟩, hpath⟩ := hanch
  have hr : Reachable s' x :=
    heapReachable_of_reachable s' r x (Reachable.root t r hroot) hpath
  exact closed_unreachable s' S (deadInv_stable s s' S hinv hsteps).1 x hr hS

/-! ## §4 Commit validation arithmetic

  `computeDeadness` marks an SCC dead when
      ext(S) = sumRefs(S) − internal(S) − deadIn(S) = 0,
  i.e. summed over the whole dead set D (union of dead SCCs):
      Σ_{c∈D} rc(c) = #(edges within D).
  `validateDead` then establishes that the captured rc words are the
  COMMIT-TIME rc values (rc recheck) and that no unmerged queue entry
  mentions a member (dirty check via markDirtyFromQueues — the deferred
  dec queues double as the SATB log; direct incs are atomic rc mutations
  caught by the recheck). Under yrc's invariant "rc counts every
  reference: heap slots, stack refs, and the roots-buffer flag" (the
  roots-buffer refs are excluded by clearing inRootsFlag on the slice
  BEFORE computeDeadness — collectCyclesImpl), we get: every member's rc
  splits into internal references (from D) and external ones, and the
  totals matching forces every external count to zero. -/

theorem sum_map_split (l : List Obj) (f g h : Obj → Nat)
    (hp : ∀ c, c ∈ l → f c = g c + h c) :
    (l.map f).sum = (l.map g).sum + (l.map h).sum := by
  induction l with
  | nil => simp
  | cons a l ih =>
    have ha : f a = g a + h a := hp a (by simp)
    have ih' := ih (fun c hc => hp c (List.mem_cons_of_mem a hc))
    simp only [List.map_cons, List.sum_cons]
    omega

theorem sum_zero_all (l : List Nat) (h : l.sum = 0) :
    ∀ x, x ∈ l → x = 0 := by
  induction l with
  | nil => intro x hx; cases hx
  | cons a l ih =>
    simp only [List.sum_cons] at h
    intro x hx
    cases List.mem_cons.mp hx with
    | inl he => subst he; omega
    | inr hm => exact ih (by omega) x hm

/-- **Validation soundness (arithmetic)**: if every member's commit-time
    rc splits as internal + external, and the collector's check
    Σ rc = Σ internal passed, then no member has any external ref. -/
theorem validated_no_external
    (members : List Obj) (rc inD extIn : Obj → Nat)
    (h_exact : ∀ c, c ∈ members → rc c = inD c + extIn c)
    (h_check : (members.map rc).sum = (members.map inD).sum) :
    ∀ c, c ∈ members → extIn c = 0 := by
  have hsplit := sum_map_split members rc inD extIn h_exact
  have hzero : (members.map extIn).sum = 0 := by omega
  intro c hc
  exact sum_zero_all _ hzero (extIn c) (List.mem_map_of_mem hc)

/-- **Validated implies closed**: bridging the counts to the graph. The
    two counting premises say what `extIn` MEANS: any stack/root ref and
    any heap edge from a non-member contributes at least one external
    count (this is the rc-exactness established by merge + validate). -/
theorem validated_closed (s : State) (D : Obj → Prop)
    (members : List Obj) (extIn : Obj → Nat)
    (hmem : ∀ x, D x → x ∈ members)
    (h_roots_counted : ∀ t c, D c → s.roots t c → 1 ≤ extIn c)
    (h_edges_counted : ∀ u c, D c → ¬ D u → s.edges u c → 1 ≤ extIn c)
    (h_zero : ∀ c, c ∈ members → extIn c = 0) :
    closed s D := by
  constructor
  · intro t x hD hroot
    have h1 := h_roots_counted t x hD hroot
    have h2 := h_zero x (hmem x hD)
    omega
  · intro u v hD hedge
    by_cases hu : D u
    · exact hu
    · have h1 := h_edges_counted u v hD hu hedge
      have h2 := h_zero v (hmem v hD)
      omega

/-! ## §5 Parallel collections: tags, partitions, cross-target liveness -/

/-- Tags are issued from a monotonic counter under gMergeLock
    (startCollection). Distinct issue times give distinct tags, so a
    stale stamp from a finished collection can never be mistaken for a
    different active collection's tag. (The implementation wraps the
    counter at 2³¹; the model assumes no wrap-around while a tag is
    active — an ABA that would need 2³¹ collections to complete during
    one collection's lifetime.) -/
theorem tags_distinct (issue : Nat → Nat)
    (hmono : ∀ i j, i < j → issue i < issue j) :
    ∀ i j, issue i = issue j → i = j := by
  intro i j heq
  cases Nat.lt_trichotomy i j with
  | inl h => have := hmono i j h; omega
  | inr h =>
    cases h with
    | inl h => exact h
    | inr h => have := hmono j i h; omega

/-- Each cell's header stores ONE stamp (claimCell CASes the whole
    word), so two active collections with distinct tags claim disjoint
    partitions. -/
theorem partitions_disjoint (stamp : Obj → Nat) (tagA tagB : Nat)
    (hne : tagA ≠ tagB) :
    ∀ x, stamp x = tagA → stamp x = tagB → False := by
  intro x hA hB
  exact hne (hA ▸ hB)

/-- **Cross-target liveness**: a cell claimed by collection B but
    referenced from OUTSIDE B's partition is never in B's dead set.
    B's internal count for the cell only includes edges from B's dead
    members; the foreign edge contributes an external count, and
    validation forces external counts to zero — so the cell's SCC fails
    the deadness check (equivalently: it is demoted). This is why
    claimCell may simply refuse foreign-claimed cells (return -1) and
    crossPend defer them: their owner provably keeps them alive this
    round, and re-registration makes them candidates for the next. -/
theorem cross_target_live (D : Obj → Prop) (claimedB : Obj → Prop)
    (members : List Obj) (extIn : Obj → Nat)
    (s : State)
    (hDsub : ∀ x, D x → claimedB x)
    (hmem : ∀ x, D x → x ∈ members)
    (h_edges_counted : ∀ u c, D c → ¬ D u → s.edges u c → 1 ≤ extIn c)
    (h_zero : ∀ c, c ∈ members → extIn c = 0)
    (u c : Obj) (hedge : s.edges u c) (hu : ¬ claimedB u) :
    ¬ D c := by
  intro hDc
  have hDu : ¬ D u := fun h => hu (hDsub u h)
  have h1 := h_edges_counted u c hDc hDu hedge
  have h2 := h_zero c (hmem c hDc)
  omega

/-! ## §6 The grace period

  A concurrent capture holds raw `(slot, value)` snapshots (TraceEntry);
  the value pointer is dereferenced later (header read in claimCell). A
  capture that overlapped our validation may have snapshotted a slot
  that USED to point into our dead set. commitDead therefore waits, for
  every other slot that is in capture phase (gSlotPhase == 1), until
  that capture ends — captures never wait on anyone, so this is bounded.

  Two obligations:
  (a) captures that started BEFORE our commit are waited out — temporal
      argument below (`grace_no_use_after_free`);
  (b) captures that start AT/AFTER our commit never snapshot a dead
      cell in the first place (`post_commit_snap_misses_dead`): they
      only read slots of cells they claim; our dead cells carry our
      still-active tag, so claimCell refuses them (never traversed),
      and no slot OUTSIDE the dead set points into it (closedness, held
      through the window by §3 stability). -/

/-- Any snapshot value read by a post-commit capture comes from a slot
    of a cell that capture claimed; claimed cells are never dead cells
    of another active collection (§5), and the dead set is closed. -/
theorem post_commit_snap_misses_dead (s : State)
    (D claimedC snap : Obj → Prop)
    (hdisj : ∀ x, claimedC x → ¬ D x)
    (hclosed : closed s D)
    (hsnap : ∀ v, snap v → ∃ u, claimedC u ∧ s.edges u v) :
    ∀ v, D v → ¬ snap v := by
  intro v hD hs
  obtain ⟨u, hu, he⟩ := hsnap v hs
  exact hdisj u hu (hclosed.2 u v hD he)

/-- One concurrent capture, with its interval in a global time order and
    the set of values it ever snapshots. `derefs x t` = the capture
    reads x's header at time t (always within its interval, always on a
    snapshotted value). -/
structure CaptureWindow where
  start : Nat
  finish : Nat
  snap : Obj → Prop
  derefs : Obj → Nat → Prop

/-- **Grace safety**: no capture dereferences a dead cell at or after
    its free time. `commitT` is when the dead set validated; `freeT` is
    when commitDead's free loop runs. The premises are exactly the
    protocol: (grace) commitDead's spin means any capture that started
    before commit has finished before we free; (miss) §6(b) above. -/
theorem grace_no_use_after_free
    (C : CaptureWindow) (D : Obj → Prop) (commitT freeT : Nat)
    (h_deref : ∀ x t, C.derefs x t → C.start ≤ t ∧ t ≤ C.finish ∧ C.snap x)
    (h_grace : C.start < commitT → C.finish < freeT)
    (h_miss : commitT ≤ C.start → ∀ x, D x → ¬ C.snap x) :
    ∀ x t, D x → C.derefs x t → t < freeT := by
  intro x t hD hd
  obtain ⟨h1, h2, h3⟩ := h_deref x t hd
  cases Nat.lt_or_ge C.start commitT with
  | inl h => have := h_grace h; omega
  | inr h => exact absurd h3 (h_miss h x hD)

/-! ## §7 The asymmetric seq/GC fence (seqs_v2.nim)

  Seq structure mutations (which may FREE the old buffer on realloc)
  must not overlap a collection, but seq-vs-seq and collection-vs-
  collection may run concurrently. The committed fence:

    mutator (acquireMutatorLock):    collector (yrcGcFenceEnter):
      1. FetchAdd gSeqActive[s] SC     1. FetchAdd gGcActive  SC
      2. Load gGcActive        SC      2. Load gSeqActive[s]  SC (each s)
      proceed iff it read 0            proceed when all read 0
      (else back off: FetchSub, spin, retry)

  Under sequential consistency all four operations occupy positions in
  one total order. Suppose both sides are in their critical sections
  simultaneously (neither has executed its matching FetchSub). The
  mutator read gGcActive = 0 AFTER its own inc: since the collector's
  inc precedes its critical section and no dec intervened, the
  collector's inc must be ordered after the mutator's read — and
  symmetrically for the collector's read. That yields a cycle in the
  total order: -/

theorem fence_mutual_exclusion
    (mutInc mutChk gcInc gcChk : Nat)  -- positions in the SC total order
    (h_mut_po : mutInc < mutChk)       -- program order, mutator
    (h_gc_po : gcInc < gcChk)          -- program order, collector
    (h_mut_read0 : mutChk < gcInc)     -- mutator read gGcActive = 0
    (h_gc_read0 : gcChk < mutInc) :    -- collector read counter = 0
    False := by omega

/-! ## §8 Deadlock freedom

  ### Locks

  The queue producers went lock-free (reserve a slot by fetch-add, then
  publish it: incs with an RMW exchange — the validation peek must see
  every completed inc barrier — decs with a release store of the desc,
  self-protecting via the unexplained rc surplus). The validation peek
  is lock-free too. What remains:
    • gMergeLock                (level 0: tag-slot claim + orphan roots)
    • stripes[i].consumerLock   (level i + 1,  i in 0..N-1: excludes
                                 DRAINS of the same stripe against each
                                 other — drains wait out the two-store
                                 publication window and close each batch
                                 with a CAS, so no producer coordination
                                 is needed)
    • gWaitLock                 (leaf: pairs gWaitCond's wait/broadcast;
                                 only ever held around a predicate check,
                                 a wait(), or a broadcast() — never while
                                 acquiring any other lock, and no other
                                 lock is held when it is taken)

  Total order:  gMergeLock < consumerLock[0] < consumerLock[1] < ...

  In fact the current paths never HOLD two of these at once — strictly
  stronger than the ascending-order requirement the theorem needs:

  **nimIncRefCyclic / nimAsgnYrc / nimSinkYrc / enqueueDec /
    registerLocal / markDirtyFromQueues**: lock-free, no locks at all;
    enqueueDec's overflow calls collectCycles with nothing held. ✓
  **drainStripe**: consumerLock[i] alone; the processing inside
    (trialDec, registerLocal into the thread-local buffer) takes no
    lock. drainAllStripes: consumerLock[i] ascending, released between
    stripes. ✓
  **startCollection / nimYrcThreadTeardown**: drain first (consumerLock,
    released), THEN gMergeLock for the slot claim / orphan spill —
    sequential, never nested. adoptOrphans: gMergeLock alone. ✓
  **validateDead / commitDead**: no locks (candidate re-registration is
    the lock-free registerLocal). ✓

  ### Blocking waits (parked on gWaitCond after a bounded spin)

    W1 backpressure (startCollection): waits for a free tag slot —
       gMergeLock is RELEASED first; slots free when collections finish
       (finishCollection broadcasts).
    W2 solo gate (runCollection): a non-solo collection waits for
       gSoloCapture = 0 — cleared when the solo collection's CAPTURE
       ends (collectCyclesImpl broadcasts), before its commit.
    W3 grace (commitDead): waits for other slots to leave capture phase
       (broadcast at the phase 1→2 transition and at finish).
    W4 fence (yrcGcFenceEnter): spins for in-flight seq ops — each is a
       short critical section that never blocks (releaseMutatorLock is
       a plain FetchSub).

  W1–W3 park on gWaitCond: the waiter re-checks its predicate under
  gWaitLock before sleeping, and every state transition that can make a
  predicate true (capture-end, collection-finish) broadcasts under the
  same lock — so a transition either happens before the re-check (the
  waiter never sleeps) or after it (the waiter is inside wait() and is
  woken). No missed wakeups, and the wait-for structure is unchanged.

  No wait cycle exists: order the blocking conditions by what they wait
  FOR. A capture phase terminates unconditionally (finite traversal, no
  waits inside — claimCell returns -1 immediately on contention). W2
  waits only on a capture; W3 waits only on captures; a collection
  executes W2 BEFORE its own capture and W3 AFTER it, so "X waits (W2)
  on S's capture" and "S waits (W3) on X's capture" cannot hold
  simultaneously: S clears gSoloCapture before entering commit, so by
  the time S is in W3, X has passed W2. W1 waits on full collections,
  which terminate because W2/W3/W4 do. Formally, the lock part is the
  same ascending-order argument as before: -/

/-- Lock levels in YRC. -/
inductive LockId (n : Nat) where
  | mergeLock : LockId n
  | consumerLock (i : Nat) (h : i < n) : LockId n

def lockLevel {n : Nat} : LockId n → Nat
  | .mergeLock => 0
  | .consumerLock i _ => i + 1

/-- All lock levels are distinct (the order is total and well-defined). -/
theorem lockLevel_injective {n : Nat} (a b : LockId n)
    (h : lockLevel a = lockLevel b) : a = b := by
  cases a with
  | mergeLock =>
    cases b with
    | mergeLock => rfl
    | consumerLock j hj => simp [lockLevel] at h
  | consumerLock i hi =>
    cases b with
    | mergeLock => simp [lockLevel] at h
    | consumerLock j hj =>
      have : i = j := by simp [lockLevel] at h; exact h
      subst this; rfl

/-- gMergeLock has the lowest level. -/
theorem mergeLock_level_min {n : Nat} (l : LockId n) (h : l ≠ .mergeLock) :
    lockLevel (.mergeLock : LockId n) < lockLevel l := by
  cases l with
  | mergeLock => exact absurd rfl h
  | consumerLock i hi => simp [lockLevel]

/-- **Deadlock Freedom** (2-thread wait cycle; N-thread follows by the
    same transitivity on the wait-for chain): impossible when every
    thread acquires locks in strictly ascending level order. -/
theorem no_deadlock_from_total_order {n : Nat}
    (held₁ waited₁ held₂ waited₂ : LockId n)
    (h1 : lockLevel held₁ < lockLevel waited₁)
    (h2 : lockLevel held₂ < lockLevel waited₂)
    (h_wait1 : waited₁ = held₂)
    (h_wait2 : waited₂ = held₁) :
    False := by
  subst h_wait1; subst h_wait2
  omega

/-! ## Summary of verified properties (all QED, no sorry)

  §1  `yrc_safety` — the collector frees only unanchored objects, which
      no thread can reach. No use-after-free at the graph level.
  §2  `no_lost_object` — the atomically published edge is traced.
  §3  `step_preserves_deadInv`, `deadInv_stable`, `garbage_stability` —
      a closed set stays closed under every mutator write, root
      copy/drop, allocation, and foreign free: snapshot garbage cannot
      be resurrected. `commit_free_safe` — freeing a commit-validated
      dead set after ANY further concurrent activity is safe.
  §4  `validated_no_external`, `validated_closed` — the Σrc = Σinternal
      check plus rc-exactness forces zero external references, i.e. the
      dead set is closed at commit time (feeding §3).
  §5  `tags_distinct`, `partitions_disjoint`, `cross_target_live` —
      concurrent collections own disjoint partitions, and a cell
      referenced across a partition boundary is never freed by its
      owner this round (soundness of claimCell's -1 + crossPend).
  §6  `post_commit_snap_misses_dead`, `grace_no_use_after_free` — with
      commitDead's grace spin, no capture ever dereferences freed
      memory.
  §7  `fence_mutual_exclusion` — the SEQ_CST Dekker pairing in
      seqs_v2.nim excludes seq structure mutation during collection.
  §8  `lockLevel_injective`, `mergeLock_level_min`,
      `no_deadlock_from_total_order` — the remaining locks form a total
      order acquired ascending; spin-waits form an acyclic wait-for
      structure (prose above).

  ## What is NOT proved

  • Tarjan/SCC implementation correctness: that `capture` computes the
    actual SCCs and that computeDeadness's per-SCC sums equal the model's
    Σrc/Σinternal for the emitted dead set (condensation, sinks-first
    order, deadIn accounting). §4 takes the counts as given.
  • rc-exactness mechanics: that merge + the dirty check + the rc-word
    recheck really imply "commit-time rc = internal + external" (§4's
    h_exact). The argument: rc is only mutated by atomic direct incs
    (caught by the recheck), merged queue entries (queues drained at
    merge; later entries caught by the dirty peek), and the collector's
    own inRootsFlag toggles (excluded from the compared word — see the
    comment above claimCell).
  • The C11 memory model: §7 assumes sequential consistency for the
    SEQ_CST operations (sound: SEQ_CST ops do form a total order) and
    the acquire/release reasoning elsewhere is informal.
  • Liveness/completeness: every dead cycle is EVENTUALLY freed.
    Aborted (dirty) SCCs and crossPend targets are re-registered as
    candidates, so they are re-examined; termination of that loop under
    adversarial mutators is not formalized. Also unproved: termination
    bounds for the four spin-waits (prose in §8).
  • Tag wrap-around: 2³¹ collections completing during one collection's
    lifetime could forge a stale stamp (noted at `tags_distinct`).

  ## Epoch stamps (generational pruning)

  Commit re-stamps proven-live cells with (epochBase|epoch, survivalAge)
  in the claim word; a capture treats a current-epoch stamp of age ≥
  YrcPromoteAge on a DESCENDANT as an opaque live external and does not
  descend. Soundness needs no new lemmas: a pruned cell is simply an
  uncaptured cell, so the captured set shrinks and every §3–§6 statement
  quantifies over a smaller S. Pruning can only ADD unexplained external
  refs to captured SCCs (a pruned predecessor's refs are never explained
  by internal/deadIn), so it can force a false "live", never a false
  "dead" — the conservative direction. Completeness (bounded float,
  ≤ ~2 epochs) rests on four hooks, each keeping a dec-witness
  registered:
    E1  roots never prune: a registered candidate is always fully
        root-scanned, stamps notwithstanding;
    E2  when a collection prunes ANY out-edge, it keeps one member of
        EVERY surviving SCC registered — not just the SCCs that pruned an
        edge themselves. This is broader than it first looks and the
        breadth is load-bearing: a pruned cell is not traced, yet its
        out-edges still count toward its targets' rc, so a pruned cell
        that is ITSELF dead (promoted while live, died later this epoch)
        contributes phantom refs that inflate an UNRELATED SCC's external
        count and misclassify that genuinely-dead SCC as a plain (non-
        prune-source) survivor. Registering only prune-source SCCs
        (the original hook) let such a survivor be re-stamped, dropped
        from the retry set, and orphaned permanently once its last
        prune-source neighbour resolved — a real leak the dumpster `fuzz`
        port surfaced (tests/yrc/tyrc_fuzz_graph.nim: ~1% of allocations
        lost, ORC-clean, un-recoverable even by repeated GC_fullCollect).
        Registering every survivor of a pruning collection closes it; the
        cost is bounded because pruning keeps the captured set small, so
        "every survivor" is only the handful actually traced;
    E3  a commit-time dec into a stamped cell re-registers the target —
        the dec may be the death blow to a cell no collection analyzed;
    E4  explicit full collects advance the epoch first, so all stamps
        are stale and nothing is pruned.
  Not formalized (and E2's original narrow form was empirically wrong —
  see above; the broadened form is validated by the fuzz port across seeds
  and sizes, not machine-checked). The epoch clock counts collections
  (YrcEpochLen=64);
  short epochs (≲ 4) resonate with the adaptive threshold — pruned
  collections are cheap, so collections and hence epoch turns speed up,
  re-tracing MORE than with no stamps — but 64 sits clear of that. A
  work-based clock (advance per N cells traced) was measured and lost on
  long-lived structures: pruning shrinks trace work, so the clock stalls
  exactly when a stale web should be re-examined, trading float for a
  resonance the default length already avoids.

  Reference: D.F. Bacon and V.T. Rajan, "Concurrent Cycle Collection in
  Reference Counted Systems", ECOOP 2001 — the deadness arithmetic is
  the condensation form of their trial deletion; the capture/validate/
  commit structure and the SATB use of the deferred-dec queues are
  yrc-specific.
-/