Two defects, one shape: something the decoder must decide was never stated, so
it fell out of where an entry landed in an unstably sorted table.
CANONICITY. Several mnemonics name one encoding -- SHL and SAL are both /4, JE
and JZ are both 0x74. Decoding a representative of each and diffing against
llvm-mc: 74 patterns, 22 agreed, 52 did not (0F 84 -> JZ where llvm-mc says JE,
A4 -> MOVS vs MOVSB, DB E2 -> FCLEX vs FNCLEX). aliases.odin now declares 57
{alias, canonical} rows, each canonical name MEASURED from llvm-mc rather than
picked, and gen.odin drops an aliased entry as it collects decode entries -- so
the name never reaches the tables. The drop is conditional on the canonical name
covering the byte-identical encoding, which is what lets MOV be the alias at the
A0-A3/B8 moffs forms while staying the only name for 88/89. Aliases stay fully
encodable; only decoding narrows. 119 of 1350 legacy entries dropped, 74 of 74
now agree.
This also fixes the four SAL/SHL failures left by the previous commit, at their
root rather than by extending a hand-written list.
ADDRESS SIZE. 0xE3's mnemonic is chosen by which counter register it tests --
the 67h axis -- and REX.W does not affect address size at all. Modelling JRCXZ
as force_rex_w made its encoding 48 E3 cb where llvm-mc emits a bare E3 cb, gave
JECXZ the bare encoding that is really JRCXZ in long mode, and left three
indistinguishable entries. Encoding_Flags gains addr_size (2 of 6 spare bits);
the matcher refuses a form the mode cannot express, the encoder emits 67h when
it differs from the default, the decoder selects on it before the operand-size
pass. All ten cases match llvm-mc, refusals included.
The tests keep no second copy of the alias table: mnemonics_eq compares
canonical_mnemonic, and run_alias_table_test asserts the stronger property the
generator guarantees -- zero encodings spelled by two surviving mnemonics. It
caught the E3 ambiguity on its own.
Also fixed: gen.odin's write_flags enumerates Encoding_Flags by hand and
silently dropped addr_size from the generated tables -- the flag read back as
its zero value and the instruction was quietly mis-modelled.
271 passed. Bites: dropping {.SAL, .SHL} returns the prime_sieve failure and the
guard names every shift-group opcode; restoring force_rex_w decodes E3 00 as
JCXZ and encodes JRCXZ to three bytes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Riok9vMpkLmo78wsVKJHhz
mnemonic_builders.odin bakes `global_index + 1` into Instruction.enc_hint so the
encoder can skip the O(forms) match scan. The index is GLOBAL, so any edit to
the encoding table that inserts or removes a form shifts every index after it
and leaves the builders naming another instruction's encoding -- and nothing
noticed: the encoder took the form, emitted its bytes and returned success.
It has happened twice. 6e17e7a2d left 2130 of 3671 builders wrong; 36af73834
regenerated both halves and cleared it; baae2636b (adding `in`/`out`) re-broke
37; 9ae9a9bf9 shifted an early mnemonic and broke 3393 of 3802 -- including
CALL, whose r/m64 builder then encoded 0F 8A (JPE) instead of FF /2, turning
every indirect call into a conditional jump. The symptom was a segfault in a
JIT'd program, arbitrarily far from the cause.
Regenerated: 3820 of 3820 builders now correct. Three guards so it cannot
return silently:
- the encoder checks, under ODIN_DEBUG, that the hinted form lies inside its
own mnemonic's ENCODE_RUNS entry. Release keeps the byte-for-byte fast path;
this is a regeneration-time mistake and only has to be caught once by
anyone running tests.
- the generator stamps BUILDER_TABLE_FINGERPRINT, an FNV-1a hash of every
ENCODE_RUNS (start, count), into the file it emits.
- run_builder_generation_test recomputes it from the loaded tables and fails
with the command to regenerate.
Known-failing at this commit: four SAL/SHL cases, a separate defect in the
alias table, fixed in the commit that follows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Riok9vMpkLmo78wsVKJHhz
`bswap` is `0F C8+rd` — the register rides in the opcode's low three bits, like
`push`/`pop`/`xchg`/`mov`, but behind an escape byte. The decoder's retry at the
+r base opcode was gated on `esc == .NONE`, so only the register-0 forms
(`0F C8` = bswap eax, `48 0F C8` = bswap rax) landed on a table entry directly;
the other seven registers came back INVALID_OPCODE. Emission was always correct
— every `bswap` this compiler has ever produced runs — but a disassembly
containing one stopped dead, which is how it surfaced: three functions in
sigil's JIT corpus disassembled to a header with no instructions under it.
The table was never the problem. BSWAP is in `tablegen/encoding_table.odin` and
in the generated decode table, exactly where it should be.
Fixing the gate meant looking at why the retry existed twice. There is a correct
+r handler further down, carrying comments that describe two bugs already found
and fixed in it — "scan ALL entries, not just the first", "scan for the sized
register rather than assuming op0" — and the copy inside the retry had received
neither, plus a third of its own. So the retry now re-runs only the LOOKUP and
falls through to the single handler, which fixes two more instruction families
that were equally undecodable:
- `xchg rAX, r` (90+rd): 0x90's run sorts NOP ahead of XCHG, and the copy
tested only the first entry for a +r form, so every `xchg rAX, r` was
rejected.
- `push bx` / `pop bx` / `mov cx, imm` (any +r under 0x66): the copy passed
`prefix` where the legacy row wants 0, since for legacy opcodes 0x66 is
operand size rather than part of the opcode's identity.
Two supporting changes. The shared handler's default answer is now the first +r
entry rather than `idx.start`, so a fall-through at 0x90 cannot answer NOP for
an `xchg`. And a base-opcode retry that does not land on a +r form is now an
error rather than a fall-through: decoding 0x0E as the 0x08 entry (OR) would be
a wrong instruction reported confidently, which is worse than an unknown byte.
Verified three ways. Every opcode in every escape map was decoded before and
after: 154 combinations changed, every one of them INVALID -> valid, and nothing
that already decoded decodes differently. All 24 BSWAP encodings and all 96
legacy +r encodings were diffed against llvm-mc's disassembly and agree
operand-for-operand. Six new decode-only cases pin the three families; five of
them fail without this change.
(Two remaining differences from llvm-mc are deliberate and documented here:
implicit accumulators are left unmaterialized so a short form re-encodes to the
short form, and `48 90` reads as `xchg rax` rather than `nop`.)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Riok9vMpkLmo78wsVKJHhz
