Two defects, one shape: something the decoder must decide was never stated, so
it fell out of where an entry landed in an unstably sorted table.
CANONICITY. Several mnemonics name one encoding -- SHL and SAL are both /4, JE
and JZ are both 0x74. Decoding a representative of each and diffing against
llvm-mc: 74 patterns, 22 agreed, 52 did not (0F 84 -> JZ where llvm-mc says JE,
A4 -> MOVS vs MOVSB, DB E2 -> FCLEX vs FNCLEX). aliases.odin now declares 57
{alias, canonical} rows, each canonical name MEASURED from llvm-mc rather than
picked, and gen.odin drops an aliased entry as it collects decode entries -- so
the name never reaches the tables. The drop is conditional on the canonical name
covering the byte-identical encoding, which is what lets MOV be the alias at the
A0-A3/B8 moffs forms while staying the only name for 88/89. Aliases stay fully
encodable; only decoding narrows. 119 of 1350 legacy entries dropped, 74 of 74
now agree.
This also fixes the four SAL/SHL failures left by the previous commit, at their
root rather than by extending a hand-written list.
ADDRESS SIZE. 0xE3's mnemonic is chosen by which counter register it tests --
the 67h axis -- and REX.W does not affect address size at all. Modelling JRCXZ
as force_rex_w made its encoding 48 E3 cb where llvm-mc emits a bare E3 cb, gave
JECXZ the bare encoding that is really JRCXZ in long mode, and left three
indistinguishable entries. Encoding_Flags gains addr_size (2 of 6 spare bits);
the matcher refuses a form the mode cannot express, the encoder emits 67h when
it differs from the default, the decoder selects on it before the operand-size
pass. All ten cases match llvm-mc, refusals included.
The tests keep no second copy of the alias table: mnemonics_eq compares
canonical_mnemonic, and run_alias_table_test asserts the stronger property the
generator guarantees -- zero encodings spelled by two surviving mnemonics. It
caught the E3 ambiguity on its own.
Also fixed: gen.odin's write_flags enumerates Encoding_Flags by hand and
silently dropped addr_size from the generated tables -- the flag read back as
its zero value and the instruction was quietly mis-modelled.
271 passed. Bites: dropping {.SAL, .SHL} returns the prime_sieve failure and the
guard names every shift-group opcode; restoring force_rex_w decodes E3 00 as
JCXZ and encodes JRCXZ to three bytes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Riok9vMpkLmo78wsVKJHhz
`bswap` is `0F C8+rd` — the register rides in the opcode's low three bits, like
`push`/`pop`/`xchg`/`mov`, but behind an escape byte. The decoder's retry at the
+r base opcode was gated on `esc == .NONE`, so only the register-0 forms
(`0F C8` = bswap eax, `48 0F C8` = bswap rax) landed on a table entry directly;
the other seven registers came back INVALID_OPCODE. Emission was always correct
— every `bswap` this compiler has ever produced runs — but a disassembly
containing one stopped dead, which is how it surfaced: three functions in
sigil's JIT corpus disassembled to a header with no instructions under it.
The table was never the problem. BSWAP is in `tablegen/encoding_table.odin` and
in the generated decode table, exactly where it should be.
Fixing the gate meant looking at why the retry existed twice. There is a correct
+r handler further down, carrying comments that describe two bugs already found
and fixed in it — "scan ALL entries, not just the first", "scan for the sized
register rather than assuming op0" — and the copy inside the retry had received
neither, plus a third of its own. So the retry now re-runs only the LOOKUP and
falls through to the single handler, which fixes two more instruction families
that were equally undecodable:
- `xchg rAX, r` (90+rd): 0x90's run sorts NOP ahead of XCHG, and the copy
tested only the first entry for a +r form, so every `xchg rAX, r` was
rejected.
- `push bx` / `pop bx` / `mov cx, imm` (any +r under 0x66): the copy passed
`prefix` where the legacy row wants 0, since for legacy opcodes 0x66 is
operand size rather than part of the opcode's identity.
Two supporting changes. The shared handler's default answer is now the first +r
entry rather than `idx.start`, so a fall-through at 0x90 cannot answer NOP for
an `xchg`. And a base-opcode retry that does not land on a +r form is now an
error rather than a fall-through: decoding 0x0E as the 0x08 entry (OR) would be
a wrong instruction reported confidently, which is worse than an unknown byte.
Verified three ways. Every opcode in every escape map was decoded before and
after: 154 combinations changed, every one of them INVALID -> valid, and nothing
that already decoded decodes differently. All 24 BSWAP encodings and all 96
legacy +r encodings were diffed against llvm-mc's disassembly and agree
operand-for-operand. Six new decode-only cases pin the three families; five of
them fail without this change.
(Two remaining differences from llvm-mc are deliberate and documented here:
implicit accumulators are left unmaterialized so a short form re-encodes to the
short form, and `48 90` reads as `xchg rax` rather than `nop`.)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Riok9vMpkLmo78wsVKJHhz
Fix the operand-size/prefix bug class reported by the sigil backend, then close
every failure a whole-ISA encode->decode->re-encode audit surfaced -- 247 -> 0
over the 1165 round-trippable legacy forms.
Encode/decode:
- MOVSX/MOVZX take 66h from the destination, not the source (CRC32, whose r/m
source drives 66h, is unchanged)
- operand-less 16-bit forms (CBW/CWD, MOVSW/CMPSW/SCASW/LODSW/STOSW,
IRET/PUSHF/POPF) emit 66h via a new opsize_16 flag
- CRC32/MOVBE and POPCNT/LZCNT/TZCNT: mandatory F2/F3 beats 66h on decode
- mandatory-prefix REP double-encode: F2/F3 no longer re-emitted as REP
- fixed-ModR/M decode: x87 (FNOP/FCHS/FADD ST(i)/...) and the 0F 01/AE/C7/1E
system-op groups (VMCALL/RDTSCP/LFENCE/ENDBR/...); gen.odin preserves the
fixed ModR/M byte, the decoder matches and consumes it
- implicit-operand round-trip (accumulator short forms, x87 ST(i), xchg) via a
positional matcher + emit path; accumulators stay implicit so hand-built
`add eax, imm` keeps the general form (typed-builder consistency)
- MOV to/from CR/DR/segment (were decoding as GPRs); reg-vs-mem disambiguation
(RDRAND vs VMPTRLD, MOVLHPS vs MOVHPS); x87 M80 operand size; PINSRW; NOP vs
XCHG at 0x90
- LOCK standalone; ENTER two immediates; PUSH/POP FS/GS (segment fixed by the
opcode + non-cacheable); BOUND marked 32-bit-only; RDRAND/RDSEED r16
(66 is operand-size, resolved against the mandatory-66 VMCLEAR by ModR/M)
Tools/tests:
- revive dump_verify_input + verify_against_llvm (stale Result API; drop the
aliases that were masking the fixed bugs); align verify_tables with the
preserved fixed-ModR/M ext
- new tests/test_narrow_widths.odin: byte-exact vs llvm-mc + round-trip guards
- fold the bit-rotted tests32 package into the main suite (i386 Mode._32 paths)
248 tests pass; verify_tables passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01115ktMoX5qGAfp1VemBzCP
Give callers a clean way to pre-size their own buffers so the encode/decode
hot paths never allocate or resize, instead of decode() silently reserving the
caller's arrays itself (removed). The library allocates nothing -- these only
grow the caller's own dynamic arrays, and only when not already big enough
(Odin's reserve no-ops when capacity already suffices).
Size-only helpers (caller manages its own memory), keyed off the input slice:
encode_max_code_size(instructions) - exact code bytes
encode_max_relocation_count(instructions) - exact reloc upper bound
decode_max_instruction_count(data) - exact ceiling (1 byte/inst)
decode_estimate_instruction_count(data) - typical estimate (~3 B/inst)
Reserve helpers (pre-size the caller's dynamic arrays; nil to skip an array):
encode_reserve(code, relocs, instructions)
code is a [dynamic]u8 grown by LENGTH (so code[:] is a valid emit
target); relocs reserved by capacity on top of existing elements.
decode_reserve(instructions, inst_info, label_defs, data, exact=false)
reserves capacity on top of existing; exact=true for the ceiling.
Error arrays grow only on the failure path, so they are intentionally not
covered. check/test green; 2282 cases; exercised end-to-end (the [dynamic]u8
code pattern, factor-in-existing, nil args, exact ceiling, reserve no-op).
Three layers on the x86 encode/decode hot paths, all byte-exact (2246
LLVM-verified cases) and roundtrip-clean:
1. Branchless: legacy-prefix emission (speculative write + conditional
advance), REX/VEX/EVEX extension-bit accumulation (gate-and-mask),
ModRM mod/disp-size selection (cmov selects), displacement emission
(widened store + ENCODE_TAIL_SLACK); decoder REX/VEX/EVEX register
extensions (arithmetic instead of if/+=8).
2. Resolve-operands-once: the previous code re-derived each user operand
~5-10x per instruction (a fresh O(n) scan of enc.ops per emission
pass). Now resolved into a [4]^Operand map a single time.
3. Single-pass gather: fold the opcode-+rb and ModR/M slot-detection
scans into that one resolve pass (3 enc.enc passes -> 1).
Net on a 100k mixed-instruction benchmark: encode ~58 -> ~54 ns/inst
(best 52). Branchless alone was a ~7% encode regression (predicted
branches, nothing to recover); the algorithmic passes recovered it and
beat baseline.
Move all ten ISA packages (x86, arm32, arm64, mips, riscv, ppc, ppc_vle,
rsp, mos6502, mos65816) from core/rexcode/<arch> to core/rexcode/isa/<arch>,
so the import pattern is now `import "core:rexcode/isa/x86"`. The shared
core stays at core:rexcode/isa.
Mechanical: relative `import "../isa"` / "../../isa" -> absolute
"core:rexcode/isa" (the only path that survives the move; the "../" and
"../.." self/generated imports move with their packages). build.lua now
builds paths as <root>/isa/<name>; stale `cd <arch>` hints in the verify
tools and the doc.odin paths updated.
WASM stays at core/rexcode/wasm for now -- it is an IR, not an ISA, and
will move under the forthcoming core:rexcode/ir once that layer lands.
All 10 arches gen/builders/check/test green; import core:rexcode/isa/x86
verified working; wasm still compiles.
