`bswap` is `0F C8+rd` — the register rides in the opcode's low three bits, like
`push`/`pop`/`xchg`/`mov`, but behind an escape byte. The decoder's retry at the
+r base opcode was gated on `esc == .NONE`, so only the register-0 forms
(`0F C8` = bswap eax, `48 0F C8` = bswap rax) landed on a table entry directly;
the other seven registers came back INVALID_OPCODE. Emission was always correct
— every `bswap` this compiler has ever produced runs — but a disassembly
containing one stopped dead, which is how it surfaced: three functions in
sigil's JIT corpus disassembled to a header with no instructions under it.
The table was never the problem. BSWAP is in `tablegen/encoding_table.odin` and
in the generated decode table, exactly where it should be.
Fixing the gate meant looking at why the retry existed twice. There is a correct
+r handler further down, carrying comments that describe two bugs already found
and fixed in it — "scan ALL entries, not just the first", "scan for the sized
register rather than assuming op0" — and the copy inside the retry had received
neither, plus a third of its own. So the retry now re-runs only the LOOKUP and
falls through to the single handler, which fixes two more instruction families
that were equally undecodable:
- `xchg rAX, r` (90+rd): 0x90's run sorts NOP ahead of XCHG, and the copy
tested only the first entry for a +r form, so every `xchg rAX, r` was
rejected.
- `push bx` / `pop bx` / `mov cx, imm` (any +r under 0x66): the copy passed
`prefix` where the legacy row wants 0, since for legacy opcodes 0x66 is
operand size rather than part of the opcode's identity.
Two supporting changes. The shared handler's default answer is now the first +r
entry rather than `idx.start`, so a fall-through at 0x90 cannot answer NOP for
an `xchg`. And a base-opcode retry that does not land on a +r form is now an
error rather than a fall-through: decoding 0x0E as the 0x08 entry (OR) would be
a wrong instruction reported confidently, which is worse than an unknown byte.
Verified three ways. Every opcode in every escape map was decoded before and
after: 154 combinations changed, every one of them INVALID -> valid, and nothing
that already decoded decodes differently. All 24 BSWAP encodings and all 96
legacy +r encodings were diffed against llvm-mc's disassembly and agree
operand-for-operand. Six new decode-only cases pin the three families; five of
them fail without this change.
(Two remaining differences from llvm-mc are deliberate and documented here:
implicit accumulators are left unmaterialized so a short form re-encodes to the
short form, and `48 90` reads as `xchg rax` rather than `nop`.)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Riok9vMpkLmo78wsVKJHhz
Internal label ids are allocation-order handles: the encoder's creation
order, or the decoder's branch-DISCOVERY order (a loop's latch names the
header before an earlier forward target). Printing labels by raw id leaked
that accident into listings — label numbers appeared out of order down the
page — and the printers' `label_names: ^map[u32]string` keyed the caller's
names by those synthesized ids, which a decode consumer cannot know without
re-deriving them (the practical result: naming "label 0" could caption a
random interior branch target).
Naming is now derived at the presentation seam, shared by every ISA
(`isa.Label_Display` in isa/print.odin):
- display numbers are assigned in ASCENDING ADDRESS order, so a listing
reads L0, L1, L2 … top to bottom regardless of id allocation;
- caller names are keyed by BYTE OFFSET (`isa.Label_Names`, with a
`distinct` Label_Offset key so an id-keyed map from the old contract
fails to compile instead of silently mis-naming);
- a named offset is guaranteed a label row even when no Label_Definition
points at it — `names[0] = "factorial"` heads a function's listing.
All ten ISA printers (x86, mips, rsp, arm32, arm64, riscv, ppc, ppc_vle,
mos6502, mos65816) drop their per-printer offset_to_label maps and
write_label helpers for the shared display; each arch re-exports
Label_Offset/Label_Names beside Label_Definition. En route this fixes an
arm32/ppc/ppc_vle bug where passing ANY names map suppressed the default
L<n> label rows for unnamed labels. Decode-side id assignment is untouched:
the reloc round-trip contract (encoder ids surviving decode) and the
sparse-id padding it relies on stay exactly as they were.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Riok9vMpkLmo78wsVKJHhz
The table had the R6 unsigned mul, both muh, and mod/modu, but not the R6
SIGNED low multiply (SPECIAL funct 0x18, sa=2 → 0x98) or the R6 signed/unsigned
divide (funct 0x1A, sa=2/3 → 0x9A/0x9B). Add MUL_R6, DIV_R6, DIVU_R6 to the
Mnemonic enum + the encoding table (all SPECIAL, RD/RS/RT, mask 0xFC0007FF,
MIPS32_R6) and regenerate the tables. Verified against llvm-mc -mcpu=mips32r6
(mul/div/divu $rd,$rs,$rt). All existing mips tests pass (166/39/65/14).
The MIPS decode tables are universal — every variant (MIPS I..64/R6, the PS1
GTE, PS2 MMI/VU, PSP VFPU) shares one table. Some primary opcodes collide
across variants: most visibly 0x37 is `LD` on 64-bit MIPS but `vfim.s` on the
PSP Allegrex VFPU. The decoder took the first table match, so `LD $ra, 0($sp)`
(0xdfbf0000) mis-decoded as `vfim.s $31, 0`.
Each Decode_Entry already carries a `feature`; the decoder just ignored it.
`decode` now takes a `features: Feature_Set` (a bit_set over `Feature`),
defaulting to `FEATURES_ALL` so existing callers are unchanged, and skips an
entry whose feature isn't enabled — the first ENABLED match wins, preserving
the most-specific-mask-first order. Added `FEATURES_MIPS_III` (MIPS I/II/III +
COP0 + FPU — the VR4300 / classic 64-bit baseline) for N64/MIPS-III consumers.
decode_smoke: opcode 0x37 decodes as LD under FEATURES_MIPS_III and (still)
as the PSP VFPU entry under FEATURES_ALL. 65 decoder checks pass.
Add mem_rip_label(label_id) so a RIP-relative memory operand can
reference a label: the encoder writes a placeholder disp32 and emits a
REL32 relocation (addend 0) at the field's byte offset, mirroring the
existing .RELATIVE jump/call path. This expresses lea reg, [rip + <label>]
(position-independent data addressing).
Add op_imm_label(label_id) for movabs reg, <label>: the imm stays kind
.IMMEDIATE (form matching unchanged) but is flagged so the encoder emits
an ABS64 relocation for the imm64 instead of a literal; imm_matches_inline
forces the full IMM64 form so a small id can't collapse to mov r64, imm32.
Both labeled forms bypass the contextless recipe fast-path (which cannot
append relocations) and fall back to the interpreter. Flags reuse spare
bits in Memory (disp_is_label) and Operand_Flags (imm_is_label) -- no
struct growth. Section 11 tests cover resolved/unresolved disp, the ABS64
movabs, and an end-to-end executed lea+load.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fix the operand-size/prefix bug class reported by the sigil backend, then close
every failure a whole-ISA encode->decode->re-encode audit surfaced -- 247 -> 0
over the 1165 round-trippable legacy forms.
Encode/decode:
- MOVSX/MOVZX take 66h from the destination, not the source (CRC32, whose r/m
source drives 66h, is unchanged)
- operand-less 16-bit forms (CBW/CWD, MOVSW/CMPSW/SCASW/LODSW/STOSW,
IRET/PUSHF/POPF) emit 66h via a new opsize_16 flag
- CRC32/MOVBE and POPCNT/LZCNT/TZCNT: mandatory F2/F3 beats 66h on decode
- mandatory-prefix REP double-encode: F2/F3 no longer re-emitted as REP
- fixed-ModR/M decode: x87 (FNOP/FCHS/FADD ST(i)/...) and the 0F 01/AE/C7/1E
system-op groups (VMCALL/RDTSCP/LFENCE/ENDBR/...); gen.odin preserves the
fixed ModR/M byte, the decoder matches and consumes it
- implicit-operand round-trip (accumulator short forms, x87 ST(i), xchg) via a
positional matcher + emit path; accumulators stay implicit so hand-built
`add eax, imm` keeps the general form (typed-builder consistency)
- MOV to/from CR/DR/segment (were decoding as GPRs); reg-vs-mem disambiguation
(RDRAND vs VMPTRLD, MOVLHPS vs MOVHPS); x87 M80 operand size; PINSRW; NOP vs
XCHG at 0x90
- LOCK standalone; ENTER two immediates; PUSH/POP FS/GS (segment fixed by the
opcode + non-cacheable); BOUND marked 32-bit-only; RDRAND/RDSEED r16
(66 is operand-size, resolved against the mandatory-66 VMCLEAR by ModR/M)
Tools/tests:
- revive dump_verify_input + verify_against_llvm (stale Result API; drop the
aliases that were masking the fixed bugs); align verify_tables with the
preserved fixed-ModR/M ext
- new tests/test_narrow_widths.odin: byte-exact vs llvm-mc + round-trip guards
- fold the bit-rotted tests32 package into the main suite (i386 Mode._32 paths)
248 tests pass; verify_tables passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01115ktMoX5qGAfp1VemBzCP
Move ENCODE_RECIPES from a startup-built (@init) static table in the library to
a generated, #loaded table like every other one. The tablegen runs
form_to_recipe over every form, serializes to x86.encode_recipes.bin, and the
loader #loads it -- no @init, no allocation; the forms remain the single source
of truth and the recipe is derived from them.
- tablegen/gen.odin: ENCODE_RECIPES added to the BLOBS manifest (which drives
both the Stage B serializer and the loader's #load); emit the generated-package
global plus an @init that fills it from ENCODE_FORMS via lib.form_to_recipe.
- encoder_recipe.odin: drop the @init, the static storage, and the ENCODE_RECIPES
declaration. form_to_recipe stays -- now solely the tablegen-time derivation.
- tables.odin, generated/{encode_tables,writer}.odin: regenerated.
- tables/x86.encode_recipes.bin: the generated blob (28260 bytes = 2355 x 12).
No behavior change: the recipes are identical to the @init ones. gen / builders /
check / test / idempotent green (2282 cases, 22 byte-stable artifacts); encode
perf unchanged.
The matcher path (generic builders, hand-built, decode->re-encode) resolves an
instruction to its encoding form by linearly scanning the forms for its mnemonic
and operand-matching each -- the dominant cost on that path. Memoize it: pack
(mnemonic, per-operand shape) into a key (immediates folded to the smallest size
class they fit, matching imm_matches_inline) and cache key -> form so a repeated
instruction shape skips the scan.
Direct-mapped, fixed 8192-slot table (64 KB, no allocation). Each slot packs the
full 48-bit key and form index into one u64, read/written with relaxed atomics,
so concurrent encode() stays safe -- a reader sees a matching key or rescans,
never a torn entry. The scan stays the source of truth (a miss runs it and
records the result), so the cache is exact.
Lookup + scan live in a non-inlined find_form() so they don't bloat encode()'s
hot loop and slow the hint path that shares it. (Routing the matcher path through
the recipe emit was tried and dropped: it costs the hint path ~1.2-1.5 ns however
isolated -- the hot loop is too codegen-sensitive -- while the cache alone is
free for the hint path.)
Realistic generic-builder mix: matcher ~52 -> ~35 ns/inst (~1.49x); hint path
unchanged. Byte-exact across 2282 + idempotent.
Extend emit_recipe to the full ModR/M + SIB + displacement addressing (register
direct, RIP-relative, absolute [disp32], and base/index/scale/disp), mirroring
the interpreter byte-for-byte, and drop the caller's reg-direct guard so memory
operands take the fast path too. Only a label/relative immediate (a relocation)
still falls back.
Realistic immediate-heavy mix: ~20.1 -> ~12.9 ns/inst vs the pre-recipe base
(~1.55x, 50 -> 77 M/s). Byte-exact across 2282 + idempotent.
Precompute each encoding form into a flat Form_Recipe -- prefix byte, escape+
opcode blob, role->operand-index slots, ext, imm size, flags -- so the encoder
replays common forms straight-line instead of re-interpreting enc.ops/enc.enc
on every instruction (the resolve scan, escape ladder, prefix/REX selection).
encode() takes the fast path when the form is hinted, eligible, has a register
r/m and a literal immediate; everything else falls through to the existing
interpreter, which stays the byte-exact source of truth. First cut:
- reg-direct ModR/M only (memory r/m falls back)
- hint path only (matcher / generic builders fall back)
- ~33% of forms eligible (VEX/EVEX, 16-bit operand-size, x87 fixed-ModR/M,
moffs/far/rel/implicit operands are marked ineligible)
Recipes are built at startup into static storage (no heap); this moves into the
table generator (#loaded like every other table) once the shape settles.
Realistic immediate-heavy mix: ~19.0 -> ~16.3 ns/inst (52.7 -> 61.3 M/s).
Byte-exact across 2282 cases + idempotent.
Next: memory r/m addressing in the fast path, then the matcher path, then the
gen-time port.
Roll the encode/decode buffer-sizing helpers (added for x86 in 49787b7de) out
to every other ISA, and document them in the cross-arch naming contract.
Per arch (arm32, arm64, mips, riscv, ppc, ppc_vle, rsp, mos6502, mos65816):
- encode_max_code_size / encode_max_relocation_count now key off the
[]Instruction slice (were int counts); bodies unchanged (* MAX_INST_SIZE).
- encode_reserve(code, relocs, instructions): grows the caller's code []u8 by
length and reserves relocs by capacity; allocates no new buffers.
- decode_max_instruction_count / decode_estimate_instruction_count: exact
ceiling and typical estimate, keyed off the min/avg instruction size per
arch (fixed-4: arm64/mips/ppc/rsp; min-2: arm32/riscv/ppc_vle; min-1: mos).
- decode_reserve(instructions, inst_info, label_defs, data, exact=false).
docs/cross_arch_design.md: helpers added to the naming contract.
No behavior change to the existing size helpers (signature only). All 10 ISAs
check + test green (x86 2282, arm32 600, arm64 461, mips 281, riscv 154, ppc 31,
ppc_vle 281, rsp 70, mos6502 148, mos65816 53).
Give callers a clean way to pre-size their own buffers so the encode/decode
hot paths never allocate or resize, instead of decode() silently reserving the
caller's arrays itself (removed). The library allocates nothing -- these only
grow the caller's own dynamic arrays, and only when not already big enough
(Odin's reserve no-ops when capacity already suffices).
Size-only helpers (caller manages its own memory), keyed off the input slice:
encode_max_code_size(instructions) - exact code bytes
encode_max_relocation_count(instructions) - exact reloc upper bound
decode_max_instruction_count(data) - exact ceiling (1 byte/inst)
decode_estimate_instruction_count(data) - typical estimate (~3 B/inst)
Reserve helpers (pre-size the caller's dynamic arrays; nil to skip an array):
encode_reserve(code, relocs, instructions)
code is a [dynamic]u8 grown by LENGTH (so code[:] is a valid emit
target); relocs reserved by capacity on top of existing elements.
decode_reserve(instructions, inst_info, label_defs, data, exact=false)
reserves capacity on top of existing; exact=true for the ceiling.
Error arrays grow only on the failure path, so they are intentionally not
covered. check/test green; 2282 cases; exercised end-to-end (the [dynamic]u8
code pattern, factor-in-existing, nil args, exact ceiling, reserve no-op).
