Add mem_rip_label(label_id) so a RIP-relative memory operand can
reference a label: the encoder writes a placeholder disp32 and emits a
REL32 relocation (addend 0) at the field's byte offset, mirroring the
existing .RELATIVE jump/call path. This expresses lea reg, [rip + <label>]
(position-independent data addressing).
Add op_imm_label(label_id) for movabs reg, <label>: the imm stays kind
.IMMEDIATE (form matching unchanged) but is flagged so the encoder emits
an ABS64 relocation for the imm64 instead of a literal; imm_matches_inline
forces the full IMM64 form so a small id can't collapse to mov r64, imm32.
Both labeled forms bypass the contextless recipe fast-path (which cannot
append relocations) and fall back to the interpreter. Flags reuse spare
bits in Memory (disp_is_label) and Operand_Flags (imm_is_label) -- no
struct growth. Section 11 tests cover resolved/unresolved disp, the ABS64
movabs, and an end-to-end executed lea+load.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fix the operand-size/prefix bug class reported by the sigil backend, then close
every failure a whole-ISA encode->decode->re-encode audit surfaced -- 247 -> 0
over the 1165 round-trippable legacy forms.
Encode/decode:
- MOVSX/MOVZX take 66h from the destination, not the source (CRC32, whose r/m
source drives 66h, is unchanged)
- operand-less 16-bit forms (CBW/CWD, MOVSW/CMPSW/SCASW/LODSW/STOSW,
IRET/PUSHF/POPF) emit 66h via a new opsize_16 flag
- CRC32/MOVBE and POPCNT/LZCNT/TZCNT: mandatory F2/F3 beats 66h on decode
- mandatory-prefix REP double-encode: F2/F3 no longer re-emitted as REP
- fixed-ModR/M decode: x87 (FNOP/FCHS/FADD ST(i)/...) and the 0F 01/AE/C7/1E
system-op groups (VMCALL/RDTSCP/LFENCE/ENDBR/...); gen.odin preserves the
fixed ModR/M byte, the decoder matches and consumes it
- implicit-operand round-trip (accumulator short forms, x87 ST(i), xchg) via a
positional matcher + emit path; accumulators stay implicit so hand-built
`add eax, imm` keeps the general form (typed-builder consistency)
- MOV to/from CR/DR/segment (were decoding as GPRs); reg-vs-mem disambiguation
(RDRAND vs VMPTRLD, MOVLHPS vs MOVHPS); x87 M80 operand size; PINSRW; NOP vs
XCHG at 0x90
- LOCK standalone; ENTER two immediates; PUSH/POP FS/GS (segment fixed by the
opcode + non-cacheable); BOUND marked 32-bit-only; RDRAND/RDSEED r16
(66 is operand-size, resolved against the mandatory-66 VMCLEAR by ModR/M)
Tools/tests:
- revive dump_verify_input + verify_against_llvm (stale Result API; drop the
aliases that were masking the fixed bugs); align verify_tables with the
preserved fixed-ModR/M ext
- new tests/test_narrow_widths.odin: byte-exact vs llvm-mc + round-trip guards
- fold the bit-rotted tests32 package into the main suite (i386 Mode._32 paths)
248 tests pass; verify_tables passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01115ktMoX5qGAfp1VemBzCP
Move ENCODE_RECIPES from a startup-built (@init) static table in the library to
a generated, #loaded table like every other one. The tablegen runs
form_to_recipe over every form, serializes to x86.encode_recipes.bin, and the
loader #loads it -- no @init, no allocation; the forms remain the single source
of truth and the recipe is derived from them.
- tablegen/gen.odin: ENCODE_RECIPES added to the BLOBS manifest (which drives
both the Stage B serializer and the loader's #load); emit the generated-package
global plus an @init that fills it from ENCODE_FORMS via lib.form_to_recipe.
- encoder_recipe.odin: drop the @init, the static storage, and the ENCODE_RECIPES
declaration. form_to_recipe stays -- now solely the tablegen-time derivation.
- tables.odin, generated/{encode_tables,writer}.odin: regenerated.
- tables/x86.encode_recipes.bin: the generated blob (28260 bytes = 2355 x 12).
No behavior change: the recipes are identical to the @init ones. gen / builders /
check / test / idempotent green (2282 cases, 22 byte-stable artifacts); encode
perf unchanged.
The matcher path (generic builders, hand-built, decode->re-encode) resolves an
instruction to its encoding form by linearly scanning the forms for its mnemonic
and operand-matching each -- the dominant cost on that path. Memoize it: pack
(mnemonic, per-operand shape) into a key (immediates folded to the smallest size
class they fit, matching imm_matches_inline) and cache key -> form so a repeated
instruction shape skips the scan.
Direct-mapped, fixed 8192-slot table (64 KB, no allocation). Each slot packs the
full 48-bit key and form index into one u64, read/written with relaxed atomics,
so concurrent encode() stays safe -- a reader sees a matching key or rescans,
never a torn entry. The scan stays the source of truth (a miss runs it and
records the result), so the cache is exact.
Lookup + scan live in a non-inlined find_form() so they don't bloat encode()'s
hot loop and slow the hint path that shares it. (Routing the matcher path through
the recipe emit was tried and dropped: it costs the hint path ~1.2-1.5 ns however
isolated -- the hot loop is too codegen-sensitive -- while the cache alone is
free for the hint path.)
Realistic generic-builder mix: matcher ~52 -> ~35 ns/inst (~1.49x); hint path
unchanged. Byte-exact across 2282 + idempotent.
Extend emit_recipe to the full ModR/M + SIB + displacement addressing (register
direct, RIP-relative, absolute [disp32], and base/index/scale/disp), mirroring
the interpreter byte-for-byte, and drop the caller's reg-direct guard so memory
operands take the fast path too. Only a label/relative immediate (a relocation)
still falls back.
Realistic immediate-heavy mix: ~20.1 -> ~12.9 ns/inst vs the pre-recipe base
(~1.55x, 50 -> 77 M/s). Byte-exact across 2282 + idempotent.
Precompute each encoding form into a flat Form_Recipe -- prefix byte, escape+
opcode blob, role->operand-index slots, ext, imm size, flags -- so the encoder
replays common forms straight-line instead of re-interpreting enc.ops/enc.enc
on every instruction (the resolve scan, escape ladder, prefix/REX selection).
encode() takes the fast path when the form is hinted, eligible, has a register
r/m and a literal immediate; everything else falls through to the existing
interpreter, which stays the byte-exact source of truth. First cut:
- reg-direct ModR/M only (memory r/m falls back)
- hint path only (matcher / generic builders fall back)
- ~33% of forms eligible (VEX/EVEX, 16-bit operand-size, x87 fixed-ModR/M,
moffs/far/rel/implicit operands are marked ineligible)
Recipes are built at startup into static storage (no heap); this moves into the
table generator (#loaded like every other table) once the shape settles.
Realistic immediate-heavy mix: ~19.0 -> ~16.3 ns/inst (52.7 -> 61.3 M/s).
Byte-exact across 2282 cases + idempotent.
Next: memory r/m addressing in the fast path, then the matcher path, then the
gen-time port.
Roll the encode/decode buffer-sizing helpers (added for x86 in 49787b7de) out
to every other ISA, and document them in the cross-arch naming contract.
Per arch (arm32, arm64, mips, riscv, ppc, ppc_vle, rsp, mos6502, mos65816):
- encode_max_code_size / encode_max_relocation_count now key off the
[]Instruction slice (were int counts); bodies unchanged (* MAX_INST_SIZE).
- encode_reserve(code, relocs, instructions): grows the caller's code []u8 by
length and reserves relocs by capacity; allocates no new buffers.
- decode_max_instruction_count / decode_estimate_instruction_count: exact
ceiling and typical estimate, keyed off the min/avg instruction size per
arch (fixed-4: arm64/mips/ppc/rsp; min-2: arm32/riscv/ppc_vle; min-1: mos).
- decode_reserve(instructions, inst_info, label_defs, data, exact=false).
docs/cross_arch_design.md: helpers added to the naming contract.
No behavior change to the existing size helpers (signature only). All 10 ISAs
check + test green (x86 2282, arm32 600, arm64 461, mips 281, riscv 154, ppc 31,
ppc_vle 281, rsp 70, mos6502 148, mos65816 53).
