An instruction carried four operands. The coprocessor transfers need
six -- `mcr p15, #0, r0, c0, c0, #0` -- and dropping the tail is not
merely printing short: an opc2 that is not zero encoded as zero, which
is a different instruction. Six operands do not fit alongside the rest
in 64 bytes, since Operand is 11 and Memory cannot go below 8, so
Instruction is 128 now and straddles two cache lines rather than one.
With room for them, the block itself turns out to have been wrong
throughout. CDP, MCR and MRC took CRd from bits 19:16, which is CRn;
CRd is 15:12. MCR and MRC read a four-bit opc1 where theirs is the
three bits at 23:21. Neither they nor CDP read opc2 at all. MCRR and
MRRC dropped CRm.
LDC and STC were worse: they took CRd from the base register's field,
read the offset with the halfword-load encoding rather than as an imm8
counting words, and their base pattern named P=0 W=0, which is the
unindexed form whose syntax is `{option}` and not what they printed.
They now have the offset, pre-indexed and post-indexed forms, with U
left out of the mask because it is the offset's sign and belongs to the
memory operand. The N bit asks the coprocessor for the long transfer
and the syntax spells that as an L, so LDCL, STCL, LDC2L and STC2L are
mnemonics of their own.
None of this is reachable from the table sweep, which decodes each
form's canonical word with every variable field zero -- for these that
means p0, which ARMv8 reserves and no assembler will take. So twelve
hand-picked words go in the pipeline test instead, checked against the
text an assembler accepts for them, including a p15 system-register
read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UmHLRF11EoWwNWCJ7JGaA
Forty-one groups of forms differed only in bits their own mask left
free, so they all matched each other's words and whichever sorted first
answered for the lot. That is where the NEON element sizes went:
`vabs.s8`, `.s16` and `.s32` are three patterns differing at bits 19:18,
and with those bits outside the mask the first one spoke for all three.
121 rows.
Two things that exposed:
- LDM and STM differ only in the writeback bit, and nothing in the
operand model recorded it, so the two forms printed identically.
Instruction carries it now and the base register wears its `!`.
- The T32 shifts left the S bit free, and their patterns sort ahead of
MOV's, so `movs.w r1, r2` came back as `lsl r1, r2, #0`. There is no
LSLS form in the table for it to have become instead; pinning the
bit at least stops it answering for a word that is not its own.
Also: r0 could not be used as an index register. The printer took a
register-offset operand to be immediate-only unless the index was
non-zero, when the class alone already says whether there is one --
`[r0, -r0]` printed as `[r0]`. And a displacement of zero lost its U
bit, which llvm-mc writes as `#-0` because it is a different word.
Against llvm-mc, A32 entries whose disassembly assembles back to the
same word: 760 of 1139, from 683. Wrong: 81, from 158.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UmHLRF11EoWwNWCJ7JGaA
`vpush {d8, d9}` came out as `vpush {r2}`. VFP list operands kept only
the 8-bit count from the encoding and stored it in a REG_LIST operand,
whose immediate the printer reads as a GPR bitmask -- so the register
bank was wrong, the start register was gone entirely, and the count was
read as a bitmask. `vldm r0, {d1, d2, d3}` printed `{r1-r2}`.
A list is now a register plus a shape: how many, and what the run steps
by. VLDM/VSTM/VPUSH/VPOP recover their start register from Vd and their
count from imm8. A GPR list stays a bitmask, since `{r4, lr}` is not a
run at all.
The NEON structure loads were wrong in a second way. VLD1-4/ST1-4 encode
their register count in the type field at bits 11:8, not in imm8, so
writing a count into the low byte overwrote size, alignment and Rm. Rm
was left at 0, which is a register post-increment, where the plain form
needs 0b1111; every one of those 23 forms encoded a writeback nobody
asked for. The count now comes from the form, the encoding writes only
Vd, and Rm is fixed in the pattern.
Deriving those counts from llvm-mc rather than by hand turned up a
dimension that was not modelled at all: the spaced forms step two
registers at a time (`vld2.8 {d2, d4}`), and VLD1 has a to-all-lanes
form written `{d2[]}`.
All 31 A32 list forms are now byte-exact against llvm-mc, from 8. The
arm32 sweep holds at 1656/1656.
Still open here: those forms only exist for .8 data, because the size
field at bits 7:6 is unencoded, and the writeback variants
(`[r0]!`, `[r0], r1`) are not modelled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UmHLRF11EoWwNWCJ7JGaA
Shrinking Instruction to 48 was the wrong call, and measuring it said so.
The premise was that a sub-cache-line struct touches fewer lines. It does --
but `#packed` aligns the struct to 1, so a 48-byte stride straddles a line
boundary 75% of the time, and the heap base is not line-aligned either. The
old 64-byte packed layout was worse still: 100% straddling, getting none of
the benefit its size implied.
Measured on an i7-9750H (L1d 32K/core, L2 256K, L3 12M), best-of-5, median of
3 interleaved rounds, against the 64-byte packed layout this branch started
from:
scan encode decode
64 packed (was) 1.000x 1.000x 1.000x
48 packed 0.909x 1.040x 1.022x
64 align(64) 1.218x 1.034x 0.806x
Decode is ~19% faster aligned, and that holds at every working set including
ones that fit entirely in L1 -- so it is split-store cost at the store ports,
not cache-line fetches. Decode writes whole Instructions, and the aligned
stores are worth more than the 33% extra bytes they move. A fourth variant --
48 bytes with `#packed` removed -- was measured to rule out the obvious
confound, and tracked 48-packed within 0.5% everywhere, so the win is
alignment and not the loss of packing.
Encode is within a few percent throughout (it is compute-bound; the form scan
dominates), and the pure read traversal is slower, but that is a synthetic
loop and its regression is codegen, not cache -- it is present even at
L1-resident sizes where a standalone struct shows no such penalty.
The Operand and Memory work from the previous commit is what makes this
possible: a 45/48-byte payload now sits inside one line with room to spare,
where the original spent all 64 bytes. The 16-19 spare bytes cost nothing over
a straddling 48-byte struct and give new fields somewhere to land.
All 11 rexcode suites match baseline; arm64 is 73/73 byte-exact against
llvm-mc; arm32's 1680/1680 sweep passes and its encode spot-checks are
unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
x86 keeps Instruction at 64 bytes -- one cache line -- by packing its memory
operand into a bit_field u64 rather than a struct. arm32 and arm64 both used
a 12-byte Memory struct, and since Memory sits in every Operand that width is
multiplied by four in every Instruction. Adopting x86's trick, plus two
smaller things, takes both ARM ISAs under the cache line.
Instruction ops[4] Operand Memory
x86 64 48 12 8
arm32 88 -> 48 72->40 18->10 12->8
arm64 64 -> 48 56->40 14->10 12->8
Memory -> bit_field u64, both ISAs. Field syntax and composite literals are
unchanged, so callers see nothing. Registers keep their type: an arm64
Register never exceeds 0x0C1F and an arm32 one never exceeds 0x401F, but the
arm64 NONE sentinel is 0xFFFF, so arm64 gives them the full 16 bits and arm32
15. What is left goes to `disp`: 23 bits on arm64 (worst case 65,520, from
LDR Q, [Xn, #imm12*16]) and 19 on arm32 (worst case 4,095, an A32 imm12) --
64x and 32x headroom respectively.
arm32 Operand also carried four tail bytes arm64 does not. `cond` was dead:
nine builders wrote it and nothing in the package ever read it, and
Instruction.cond already exists. shift_type/shift_amt/lane now ride inside
the union alongside the register they describe -- they only ever apply to a
register operand -- via a `using` bit_field, so op.reg, op.shift_type,
op.shift_amt and op.lane still read and write exactly as before.
arm32 Instruction packs cond, operand_count, mode, length and the two flag
bits into one 16-bit word; they need 13 bits between them and were spending
six bytes. `using` again keeps the field names, with the one exception that
inst.flags.sets_flags is now inst.sets_flags (five call sites).
Verified: every rexcode suite matches baseline; both generators stay
idempotent; arm64 is 73/73 byte-exact against llvm-mc on both encode and
decode round-trip; arm32's 1680/1680 sweep still passes and a memory/shift
encode spot-check is byte-identical to what the same code produced before
this commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
NEON reuses one operand shape across every element width, so `vadd.i8` and
`vadd.f32` are both DPR,DPR,DPR and only the type separates their encodings.
The type existed nowhere in the data: the encoder could reach the first form
of a shape and no other, and the printer reconstructed a suffix from the bit
pattern at print time. 489 of 1680 forms -- 29% of the table -- were
unreachable, and `inst_vadd(d0,d1,d2)` could only ever produce VFP vadd.f64.
Add `Data_Type` and carry `dt: [2]Data_Type` on Instruction, Encoding and
Decode_Entry. Two slots because the convert family names both ends
(`vcvt.s32.f32`); everything else leaves the second .NONE. In A64 the
arrangement belongs to each operand (`add v0.4s, v1.4s, v2.4s`); in A32 it
belongs to the instruction, which is why it goes here and not on Operand.
Instruction does not grow: it lands in bytes that were already padding, so
88 stays 88. Encoding and Decode_Entry go 21 -> 23, which is +3,360 B per
table, +6.7 KB in all.
The per-form type is derived from llvm-mc rather than hand-written: assemble
each form's canonical word, disassemble it, take the suffix. 942 forms carry
one, 38 carry two. (`.w` is the Thumb wide qualifier, not a type, and is
excluded.)
Effect: of 202 shape groups holding more than one form, 168 are now separated
by the type -- 429 of the 489 unreachable forms become selectable. `dt` left
at .NONE means "unspecified" and still takes the first matching form, so
every existing caller behaves exactly as before.
It also fixes printing. The old inference could only ever produce one type,
so the whole convert family printed `vcvt.f32` -- 13 forms sharing one string
that no assembler accepts. They now print `vcvt.f32.s32`, `vcvt.f64.f32`,
`vcvta.u32.f64`, and so on.
Verified: vadd.i8/i16/i32/i64/f32 encode to f2010802 / f2110802 / f2210802 /
f2310802 / f2010d02, matching llvm-mc exactly; all 11 rexcode suites are
identical to baseline.
Still unreachable, 60 forms in 34 groups: register lists (VLD2-4/VST2-4),
LDM/STM addressing modes, and a few lane-indexed and fixed-point convert
forms whose element size is not captured by the type alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Move all ten ISA packages (x86, arm32, arm64, mips, riscv, ppc, ppc_vle,
rsp, mos6502, mos65816) from core/rexcode/<arch> to core/rexcode/isa/<arch>,
so the import pattern is now `import "core:rexcode/isa/x86"`. The shared
core stays at core:rexcode/isa.
Mechanical: relative `import "../isa"` / "../../isa" -> absolute
"core:rexcode/isa" (the only path that survives the move; the "../" and
"../.." self/generated imports move with their packages). build.lua now
builds paths as <root>/isa/<name>; stale `cd <arch>` hints in the verify
tools and the doc.odin paths updated.
WASM stays at core/rexcode/wasm for now -- it is an IR, not an ISA, and
will move under the forthcoming core:rexcode/ir once that layer lands.
All 10 arches gen/builders/check/test green; import core:rexcode/isa/x86
verified working; wasm still compiles.
