fix(xxd): signed left-shift overflow in -r mode #40246

(AI-assisted)

Problem:
Zig build failing since 966e7a98f5:

    FAILED   ...r/work/neovim/neovim/test/functional/editor/xxd_spec.lua @ 22: xxd handles long lines in revert mode
    Expected values to be equal.
    Expected:
    0
    Actual:
    134
    stack traceback:
    ...r/work/neovim/neovim/test/functional/editor/xxd_spec.lua:26: in function <...r/work/neovim/neovim/test/functional/editor/xxd_spec.lua:22>

`huntype` parses the address column by repeatedly left-shifting
`want_off` (a signed `long`) by 4 bits per hex digit. After 16 hex
digits the value occupies all 64 bits; the next shift moves a non-zero
bit into the sign bit -- signed overflow, i.e. undefined behavior.

This happens on the Zig build because it enables UBSAN. The bug was
pre-existing, but 966e7a98f5 added a test that exercises it.

Solution
Shift through `unsigned long` and cast back to `long`, making the
overflow well-defined wrap-around.
This commit is contained in:
Justin M. Keyes
2026-06-14 16:00:05 -04:00
committed by GitHub
parent 6f3446c970
commit f350e39b77

View File

@@ -442,14 +442,16 @@ static int huntype(FILE *fpi, FILE *fpo, int cols, int hextype, long base_off)
p = 0;
continue;
}
want_off = (want_off << 4) | n1;
// Cast through unsigned to avoid signed left-shift overflow (UB) when
// garbage input feeds more than ~16 hex digits into the address column.
want_off = (long)(((unsigned long)want_off << 4) | (unsigned)n1);
} else { // HEX_BITS
if (n1 < 0) {
p = 0;
bcnt = 0;
continue;
}
want_off = (want_off << 4) | n1;
want_off = (long)(((unsigned long)want_off << 4) | (unsigned)n1);
}
continue;
}