SDL_CalculateRGBSize, SDL_CalculateYUVSize: set the error indicator

These functions historically didn't set the error indicator on overflow.

Before commit 447b508a "error: SDL's allocators now call SDL_OutOfMemory
on error", their callers would call SDL_OutOfMemory() instead, which was
assumed to be close enough in meaning: "that's a silly amount of memory
that would overflow size_t" is similar to "that's more memory than
is available". Now that responsibility for calling SDL_OutOfMemory()
has been pushed down into SDL_calloc() and friends, the functions that
check for overflows might as well set more specific errors.

Signed-off-by: Simon McVittie <smcv@collabora.com>
This commit is contained in:
Simon McVittie
2024-01-18 19:41:26 +00:00
committed by Sam Lantinga
parent e9a0edc847
commit 81ac656b7c
3 changed files with 28 additions and 22 deletions

View File

@@ -47,28 +47,27 @@ static int SDL_CalculateRGBSize(Uint32 format, size_t width, size_t height, size
{
if (SDL_BITSPERPIXEL(format) >= 8) {
if (SDL_size_mul_overflow(width, SDL_BYTESPERPIXEL(format), pitch)) {
return -1;
return SDL_SetError("width * bpp would overflow");
}
} else {
if (SDL_size_mul_overflow(width, SDL_BITSPERPIXEL(format), pitch)) {
return -1;
return SDL_SetError("width * bpp would overflow");
}
if (SDL_size_add_overflow(*pitch, 7, pitch)) {
return -1;
return SDL_SetError("aligning pitch would overflow");
}
*pitch /= 8;
}
if (!minimal) {
/* 4-byte aligning for speed */
if (SDL_size_add_overflow(*pitch, 3, pitch)) {
return -1;
return SDL_SetError("aligning pitch would overflow");
}
*pitch &= ~3;
}
if (SDL_size_mul_overflow(height, *pitch, size)) {
/* Overflow... */
return -1;
return SDL_SetError("height * pitch would overflow");
}
return 0;