[MacPorts] #74226: outguess: intermittent SIGBUS/crash during JPEG writing caused by config.h's obsolete #define size_t unsigned

[MacPorts]

#74226: outguess: intermittent SIGBUS/crash during JPEG writing caused by
config.h's obsolete #define size_t unsigned
------------------------+--------------------
 Reporter:  HybridBits  |      Owner:  (none)
     Type:  defect      |     Status:  new
 Priority:  Normal      |  Milestone:
Component:  ports       |    Version:
 Keywords:              |       Port:
------------------------+--------------------
 '''Port:''' outguess
 '''Related to:''' #74183 (separate bug, same port -- see note at the end)
 '''macOS version:''' macOS Sequoia 15.7.7 and macOS Tahoe 26.2 (reproduced
 on both)
 '''Xcode version:''' 16.2
 '''Architecture:''' arm64 (Apple Silicon)

 == Summary ==

 outguess intermittently crashes with SIGBUS / EXC_BAD_ACCESS
 (KERN_PROTECTION_FAILURE) while writing the compressed JPEG output,
 leaving a 0-byte output file. The root cause is a stale, unconditional
 macro in config.h:

 {{{
 /* Define to `unsigned' if <sys/types.h> doesn't define.  */
 #define size_t unsigned
 }}}

 This autoheader-generated line's own comment describes conditional intent
 ("if <sys/types.h> doesn't define it") but the actual generated macro has
 no guard around it at all. On any modern 64-bit platform, <sys/types.h>
 already provides a correct, 8-byte size_t -- so this macro unconditionally
 and silently overrides it with a 4-byte unsigned throughout any file that
 includes config.h.

 jpg.c includes config.h. The bundled/patched libjpeg (jpeg-6b-steg) does
 not. The result: jpg.c and the JPEG library disagree about the layout of
 struct jpeg_destination_mgr -- specifically its size_t free_in_buffer
 field, declared correctly as 8 bytes in jpeglib.h, but compiled as 4 bytes
 wherever jpg.c's copy of config.h has already redefined the size_t token
 before jpeglib.h gets included.

 Concretely: jpg.c's init_destination() writes dest->pub.free_in_buffer =
 BUFSIZE; (256) as a 4-byte store. Code inside the JPEG library (e.g.
 jcmarker.c's emit_byte()) reads that same field back as a genuine 8-byte
 size_t, picking up 4 bytes of uninitialized garbage in the high half along
 with the correct 256 in the low half. The resulting value is enormous, so
 the buffer-refill check (if (--dest->free_in_buffer == 0)) never fires,
 and the output pointer walks past the end of its 256-byte scratch buffer
 until it eventually hits unmapped/read-only memory and crashes.

 == How this was found ==

 This surfaced while investigating a report of intermittent crashes and
 0-byte output specifically when the -e (error-correcting encoding) flag
 was combined with a from-source arm64 rebuild -- worse on Sequoia than on
 Tahoe, consistent with ASLR-dependent memory layout affecting whether the
 eventual out-of-bounds write happens to land somewhere fatal. Under an
 AddressSanitizer-instrumented build it reproduced 100% reliably (20/20
 runs), which made it possible to trace precisely via lldb:


 A conditional watchpoint on cinfo->dest->next_output_byte (breaking only
 when the pointer left the legitimate 256-byte buffer range) caught the
 pointer walking one byte past the end.
 cinfo->dest->free_in_buffer printed as an enormous garbage value instead
 of 256, and was already wrong immediately after the assignment that's
 supposed to set it -- not corrupted later.
 A raw memory dump (x/2gx) at that address showed the low 4 bytes correctly
 holding 256 and the high 4 bytes as untouched poison -- direct proof of a
 32-bit store into what should be an 8-byte field.
 Comparing the literal preprocessed struct definition each file's compiler
 produces (cc -E ... | grep -A20 "struct jpeg_destination_mgr") confirmed
 it directly: jpg.c compiles the field as unsigned free_in_buffer;, while
 jcmarker.c (inside the JPEG library) compiles the identical struct as
 size_t free_in_buffer;.
 Traced the disagreement to config.h's unconditional size_t override.


 == Patch ==

 Disable the obsolete override (kept as a comment, matching the file's
 existing style, rather than deleted outright, to preserve the historical
 context):

 {{{
 --- a/config.h
 +++ b/config.h
 @@ -12,7 +12,8 @@
 /* Define if you have a working `mmap' system call.  /
 / #undef HAVE_MMAP */

 -/* Define to unsigned' if <sys/types.h> doesn't define.  */ -#define
 size_t unsigned +/* Define to unsigned' if <sys/types.h> doesn't define.
 /
 +/ DISABLED -- see ticket for full explanation. On any modern 64-bit




 platform <sys/types.h> already provides a correct size_t; this





 unconditional override was causing a real struct-layout mismatch





 between jpg.c and the bundled JPEG library. /
 +/ #define size_t unsigned */





 /* Define if you have the ANSI C header files.  /
 / #undef STDC_HEADERS */
 }}}

 == Testing ==

 After applying this patch and rebuilding (both an ASan debug build and a
 plain optimized release build):


 Ran the same -e repro command that crashed 20/20 times before the fix, 20
 times against the patched build: 20/20 succeeded cleanly, full successful
 embeds each time, no crashes, no ASan errors.
 Also confirmed against a plain, non-ASan optimized release build (not just
 the debug/sanitizer build) -- same result, no crashes.


 == Possible relevance to #74183 ==

 This is a separate bug from #74183 (which is a stack buffer overflow in
 steg_adjust_errors, unrelated code path, confirmed against the MacPorts-
 packaged 0.4 build specifically). This ticket's bug was found and fixed
 against the original 2001 0.2 source. It has not been checked whether the
 0.3/0.4 lineage (which MacPorts packages) inherited the same config.h
 size_t override -- it may be worth checking whether that shim is still
 present there too, since if so this same fix would likely apply.

 Happy to provide the full patched jpg.c/config.h, further test details, or
 check the 0.4 source directly if that would help move this forward.

-- 
Ticket URL: <https://trac.macports.org/ticket/74226>
MacPorts <https://www.macports.org/>
Ports system for macOS