[MacPorts] #74183: outguess @0.4: -e (error correction) crashes with SIGABRT / stack smashing detected in steg_adjust_errors
MacPorts
noreply at macports.org
Sun Jul 12 18:28:29 UTC 2026
#74183: outguess @0.4: -e (error correction) crashes with SIGABRT / stack smashing
detected in steg_adjust_errors
-------------------------+--------------------
Reporter: HybridBits | Owner: (none)
Type: defect | Status: new
Priority: Normal | Milestone:
Component: ports | Version:
Resolution: | Keywords:
Port: outguess |
-------------------------+--------------------
Comment (by HybridBits):
== Update: root cause found, patch attached, confirmed fixed ==
Follow-up to my original report. I was able to trace this down to an
actual bug in the source, not just something specific to the MacPorts
build — and it appears to affect '''every version''', including the
original 2001 0.2 release from Niels Provos' own site (provos.org), not
just the 0.3/0.4 lineage this port is based on. The reason it hadn't
surfaced before is almost certainly platform/toolchain-dependent (see "Why
this went unnoticed" below), not a difference between the versions
themselves.
== Root cause ==
The crash is an off-by-one stack buffer overflow in `steg_adjust_errors()`
(in `outguess.c`).
{{{
void
steg_adjust_errors(bitmap *bitmap, int flags)
{
int i, j, n, many, flag;
int priority[ERRORBITS], detect[ERRORBITS]; // stack arrays,
valid indices 0..ERRORBITS-1
many = ERRORBITS - steg_errors;
for (j = 0; j < many && j < steg_err_cnt; j++) {
priority[j] = steg_err_buf[j];
detect[j] = bitmap->detect[priority[j]];
}
...
for (i = j; i < steg_err_cnt; i++) {
for (n = 0; n < j; n++)
if (detect[n] < bitmap->detect[steg_err_buf[i]])
break;
if (n < j - 1) {
memmove(detect + n + 1, detect + n, (j - n) *
sizeof(int));
memmove(priority + n + 1, priority + n, (j - n) *
sizeof(int));
}
...
}
}
}}}
`steg_errors` is bounded to the range 0–4 elsewhere in the file
(`steg_embedchunk` returns early once it exceeds 3), so `many` can be as
large as `ERRORBITS` itself. That means the first loop can leave `j ==
ERRORBITS`.
The second loop then performs an insertion-sort-style shift: to make room
at index `n`, it moves elements `[n, j)` up to `[n+1, j+1)` — which writes
into index `j`. That's safe only when `j < ERRORBITS`. When `j ==
ERRORBITS` (which the first loop permits), this writes to
`detect[ERRORBITS]` / `priority[ERRORBITS]` — one element past the end of
both stack-allocated arrays, corrupting the stack canary and producing
exactly the crash reported here (`__stack_chk_fail` → `SIGABRT`).
Whether this triggers is data-dependent: it requires enough bits to be
flagged as "locked/conflicting" (via `steg_err_buf`, populated in
`steg_embedchunk`) to reach `ERRORBITS` before the periodic per-block
reset. That's why it doesn't crash on every embed — only on
message/image/key combinations that happen to accumulate enough conflicts
in one error-correction block.
== Why this went unnoticed until now ==
I was able to reproduce the identical crash (same signal, same backtrace
through `steg_adjust_errors` → `steg_embedchunk` → `steg_embed` →
`do_embed`) on a completely independent, freshly-compiled arm64-only build
of the '''original 0.2 source''' from provos.org — i.e. this is not
specific to whatever patches the 0.3/0.4 lineage added. My working theory
is that this binary had previously only ever been run via Rosetta 2
(x86_64) translation on Apple Silicon, and/or on older x86_64 toolchains
without stack-protector-strong enabled by default, and one of those
factors likely avoided tripping the canary even though the same out-of-
bounds write was presumably happening. A native arm64 build with a modern
Clang's default stack protector catches it reliably instead of letting it
silently corrupt memory.
== Patch ==
Minimal, one-line fix — reserve one slot of headroom so `j` can never
reach `ERRORBITS`:
{{{
--- a/outguess.c
+++ b/outguess.c
@@ -131,7 +131,9 @@ steg_adjust_errors(bitmap *bitmap, int flags)
int i, j, n, many, flag;
int priority[ERRORBITS], detect[ERRORBITS];
- many = ERRORBITS - steg_errors;
+ /* -1: the loop below may write priority[j]/detect[j] when
shifting
+ * to make room for an insert; that index must stay in bounds. */
+ many = ERRORBITS - 1 - steg_errors;
for (j = 0; j < many && j < steg_err_cnt; j++) {
priority[j] = steg_err_buf[j];
detect[j] = bitmap->detect[priority[j]];
}}}
This doesn't change the algorithm's behavior in any meaningful way (it
caps the max number of tracked high-priority error-correction slots at
`ERRORBITS - 1` instead of `ERRORBITS`) — it just keeps the later
insertion-sort shift from ever writing outside the array.
== Testing ==
After applying this patch and rebuilding natively for arm64:
* Ran `-e` (error-correcting encoding) against 5 different JPEG images
with a 656-byte payload — all 5 completed successfully (previously crashed
5/5 before the patch, same inputs).
* Successfully retrieved the embedded data back out of the resulting
output files, confirming round-trip correctness, not just "didn't crash."
* Sample successful output (one of five):
{{{
Extracting usable bits: 41870 bits
Correctable message size: 19052 bits, 45.50%
Encoded '...' with ECC: 5248 bits, 656 bytes
Embedding data: 5248 in 41870
Bits embedded: 5344, changed: 2156(40.3%)[79.0%], bias: 1545, tot: 41794,
skip: 36450
Foiling statistics: corrections: 976, failed: 7, offset: 66.845679 +-
183.930449
Total bits changed: 3701 (change 2156 + bias 1545)
Storing bitmap into data...
Writing .../out.jpg....
}}}
Happy to provide any further detail (full crash logs, test images, or a
proper unified diff against a specific released tarball) if that's useful
for getting this into the port. Given upstream (`resurrecting-open-source-
projects/outguess` on GitHub) is archived and can no longer accept issues
or PRs, this may be the only place left where a fix like this can land.
--
Ticket URL: <https://trac.macports.org/ticket/74183#comment:2>
MacPorts <https://www.macports.org/>
Ports system for macOS
More information about the macports-tickets
mailing list