Call for Legacy Support Testing

[Fred Wright]

On Wed, 4 Jun 2025, Sergey Fedorov wrote:

> On ppc64 tests fail:

This much is sufficient for any given failing test:
[...]
> test/test_clocks
> *** clock mach_continuous_time failed (-6): Too many consecutive unchanged
> samples
>  @125: 18446744073709551615->18446744073709551615(0), numsame = 125/124,
> retries = 0
> *** test clock mach_continuous_time failed (-6): Too many consecutive
> unchanged samples
>  @125: 18446744073709551615->18446744073709551615(0), numsame = 125/124,
> retries = 0
> test_clocks failed.
> make: *** [run_clocks] Error 1
[...]

> (Universal testing also fails on ppc64 part.)

It's surprising that this would care about ppc vs. ppc64, though maybe 
it's just luck.  You might try it a few times to see if it's consistent.

Testing clocks is tricky because they're inherently moving targets.  It's 
not just a "try X and expect Y" kind of test.  One has to come up with 
criteria that are tight enough to catch real problems, while being loose 
enough to accommodate normal variability.  The old clock test copped out 
by being what was really a manual test masquerading as an automatic test. 
The only way to see whether it passed was by inspecting the output. 
Given that I typically test literally hundreds of combinations of CPU, OS 
and SDK, I *really* don't want to have to manually check every test 
output. :-) Which is why I wrote the new test.

It's known that the current new clock test has some occasional false 
positives, but I thought the failure rate was low enough to live with for 
this release.  Maybe that's not true of the G5.  AFAIK, all issues are 
with the test, not the implementation.  Note that, although 
mach_continuous_time() is being provided by legacy-support, it's just 
mach_absolute_time() (a standard OS function since 10.0) with an offset 
that should be constant in any given run (and zero if the system has never 
slept since boot).  It may be that the combination of the CPU speed and 
the timebase rate is such that the clock appears "stuck" for longer than 
the test tolerates, though that doesn't explain any difference between 
running as ppc and as ppc64, nor why it doesn't also show up with 
mach_absolute_time().

Running the clock test with the "-DVV" options will capture useful 
logfiles on failures (and report as such).  The Makefile recognizes the 
TEST_ARGS environment variable as options to pass to all tests, but it 
won't work in the port environment unless you whitelist it in 
macports.conf.  But after "port test" has completed, the test is already 
lying around as ${wrksrcdir}/test/test_clocks, so you can just run it 
directly from there. Send me any logs privately; you definitely don't want 
to post that much verbiage to the list.

> If you could make a provisional fix, I will test that.

OK.  Setting up a repo for the macports-legacy-support project (if you 
haven't already done so) would probably be a more convenient way to do 
this.  Then, in the repo root, it's just:

 	$ make -s run_clocks TEST_ARGS=-DVV

To run the full set of tests, including testing all three versions of the 
library:

 	$ make -s test_all

I'll need one or more capture logs to look at.  If this is the only issue 
and it only applies to the G5, we might consider releasing this version 
anyway, particularly given the libgit2 issue.

Fred Wright