[MacPorts] #59691: After a time, MacPorts cannot determine the Xcode version, on Snow Leopard

[MacPorts]

#59691: After a time, MacPorts cannot determine the Xcode version, on Snow Leopard
-------------------------+--------------------
 Reporter:  ryandesign   |      Owner:  (none)
     Type:  defect       |     Status:  new
 Priority:  Normal       |  Milestone:
Component:  base         |    Version:  2.6.2
 Keywords:  snowleopard  |       Port:
-------------------------+--------------------
 After a Snow Leopard machine has been up for a time, MacPorts becomes
 unable to determine what Xcode version is installed.

 For example, most recently,
 [https://build.macports.org/builders/ports-10.6_x86_64-builder/builds/7473/steps
 /install-port/logs/stdio build 7473]
 was able to determine the Xcode version:

 {{{
 DEBUG: Xcode 3.2.6
 }}}

 And the next build in the batch,
 [https://build.macports.org/builders/ports-10.6_x86_64-builder/builds/7474/steps
 /install-port/logs/stdio build 7474],
 was unable to:

 {{{
 DEBUG: Xcode none
 }}}

 This in turn caused all subsequent builds to fail with:

 {{{
 Error: Port foo requires a full Xcode installation, which was not found on
 your system.
 Error: You can install Xcode from the Mac App Store or
 https://developer.apple.com/xcode/
 }}}

 even for ports that don't specify `use_xcode yes`.

 Restarting the machine fixes the problem.

 When the problem is occurring, `xcodebuild -version` still returns the
 expected Xcode version information when run manually from the command
 line. So I don't think the problem is Xcode.

 How long the system will work after a restart seems to vary. In this most
 recent case, the problem occurred after just 12 days of uptime. But the
 10.6-10.8 workers have been unusually busy, working non-stop to rebuild
 packages after the switchover to libc++, so the number of builds or
 perhaps the number of times MacPorts tries to exec `xcodebuild -version`
 (or perhaps the number of times it tries to exec anything) may be a more
 relevant measure.

 I am now wondering if the problem is that a tmp directory needed by
 MacPorts or Tcl to execute the `xcodebuild -version` command and process
 its results is getting deleted or its permissions are getting set
 incorrectly, which restarting the machine fixes, or if perhaps we are
 leaking file descriptors and at some point we just don't have any free
 descriptors left to use one to get the `xcodebuild` process's output.
 Perhaps we could log relevant tmp directory permissions and number of open
 files in mpbb somewhere.

 I have not observed this on any other macOS version, only on Snow Leopard.
 Yet it has been a persistent problem for our x86_64 Snow Leopard buildbot
 worker for the three years the new buildbot system has been online. (I
 don't remember if we saw the problem on the prior buildbot system.) When
 the problem occurs, it causes builds to fail; someone has to notice that,
 restart the worker, and manually reschedule the failed builds. It would be
 good to identify and fix the cause of the problem, since aside from making
 the life of the buildbot administrator easier, it may turn out to be
 responsible for other as yet unexplained MacPorts misbehavior.

 It's not a corruption of the OS on a specific macOS install. When we
 switched to libc++, we switched to a new 10.6 virtual machine, which
 exhibits this same problem that we saw on the old libstdc++ VM. I don't
 think I've seen the problem on the old or the new i386 Snow Leopard
 worker, but that needs to be restarted often for other reasons (kernel
 panic when building perl modules sometimes, due to i386 VMs not being
 truly supported by VMware ESXi) so we may just be avoiding the problem
 there by restarting the system before the problem hits.

-- 
Ticket URL: <https://trac.macports.org/ticket/59691>
MacPorts <https://www.macports.org/>
Ports system for macOS