Buildbot Performance

[Christopher Nielsen]

It used to be that you had to pay for vCenter, to get deep insights into VM performance. But ‘esxtop' is certainly a great starting point. And perhaps it does provide enough info, to get an idea of where time is being spent.

> On 2021-05-17-M, at 02:55, Daniel J. Luke <dluke at geeklair.net> wrote:
> 
> I'm not an ESXi expert but after a quick look through some of their documentation it looks like there's an `esxtop` command that can show some information. More info here: https://kb.vmware.com/s/article/1005362 (some google searches seem to indicate that when the oversubscription is a problem, it's usually because ESXi is waiting for 'enough' cores to be available to start all of the vCPUs - and that this used to be much worse in older ESXi versions, but can still be a problem). This post also has information that looks useful: https://www.heroix.com/blog/vmware-vcpu-over-allocation/ <https://www.heroix.com/blog/vmware-vcpu-over-allocation/>

Ryan, you’re right, it wasn’t bad before. And let me clarify that none of my comments were meant to criticize our setup; rather, I’m simply trying to provide some ideas to potentially improve build times.

However, things have definitely slowed down with one of the Xserve’s out of the mix. And if we could get back to four, with builders spread evenly amongst the hardware, that would help.

As for overcommitment: I’m simply suggesting that we reduce the number of vCPUs per builder, from eight to six. Otherwise, the formula MacPorts base uses to calculate builds jobs and such is fine as-is, and I’m not advocating that we change it.

> On 2021-05-16-S, at 15:38, Ryan Schmidt <ryandesign at macports.org> wrote:
> 
>> So we need to do something, as the buildbots simply can’t keep up.
> 
> I think they've been keeping up fine, except when a bunch of huge things need to be built, which I find understandable.
> 
>> Upgrading them to six-core Xeons would absolutely help, for sure. But I’m quite certain that we could also improve the situation, by reducing the level of CPU overcommitment. And reducing the vCPUs per VM would help, as we simply don’t have the physical CPU power to support eight/VM.
> 
> When you say reduce CPU overcommitment, by what means are you referring to, other than reducing CPUs per VM?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20210517/d764c518/attachment.htm>