Buildbot Performance

[Jason Liu]

If the guests on a virtual server are exerting a heavy enough load that the
virtual host is not able to obtain the resources it needs, then the entire
system's performance, both physical and virtual, can be affected. I'm not
claiming to be familiar enough with the specifics of the situation to claim
that this is what's happening, but if it is, then using CPU pinning can
help. It's basically analogous to the situation where you overcommit the
memory too much, and the virtual host isn't able to have enough memory to
do the tasks it needs to.

On the virtual servers which I set up for my customers, I have the
hypervisor set up to automatically pin all but 2 cores. So for example, on
an 8 core machine, I pin cores 0-5 for all of the VM settings, so that none
of the guests are able to use cores 6 and 7. This effectively removes 2
cores from the pool of CPUs available for the guest to use, which means
that the virtual host will always have those 2 cores available to it at all
times. This allows my customer to run, on average, 10 guests on the virtual
server simultaneously, and everything stays performant, even under heavy
vCPU loads.

The other thing I do is that I tell my customers to never overcommit on
memory. My rule of thumb is that the sum of all simultaneously running
guests should never exceed the amount of physical RAM minus 4 GB. So, on a
server with 128 GB of RAM, the sum of the memory allocated to the guests
running simultaneously should never add up to more than 124 GB of memory.

-- 
Jason Liu


On Mon, May 17, 2021 at 6:24 PM Ryan Schmidt <ryandesign at macports.org>
wrote:

> On May 17, 2021, at 13:13, Jason Liu wrote:
>
> > Regarding CPU overcommitment: Are the virtual hosts doing any sort of
> CPU pinning? Many virtualization products have the ability to specify which
> of the pCPU cores a guest is allowed to use. As far as I can remember,
> products like KVM and ESXi can do CPU pinning, while VirtualBox cannot.
>
> Nope, nothing like that is set up. Is there any reason why we would want
> that?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macports.org/pipermail/macports-dev/attachments/20210517/03ec4cd8/attachment.htm>