MacPorts webapp project updates

[Ryan Schmidt]

On May 18, 2020, at 16:29, Mojca Miklavec wrote:

> On Mon, 18 May 2020 at 21:47, Ryan Schmidt wrote:
>> On May 17, 2020, at 11:48, Arjun Salyan wrote:
>> 
>>> We also plan to add a caching engine so that exploring the webapp remains smooth even with the ever-growing port statistics data. This makes the need for a new host even more important.
>> 
>> Not sure specifically what you're planning with regard to caching, but note that we already have an arrangement with the MaxCDN content delivery network and we currently use it for www, guide, distfiles and packages. We can also use it for the ports webapp if desired. It would just be a matter of tuning the settings to decide how long the data should be cached.
> 
> The caching is primarily not so much about big data files (or actual
> troubles with tons of visitors) as it is about whether the query "Show
> me on which macOS versions the wine port is broken" needs to be
> executed in the database every single time any random visitor opens or
> refreshes the port page (even if the port didn't see any changes for
> months and meanwhile thousands of visitors query the page). I don't
> know to what extent MaxCDN can help here given that the main problem
> is the lack of memory required to even run the site on the existing
> server.
> 
> We can certainly check whether we could make use of MaxCDN (I don't
> know to what extent a page that's dynamic in nature can be supported
> there), but the immediate "urgency" is to find an alternative hosting
> solution altogether.

The CDN could probably help. If we set that up for the web app, then when a user requests a port page the CDN would check if it has a cached copy of the page and it is not "too old", and if so it prints the cached copy. It only contacts the origin server if it doesn't already have an up-to-date version of the page.

The origin server gets to decide what "too old" means by setting the Cache-Control headers that specify when the contents of the page should be considered expired. Or the origin server's headers can be overridden by settings in the CDN. How to set the page expiration time depends on how old you want content to be able to get. For example, if I update a port today, what's the maximum amount of time you want users to have to wait before they see that change reflected on the web? Or tickets or build results or whatever other information is part of the page.