[kwlug-disc] balancing CPU and memory for highly variable memory needs

[Khalid Baheyeldin]

On Mon, Oct 21, 2024 at 12:17 AM L.D. Paniak
<ldpaniak at fourpisolutions.com> wrote:
> I would be surprised if you could have a process accurately report the
> maximum amount of RAM it required before running.

Those are exactly my thoughts : how would a process know a priori how
much memory it will use?
There can be a system where a monitoring process records how much
max memory a process uses (with a single threaded compile), then records
that somewhere. Then on subsequent runs, if there is such a file, some
smart script can be used to parallelize those that consume less than
a certain threshold (based on how much RAM is free now), and single
thread those that are above the threshold.

Can be done, but no way to know without actually having an initial run.