See, this is why I prefer technical discussions first and *maybe*
patches later:
1. I'm not sure exactly what feature you want;
2. That's a bit too much change for me to read the diff in-line and
insta-review it. It requires more effort. Which I'm not willing to do
if I don't know what the change is supposed to be doing.
>Using forx in parallel mode with a large number of arguments can lead to
>all instances not being able to finish their jobs due to resource
>constrains.
That's only going to be a problem for an insane amount of arguments,
which have to be explicitly listed in the block, so I don't think it's
a real problem in the first place. If your machine isn't powerful
enough to handle 5k arguments, don't use forx -p with a block that
has 5k arguments?
> With an option to specify a max amount of parallel instances
>this can be handled properly.
From what I can understand at first glance, your change puts a cap
of maxpar over the amount of arguments that can be processed, and the
remaining arguments are just ignored. If it is the case, I don't think
it's useful, because maxpar has to be input by the user, and the user
can already know how many arguments there are in the block, so the
list can be truncated prior to the forx invocation.
Better semantics for -P $maxpar (what I thought you had implemented
before looking at the diff) would be a cap over the number of
arguments that are *processed in parallel*, akin to make -j$maxpar,
while still processing the full list; that's a feature that I would
consider implementing, but it's a little more involved, requiring a
full refactor of the loop to always have a window of $maxpar children
at a time.
--
Laurent
Received on Thu Jul 11 2024 - 17:32:17 CEST