Re: staggering runsv startup

From: Rafal Bisingier <ravbc_at_itec.com.pl>
Date: Fri, 5 Jun 2015 11:04:58 +0200

Hi,

On 2015-06-05 at 00:01 CEST
Laurent Bercot <ska-supervision_at_skarnet.org> wrote:

>On 04/06/2015 22:41, Jameson Graef Rollins wrote:
>> What I would like is to somehow stagger the startup of the processes, to
>> avoid the resource contention. I could do this by putting a random
>> sleep into the ./run scripts, but this would also cause random startup
>> delays on subsequent process restarts via "sv restart" or the like
>> (which we occasionally need to do).
>
> You could also have an "initial" ./run that would sleep a random amount
>of time, then atomically replace itself with the "normal" run script before
>executing itself.
> Of course, it requires read-write service directories. This is one of
>the reasons why I generally advocate keeping the working copy of
>service directories in a tmpfs.

There's even simpler method for this exact task. What is needed here is
really a random sleep before starting a service run script, but only in
some circumstances (eg. if it's run shortly after boot). What I'd
propose is to add a simple check to every run script in the form:

if [ -e /signaling_file ] ; then
        sleep RANDOM_VALUE
fi

and a new service with really simple run script:

sleep LONG_ENOUGH
rm /signaling_file

Of course it requires to recreate this signaling_file on every boot
before running runsvdir (or any other process supervision), but that is
a simple task. And of course this signaling_file, don't need to be on
root-fs, and don't even need to be a file, but could be reduced to:

total=`ls -1 /service | wc -l`
down=`ls -1 /service/*/down | wc -l`
running=`sv s /service/* |grep '^run: .* [0-9]*[0-9][0-9]s' |wc -l`
if [ $(( $total - $down - $running )) -gt SOME_SMALL_VALUE ] ; then
        sleep RANDOM_VALUE
fi

This last solution doesn't even need a writable fs or additional
"managing" service

-- 
Greetings
Rafal Bisingier
Received on Fri Jun 05 2015 - 09:04:58 UTC

This archive was generated by hypermail 2.3.0 : Sun May 09 2021 - 19:44:19 UTC