Re: s6-svstat up, down and ready time not correct after system timestamp update.

From: Laurent Bercot <ska-skaware_at_skarnet.org>
Date: Tue, 18 Jul 2023 20:35:02 +0000

>I am setuping s6 for managing services on mine Linux embedded system.
>Everything is fine. But I faced issue related to system datetime change.
>My system does not have RTC, but it has GNSS module (managed by gpsd).
>After GNSS get the location and time chronyd service update system time.

  And there's your problem: you cannot rely on timestamp data across
system clock changes, so any service you would need accurate timestamp
data for would need to be started *after* chronyd first updates your
system clock.

  I know it's a pain. I have modified s6, and all my software, to rely
on CLOCK_REALTIME as little as possible. But sometimes it's unavoidable,
and then you need to be able to trust your system clock, or only use it
after it has been updated to something reasonable.


>I found that s6-svstat uses CLOCK_REALTIME and I think it will be more robust to use CLOCK_MONOTONIC.

  Unfortunately, no, it will not. The timestamps written by s6-supervise,
and used by s6-svstat, are snapshots of the absolute date at a given
moment: this is exactly what CLOCK_REALTIME is for. It's the same kind
of time data that, for instance, s6-log prepends log lines with when you
use the t directive: an absolute timestamp. It is really wallclock time,
not stopwatch time.

  If stopwatch time (i.e. CLOCK_MONOTONIC) were to be used here, sure,
you would get more stability across system clock changes for
s6-svstat -o updownfor,readyfor ; but you would lose all meaning for
s6-svstat -o updownsince,readysince as well as s6-svdt, s6-permafailon
and maybe others. Stopwatch time can only be used to compute intervals,
never to share absolute timestamps.

  There is no absolute point of reference for sharing CLOCK_MONOTONIC
time across processes, unless you store an offset in the filesystem
at boot time and by convention all your software accesses this offset.
This isn't specified anywhere, because sharing absolute time across
processes is already covered by CLOCK_REALTIME, which is significantly
simpler - it only gets messy when the system performs a significant
system clock change, which should only happen at most once and in the
early life of a system. You are in this case and I know it's ugh, but
there is nothing I can do about that.


>Is the patch correct? Maybe I miss something (maybe some other utils also need to be patched).

  Even if you wanted to use stopwatch time, no, the patch would not be
correct, because CLOCK_MONOTONIC has no absolute meaning - and a tain
containing a raw CLOCK_MONOTONIC value would be unusable.
  The function you are looking for is tain_stopwatch_read():
https://git.skarnet.org/cgi-bin/cgit.cgi/skalibs/tree/src/libstddjb/tain_stopwatch.c
but it has to be paired with an initial call to tain_stopwatch_init(),
which computes the offset so that you get reasonable absolute times -
but this offset is only valid for the current process, and you should
never share these absolute times with other processes, because they
will be increasingly inaccurate with clock drift and difference between
time of creation and time of use.

  In fact, s6-supervise *does* use CLOCK_MONOTONIC: there is an initial
call to tain_now_set_stopwatch_g() here:
https://git.skarnet.org/cgi-bin/cgit.cgi/s6/tree/src/supervision/s6-supervise.c#n836
This means that afterwards, timestamps obtained by calls to tain_now_g()
will use CLOCK_MONOTONIC. But these timestamps are only used internally,
so that the timeout computations for iopause() (the main event loop)
are resilient to system clock changes - because s6-supervise, as you are
experiencing, can be used very early, before the system clock is
properly
set, so there is a reason to use CLOCK_MONOTONIC here.
  The timestamps that are meant to be shared with other processes,
however, are all obtained via tain_wallclock_read(), which uses
CLOCK_REALTIME, and that is on purpose.

  I understand this is not the answer you were looking for, but it's the
only one I've got. If you cannot live with the inaccurate report of
s6-svstat -o updownfor,readyfor then my suggestion is to use s6-rc to
delay the start of your sys-dbus (and friends) services until your
initial system clock change has happened.

--
  Laurent
Received on Tue Jul 18 2023 - 22:35:02 CEST

This archive was generated by hypermail 2.4.0 : Tue Jul 18 2023 - 22:35:31 CEST