The s6-permafailon program
s6-permafailon is a program that is meant to be used in the ./finish script of a service directory supervised by s6-supervise. When used, it reads and analyses the death tally of a service (i.e. the recent process death events that happened), and if the death tally matches a given pattern, it causes permanent failure of the service, i.e. it tells the supervisor not to try and restart it.
Interface
s6-permafailon secs deathcount events prog...
- s6-permafailon must have the service directory of the tested service as its current directory. This is the default if it is called from the finish script of the service.
- It reads the death tally of the service, which is maintained by s6-supervise.
- If the supervised process has died at least deathcount times in the last secs seconds with a cause listed in events, then s6-permafailon exits 125.
- Else s6-permafailon execs into prog....
events is a comma-separated list of events. An event can be one of the following:
- An exit code, which is an integer between 0 and 255. Example: 1
- An exit code interval, which is two exit codes separated by a dash. Example: 1-50
- A signal name, or a signal number preceded by "SIG". Examples: SIGTERM, sigabrt, sig11
Usage
- s6-supervise detects when the ./finish script of its service exits 125, and stops respawning the service. So, if the ./finish script is a chain-loading command line starting with a s6-permafailon invocation (or containing such an invocation), when s6-permafailon exits 125, then the ./finish script also exits 125 (because it is the same process), and the service is then marked as failing permanently.
- The ./finish script is naturally a chain-loading command line if it is written in the execline language. It can also be made into a chain-loading command line from a shell script by using exec s6-permafailon secs deathcount events rest-of-chainloading-cmdline...
- Multiple invocations of s6-permafailon can be chained, in order to test several death patterns.
- If a permanent failure is triggered and secs is high, it is possible that when the administrator manually launches the service again, the next death triggers a permanent failure again. If this is not wanted, the administrator should clear the death tally with the s6-svdt-clear command.
- The current death tally can be viewed via the s6-svdt command.
Example
s6-permafailon 60 5 1,101-103,SIGSEGV,SIGBUS prog... will exit 125 if the service has died 5 times in the last 60 seconds with an exit code of 1, 101, 102 or 103, a SIGSEGV or a SIGBUS. Else it will chainload into the prog... command line.