Daemontools: To Ensure That Your Processes Are Always Running

In production environment, it is often required to restart the process if it dies. Not doing so fast enough may result in user facing downtime which can impact the business, hence automation of this job is critical to reduce response time.

There are several tools which do similar jobs like Daemontools, Supervisor, bash scripts, Monit etc. We are going to talk about Daemontools since it is very light, implements basic feature set and allows you to do more over its standard commands. It is written by D. J. Bernstein, the creator of djbdns and several awesome tools.

Daemontools runs the processes as its own child process and spawns a new one as soon as the older dies. You have to ensure that the process you are trying to run using daemontools runs in foreground and is not a daemon in itself. The svscan program of Daemontools will scan the present working directory and its subdirectories and launch one supervise process for each run script found.

Installation:
For Fedora, Red Hat and other Enterprise Linux you can use untroubled.org's daemontools rpm. Daemontools itself was written in 11 years ago (and it still works like a charm!) so don't worry about the stale rpms.
For Ubuntu server and similar distributions, you can install deamontools and daemontools-run packages and you would be good to go.

In case you want to do a manual install, instructions can be obtained from the Daemontools website.

Usage:

Once the daemontools is installed, all you need to do is create an executable script named run and put the command to launch the process in the script. Start the svscan tools after this and let the magic happen. 

Let us see this though an example. Say you have 2 services, nginx and php-fpm which you want to restart immediately if they die then here is how your directory structure should look like:

Deamontools install dir (usually /etc/service)

├── nginx

│   └── run

└── php-fpm

    └── run

run file should have a shebang like #!/bin/bash or #!/usr/bin/python defined and it should have executable bit set. Once you are done with this, start svscan either manually by running svscan command or if you have a SysV init script then use that. 

I also suggest you to go through "man svc" to get an idea of commonly used commands of Daemontools.