Skip to content

Supervisor

This section describes how to use Supervisor (http://supervisord.org/) to manage the corridor processes for each component of the platform. Using supervisor makes it easy to monitor the status of the application and it manages startup initialization, starting, stopping, and even log routing for the applications.

To use Supervisor, it needs to be installed in the system and the daemon supervisord should be running to manage the application processes.

Installation

The official installation instructions can be found at http://supervisord.org/installing.html

The recommended method of installation is to use the package manager provided by the Operating System (apt-get, yum, etc.). This will set up system configs like systemd/initd configs to ensure Supervisor runs correctly.

RedHat:

yum install supervisor

Ubuntu:

apt install supervisor

Using pip:

If using any other system, Supervisor can also be installed with pip as:

pip install supervisor

Installing with pip is possible on all systems, but requires manual configuration as mentioned in http://supervisord.org/installing.html#creating-a-configuration-file. And also requires additional setup to initialize it on startup as described in http://supervisord.org/running.html#running-supervisord-automatically-on-startup.

Configurations

Once installed, the supervisor configuration needs to be created per process that needs to be managed as described in http://supervisord.org/running.html#adding-a-program

Some useful configurations in supervisor are:

  • command: The command to execute. Note, if 2 commands need to be executed, use bash -c "command1; command2"
  • stdout_logfile: Log file location for the stdout logs (%(program_name)s and %(process_num)01d can be used as variables)
  • stderr_logfile or redirect_stderr: Log file location for the stderr logs, or redirect all the stderr logs to the stdout stream and hence have a common file for both
  • user: The user to run the process as
  • environment: The environment variables to be set before the process is run
  • numprocs: The number of processes to run

Here are some example configuration files for the Corridor components:

Web Application server:

[program:corridor-app]
command=INSTALL_DIR/venv-app/bin/corridor-app run
environment=
  CORRIDOR_INSTANCE_PATH=INSTALL_DIR/instances/INSTANCE_NAME
stdout_logfile=/var/log/corridor/%(program_name)s.log
redirect_stderr=true
user=root

API server:

[program:corridor-api]
command=
  bash -c "INSTALL_DIR/venv-api/bin/corridor-api db upgrade && INSTALL_DIR/venv-api/bin/corridor-api run"
environment=
  CORRIDOR_INSTANCE_PATH=INSTALL_DIR/instances/INSTANCE_NAME
stdout_logfile=/var/log/corridor/%(program_name)s.log
redirect_stderr=true
user=root

API - Celery worker:

[program:corridor-worker-api]
command=INSTALL_DIR/venv-api/bin/corridor-worker run --queue apilocal
environment=
  CORRIDOR_INSTANCE_PATH=INSTALL_DIR/instances/INSTANCE_NAME
stdout_logfile=/var/log/corridor/%(program_name)s.log
redirect_stderr=true
user=root

Spark - Celery worker:

[program:corridor-worker-spark]
command=INSTALL_DIR/venv-api/bin/corridor-worker run --queue celery
environment=
  CORRIDOR_INSTANCE_PATH=INSTALL_DIR/instances/INSTANCE_NAME
stdout_logfile=/var/log/corridor/%(program_name)s.log
redirect_stderr=true
user=root

Jupyter Notebook:

[program:corridor-jupyter]
command=INSTALL_DIR/venv-jupyter/bin/corridor-jupyter run
environment=
  CORRIDOR_INSTANCE_PATH=INSTALL_DIR/instances/INSTANCE_NAME
stdout_logfile=/var/log/corridor/%(program_name)s.log
redirect_stderr=true
user=root