Dev Blog
Normally, when you use computer, there are numerous background processes which allows you to use them as a services. That is, these processes are always ready to be used. Most obvious examples for web applications are HTTP Server, Database servers - MySQL, PostgreSQL, MongoDB, ElasticSearch or many, many other. When developing web application it is sometimes desirable to have strictly application-related service which cooperates with web application in some way. The use case I plan to describe here is a service that will react on RabbitMQ messages sent by front-end. The messages are generated by using web application and the reason for using message queue is that often these tasks require additional permissions or are time consuming.
The messages are the kind of triggers for batch processing, which might include:
- Generating configuration files
- Dumping database backups
- Getting SSL certificates
Using CRON
The first idea I came up with was to call PHP script via CRON job. Adding CRON job is very easy and reliable. To minimize delays, the cron job was set to run every minute. While this might be acceptable for long running tasks, for those which should have rather instant effect it is rather long. For instance, when user saves the website, configuration file needed to be generated. But then, our unlucky visitor might need to wait minute until changes will be applied. Actually it is possible to run CRON job more often than once a minute. I've made with sleep command with increasing delays.
Example of CRON job called every 10 seconds
* * * * * /var/www/my-app/cron.sh * * * * * sleep 10 && /var/www/my-app/cron.sh * * * * * sleep 20 && /var/www/my-app/cron.sh * * * * * sleep 30 && /var/www/my-app/cron.sh * * * * * sleep 40 && /var/www/my-app/cron.sh * * * * * sleep 50 && /var/www/my-app/cron.sh
Rather ugly hack but might work in your case. There are major flaws in this setup:
- The script is executed every 10 seconds no matter if it really should ran or not
- The real problem here is the aforementioned word task. When application requires more than one in-background operation - if not it will require in future, trust me ;) - the CRON setup will be either convoluted or the cron.sh script will have to decide which taks to run
- If some long running task will exceed 10 seconds time, another job will start - some locking or queue mechanism is required
RabbitMQ to the rescue
The message queue, in this case RabbitMQ Message Queue Server,
will intercept incoming messages and publish them for use by
another proces, server or whatever it is connected to it. Using
message queue will allow easy adding of new tasks without
worrying of anything at all.
The RabbitMQ setup consists of three parts:
- Web application sending messages
- RabbitMQ server listening and dispatching messages
- PHP process which listens and responds to messages
Running PHP Script as Service
Running PHP as a service does not sound too good approach because PHP is mainly intended for building web applications. However major advantage is that service can use existing application facilities and code. The process itself is lightweight and most of the time is simply waiting for incoming messages. There are some extra things to keep in mind, especially cleaning up after message is processed. Cleaning up means closing connections to database, close opened files, free memory etc. Operations which should be normally done, but often are ignored because of momentary nature of PHP. When designing PHP tasks which are supposed to be ran on continuously executed script just need more attention to not left garbage behind or memory leaks will occur.
Setting up Service
To create system service we need to create configuration file
instructing system of what script should be ran. In Ubuntu for
systemd the configuration file is located
in /etc/systemd/system/
and should end with
.service
extension. In the example below we will
name it hosting.service
. The file contains minimal
configuration to be used as service:
[Unit] Description=Hosting Updating Service [Service] User=root Type=simple TimeoutSec=0 PIDFile=/var/run/hosting.pid ExecStart=/bin/bash /var/www/hosting/service KillMode=process Restart=on-failure RestartSec=42s [Install] WantedBy=default.target
The configuration is pretty self explanatory, the most important options are:
-
PIDFile
- which will create file indicating that the service is up. Just make it unique. -
ExecStart
- command to run. I recommend using bash script to execute PHP script - later on it.
The ExecStart
directive must start with
/
The script can be symlinked or copied over
to /etc/systemd/system/
directory. After
modifying script the sudo systemctl
daemon-reload
command need to be executed to instruct
systemd that config was updated.
Startup Script
As mentioned earlier, the PHP is not executed directly, but it is
called from inside bash script. This approach is used to setup
proper working directory for PHP script and to allow changes
without modifying configuration file. The script
/var/www/hosting/service
is just a PHP launcher:
#!/bin/bash # This file should be ran as a service by hosting.service DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" cd $DIR ./maslosoft hosting:service
The first lines are setting up proper working directory, then the PHP script is executed.
Service Install Script
To avoid repeatable manual tasks and make it more robust, service installing script can be created. With the install script, whole code can be shipped along with application. The install script, let's name it install-service.sh contains instructions to setup our script as a daemon:
#!/usr/bin/env bash DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" file="$DIR/hosting.service" sudo ln -s "$file" "/etc/systemd/system/hosting.service" sudo systemctl enable hosting sudo systemctl start hosting systemctl status hosting
Similarly to service
script, the
install-service.sh
is setting up proper working
directory, then creates symlink to configuration file,
enables and starts service and finally displays status. The
service installing script can be ran from any directory, as it
detects its own location.
The installing script must be ran with sudo
, ie
sudo install-service.sh
From now on, if everything went fine,
calling systemctl status hosting
should display
similar output:
● hosting.service - Hosting Updating Service Loaded: loaded (/etc/systemd/system/hosting.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2019-07-02 13:46:32 CEST; 1 weeks 0 days ago Main PID: 13607 (bash) Tasks: 2 (limit: 4915) CGroup: /system.slice/hosting.service ├─13607 /bin/bash /var/www/hosting/service └─13623 php ./maslosoft hosting:service lip 02 13:46:32 server-1 systemd[1]: Started Hosting Updating Service.