SYSTEM DAEMON: AUTOMATED PROCESSES
- Pedro Jose Sanchez Luling
The System Daemon is a separate program that runs in background. Its goal is to execute two types of automated processes
Background Jobs requested by Linkcare API that must be executed asynchronously
System maintenance tasks that are repeated periodically.
Customizable external services that must be executed periodically
To start the System Daemon it is necessary to execute the following PHP file.
Executable file | start_system_daemon.php |
|---|
When started, the System Daemon runs an infinite loop that checks periodically whether it is necessary to launch a Background Job or a System Maintenance Task.
Logs generated during the execution of the System Daemon are redirected to STDOUT
Normally the System Background Tasks are installed as a separate service (Worker) as explained in SYSTEM OVERVIEW
Background Jobs
Some API functions generate a request to execute a background job. This generally happens to improve performance, because it is not necessary for the API function to wait until the background job is finished to return a response.
A typical example is sending a notification to a user when something happens. For example after invoking event_insert () to create an EVENT OBJECT, generally a notification (email, SMS, etc.) is sent to the target user, but this can be done in background because it doesn't affect the outcome of the API function and it make take some time to send notifications.
Background Job are triggered using Redis Cache Server as mean of communication. This basically works as follows:
When the API needs to execute a background task. it uses the Resque library to place the Request in the Redis Server Cache.
A background process checks periodically the Redis Server Cache to verify if there exist new requests.
Until API Version 2.7.21 the implementation of the background jobs was done using an external component (see † BACKGROUND JOBS WITH PHP-RESQUE) that has been substituted by a proprietary implementation.
System Maintenance Tasks
System processes are launched asynchronously so that different processes can run concurrently. This is accomplished by running each process in a separate thread. Nevertheless, it is not allowed to launch the same process more than once if the previous instance has not ended.
For example, if a process is scheduled to launch each 3 minutes but the previous execution has not ended yet, then a new process will not be started.
Process | Description |
|---|---|
OUTDATED_TASKS_CHECK | Looks for open TASKS that are outdated and executes the appropriate actions on them. A TASK can be outdated due to one of the following reasons
Criteria to find outdated TASKS:
Renamed in API Version 2.7.21. In previous versions the name of the Process was RECURSIVE_TASKS_CHECK |
AVAILABLE_TASKS_CHECK | Looks for TASKs that can be executed by a CASE and sends a notification to the user (if there was no previous notification) The goal is to discover TASKs whose scheduled datetime has arrived, and it warns the patient about the availability of that TASK. Criteria to find TASKs that must be notified:
|
PURGE_DATABASE | Cleans objects marked as 'deleted' at least one month ago in DB (ADMISSIONs, TASKs, etc..). Objects marked as deleted in the last month are preserved. Currently this period is not configurable. |
SERVICE_TASKS | Looks for open TASKs assigned to role "SERVICE". If the scheduled datetime of the TASK has arrived, then the TASK is closed. |
EXPIRED_ADMISSIONS | Looks for active ADMISSIONS in stages "ENROLL", "ADMISSION" or "PAUSE" that must be discharged automatically because the PROGRAM or SUBSCRIPTION have a maximum duration configured. The date to do the automatic discharge is calculated from the ENROLL DATE plus the configured SUBSCRIPTION OBJECT or PROGRAM OBJECT duration |
DELAYED_NOTIFICATIONS | Looks for notifications that were scheduled to be sent at a specific datetime instead of sending them immediately. Introduced in API Version 2.7.25 |
EXPIRED_SESSIONS | Removes active sessions that have not been active for a pre-configured amount of time (see MAX_INACTIVE_SESSION_TIME in WEB SERVICES CONFIGURATION ) Introduced in API Version 2.7.25 |
ADMISSIONS_UPDATE | This background process is executed only by a call to the API function subscription_admissions_update () Updates the version of outdated ADMISSIONS of a SUBSCRIPTION. Introduced in API Version 2.7.31 |
Periodicity
There are 2 ways to define the periodicity of a System Task:
Frequency: The process must be repeated periodically after a predefined time lapse
Fixed time: The process must be executed once each day when the scheduled time arrives
The following table shows the default configuration of the System Tasks
Process | Frequency | Fixed time | Comments |
|---|---|---|---|
OUTDATED_TASKS_CHECK | 300s | - | Renamed in API Version 2.7.21. In previous versions the name of the Process was RECURSIVE_TASKS_CHECK |
AVAILABLE_TASKS_CHECK | 300s | - |
|
PURGE_DATABASE | - | 00:00 |
|
SERVICE_TASKS | 60s | - |
|
EXPIRED_ADMISSIONS | - | 01:00 | Introduced in API Version 2.7.22. |
DELAYED_NOTIFICATIONS | 300s |
| Introduced in API Version 2.7.25 |
Customize configuration of System Maintenance Tasks
It is possible to change the default configuration of the System Tasks in a PHP file called daemon_configuration.php into the directory conf located at document root level.
Example:
<?php
$scheduledTasksFrequency[SystemMaintenance::DAEMON_OUTDATED_TASKS] = ['frequency' => 300, 'time' => null];
$scheduledTasksFrequency[SystemMaintenance::DAEMON_AVAILABLE_TASKS] = ['frequency' => 300, 'time' => null];
$scheduledTasksFrequency[SystemMaintenance::DAEMON_PURGE_DB] = ['frequency' => null, 'time' => '00:00'];
$scheduledTasksFrequency[SystemMaintenance::DAEMON_SERVICE_TASKS] = ['frequency' => 60, 'time' => null];
$scheduledTasksFrequency[SystemMaintenance::DAEMON_EXPIRED_ADMISSIONS] = ['frequency' => null, 'time' => '00:01'];
$scheduledTasksFrequency[SystemMaintenance::DELAYED_NOTIFICATIONS] = ['frequency' => 300, 'time' => null];Customizable external services
Introduced in API Version 2.7.26
Sometimes it may be desirable to use the System Daemon to execute processes that perform specific jobs that are out of the scope of the maintenance of the system.
For example, a developer can implement a service to fetch information about patients from an external source, and import them into the Linkcare platform. This process may be executed on a daily basis, but we need a mechanism to execute it periodically and handle the messages or errors returned by the service.
The System Daemon offers a way of integrating the execution of external services. The requirements for integrating an external service are:
It must be callable via a HTTP POST request
The service must return a JSON response (Content-type: application/json) with an specific format explained below
Expected response format
The JSON response received from the external service must contain the following properties:
code: a string indicating the result of the execution of the service. It must be one of the following values:
success: indicates that the service was executed with no errors
idle: indicates that the service was invoked but there was no job to do
error: some error happened during the execution of the service
message: A string that will be displayed in the logs generated by the System Daemon when the service is executed (independently of the reponse code)
error_details: an optional array of strings with detailed information of any possible error occurred during the execution of the service. This messages will be displayed in the logs generated by the System Daemon
Example
{
"code": "success",
"message": "Patient import result: Success: 0, Failed: 0",
"error_details": []
}
External service configuration
You can configure as many external services as necessary.
For each service configured a separate process is executed. This process is the one that executes periodically the service based on the schedule configuration.
The configuration of the External Services must be done defining a global variable called $GLOBALS['EXTERNAL_BACKGROUND_SERVICES'] in a PHP file called external_services_daemon.php into the directory conf located at document root level.
The configuration variable must be an array of JSON objects, where each object is the configuration of a single external service.
Example
<?php
$GLOBALS['EXTERNAL_BACKGROUND_SERVICES'] = '[{
"name": "import_patients",
"description": "Import patients from HIS"
"endpoint": "https://service_endpoint",
"parameters": {},
"timeout": 300,
"allow_concurrency": false,
"execution_plan": {
"period": "daily",
"timezone": "Europe/Madrid",
"time": "15:57",
"intervals": {
"on_timeout": 300,
"on_idle": 300,
"on_success": 10,
"on_error": 10
}
}
}
]';Configuration parameters
For each service you can configure the following parameters:
name: Name assigned to the service. This name must be unique along all configured services. It will be used to generate logs
description: Brief description of the service
endpoint: Complete URL that will be invoked by the System Daemon to execute the service
parameters: Additional parameters that should be sent to the service as part of the body of the HTTP POST request. Parameters are sent in JSON format exactly as defined in the configuration
timeout: number of seconds to consider that the service is not responding
allow_concurrency: (true/false) Used for concurrency control. If true, a lock mechanism will be set while it is executing. During this period, no other System Daemon will try to execute the service (even though there are many System Daemon instances). Note that the lock is only effective while the timeout period is not exhausted. When the service returns a response or a timeout happens, the locking mechanism will be removed.
execution_plan: contains the details to define the execution schedule:
period: (daily/interval) indicates whether the service must be executed once per day or periodically after an interval
timezone: ISO timezone expression (e.g. Europe/Madrid). This value is used only if the execution period is “daily” and defines under which timezone has been defined the execution time.
time: hour expressed in format hh:mm:ss. Is the hour of the day (in the timezone previously defined) when the service will be executed. This parameter only has effect if the execution period is “daily”
intervals: time lapse (in seconds) to wait after the execution of the service until the next execution. This parameter only has effect if the execution period is “interval”. Note that it is possible to define different intervals depending on the last response of the service:
on_timeout: time to wait if the last execution ends by a timeout.
on_idle: time to wait in the result code returned by the last execution was “idle”
on_success: time to wait in the result code returned by the last execution was “success”
on_error: time to wait in the result code returned by the last execution was “error”