These last days I was working on a set of Perl scripts to send distributed tasks to different servers using Gearman job server and his API. Basically to do this I used a total of a 3 servers:
– 1 server acting as a client and running the gearman-job-server daemon and running the client script to send the task name to the queue of the job server.
– 2 servers acting as workers (this servers will run the jobs) that basically it connects to the gearman job server and get the tasks name from the queue, and run the script associated with the task name.
Basically I run the script cron_client.pl script on the client server to send tasks on the job server. I run this script with the cron daemon at the time that I would like to send the tasks to the queue and immediately the workers will run it. An example usage of this script may be:
./cron_client.pl –tasks task1 task2
It takes as parameter the task name identifier that the workers will take and run the script associated with it.
For the workers part I run the script cron_worker.pl on the workers servers (it runs in an infinite bucle so we can setup it as a daemon process in background) and at the time that one task is put from the client it will takes this task name and look for the path to the script associated with the task name, defined on a json file and it will runs the script.
So with this way we can distribute some work load from some tasks between the worker servers. To see with more detail how to install the environment and how to setup it check the Github repository: https://github.com/opentodonet/distributed-cron
More information about Gearman: http://gearman.org/documentation/