Let’s say you have a running Sidekiq system and for whatever reason you need to enqueue jobs from another not ruby-based environment or system. It’s actually quite simple as you can directly push your jobs into Redis without the Sidekiq gem.
First, let’s create a very simple worker and enqueue it:
1 2 3 4 5 6 7 8 9 10 |
|
Sidekiq gives us back a unique ID for that specific job. Let’s have a look inside Redis:
1 2 3 4 5 6 7 8 |
|
Now for simple jobs (as opposed to scheduled jobs), Sidekiq use two different keys: a list and a set. The set called queues
by default only store the queues names. The list named queue:nameofthequeue
actually store the job informations. Let’s have a closer look:
1 2 |
|
A-Ha! So a job is simply a hash serialized in JSON. Those are the required keys:
retry
(boolean): tells Sidekiq whether to retry or not a failed jobqueue
(string): self-explanatory!class
(string): the class of the worker that will be instantiated by Sidekiqargs
(array): the arguments that will passed to the worker’s contructorjid
(string): the unique ID of the jobenqueued_at
(float): the timestamp when the job was enqueued
Pretty simple, huh? So, to enqueue a job yourself you have to:
- Generate a unique ID.
- Serialize the payload using JSON.
- Add the name of the queue to the
queues
set (usingSADD
). - Push the payload to the
queue:myqueue
list (usingLPUSH
).
Pretty simple, eh? Now you might be wondering, what about scheduled job? Well it’s not that much more complicated! First let’s push a scheduled job:
1 2 |
|
Then have a look at what’s inside Redis:
1 2 3 4 5 6 7 8 |
|
At first sight, there’s not much difference besides the fact that the job is stored in a sorted set (zset
) instead of a list. But it’s in a sorted set, that means it must have score:
1 2 |
|
The score is actually the time at which the job is supposed to be executed! This allow Sidekiq to use ZRANGEBYSCORE
to simply pop the jobs that should be executed 1. Now if you want to enqueue a scheduled job, you just have to add it to the schedule
sorted set using ZADD
!
And really, that’s all there is to it! As long as you know the worker’s class name and the arguments it takes, you can enqueue jobs from any programming language. Or even directly inside Redis if you wish so!
For the sake of completion, here’s a naive implementation in Go:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
|