Serialize per resource
Run at most one job per resource at a time — never two balance syncs for the same account concurrently — while different accounts still run in parallel.
The scenario#
You sync a bank account's balance from an external provider. Running two syncs for the same account at the same time corrupts the balance — they race on the same rows. But two syncs for different accounts are completely independent and should run in parallel for throughput.
You don't care if a duplicate sync for account 42 gets enqueued while another is running — you only care that they never execute at the same time. That's exactly what :while_executing gives you: a lock taken on the server, keyed on the arguments, held only for the duration of perform.
The worker#
:while_executing locks just before perform runs and releases just after. Because the lock is keyed on the arguments, account_id becomes the thing that's serialized:
class SyncAccountBalanceJob
include Sidekiq::Job
sidekiq_options lock: :while_executing,
on_conflict: :reschedule
def perform(account_id)
account = Account.find(account_id)
account.sync_balance_from_provider!
end
endlock: :while_executing— the lock is taken on the server, right beforeperform, and released right after. It has no effect at enqueue time.on_conflict: :reschedule— v9 locks are non-blocking: when a copy for the same account is already running, this copy's single acquisition attempt fails and the conflict strategy fires immediately.:reschedulere-enqueues this copy to run later (~5s by default) instead of dropping it. Because:while_executingconflicts happen during execution, this is a server conflict strategy.
What happens on a duplicate#
Two syncs for the same account, arriving close together.
Say two SyncAccountBalanceJob copies for account 42 land on your workers at nearly the same time:
- Copy A reaches
perform.:while_executingacquires the lock keyed on[42]and starts syncing. - Copy B reaches
performa moment later and makes its single attempt to acquire the same lock. It's held by A, so B's attempt fails right away — v9 locks are non-blocking, so B does not wait. on_conflict: :reschedulefires immediately: B is re-enqueued to run later (~5s by default). It is not resumed in place when A finishes — it comes back around as a fresh job and tries again.- When B's rescheduled run lands and account
42is free, it acquires the lock and syncs — the two runs happen one after another, never concurrently.
Meanwhile a SyncAccountBalanceJob for account 43 has a different digest, so it never contends with either — it runs in parallel.
Why it serializes per resource#
Uniqueness is computed from the worker class, the queue, and the arguments. Two calls with the same account_id produce the same digest and therefore compete for one lock; two calls with different account_ids produce different digests and never see each other. The account_id argument is the serialization key — no extra configuration needed.
If your perform takes arguments that shouldn't affect uniqueness (a timestamp, a request id), narrow the digest to just the resource id with lock_args_method:
class SyncAccountBalanceJob
include Sidekiq::Job
sidekiq_options lock: :while_executing,
on_conflict: :reschedule,
lock_args_method: :unique_args
def self.unique_args(args)
[args.first] # serialize on account_id only, ignore the rest
end
def perform(account_id, requested_at)
account = Account.find(account_id)
account.sync_balance_from_provider!(as_of: requested_at)
end
endThe pitfall: it does not stop duplicate enqueues#
:while_executing locks on the server, only around execution. It does not run at enqueue time, so it will not stop a duplicate copy from being pushed onto the queue. Fire perform_async(42) ten times and all ten land in the queue — :while_executing only guarantees they don't run at the same time; they run one after another.
Where to go next#
- Enqueue once, run serialized — when you need both a single queued copy and serialized execution, use
:until_and_while_executing. - Bounded concurrency — allow up to N concurrent runs of the same digest instead of exactly one, with
lock_limit.