How do I handle poison-pill messages and a Dead Letter Queue in the queue?

Short answer: a job that throws every time and is auto-retried forever locks the queue and crashes the worker — that’s a poison pill. The fix is to bound the retries and route the bad job to a DLQ instead of re-queueing it.

Short answer

What you’re hitting is classic: the job blows up at the same spot, Laravel says “let me try again,” it blows up again, and the queue can’t move. You have to break the loop. Retrying is only safe if the job is idempotent; I wrote up the rules for safely repeating the same request in the idempotency post.

Why

1. failed_jobs is already your DLQ. When the cap is reached, Laravel moves the job to the failed_jobs table automatically — no extra infrastructure to build, that table is your Dead Letter Queue.
2. A Fatal Error doesn’t go through the normal retry path. Laravel’s normal retry doesn’t cover a Fatal Error that kills the worker outright (not an exception). Supervisor restarts the worker, but if the same job gets pulled again the loop never closes.
3. A silently failing job is the worst case. If a job lands in failed_jobs and nobody is told, the queue moves on but the outcome that job was supposed to produce never happens.
4. One bad job starves the whole queue. If risky job types share a queue with everything else, a single poison message blocks every other job behind it.

What to do

1. Bound the retries. Put tries (or maxExceptions) and a backoff on the job. Now it’s attempted 3-5 times, not infinitely. Once it hits the cap, Laravel stops re-queueing it.
2. Define a failed() method on the job. Put your cleanup/compensation logic there so half-finished side effects get unwound when the job lands in the DLQ.
3. Wire up alerting. Write a JobFailed listener that fires a Slack/Sentry notification the moment a job lands in failed_jobs. Admins should know instantly. After inspection you replay the fixed job with queue:retry.
4. Keep a counter for Fatal Errors. Keep a “seen this job id N times” counter (in Redis) and force the job into failed_jobs once it crosses the threshold; Supervisor bringing the worker back up doesn’t close the loop by itself.
5. Separate queues by risk, make jobs idempotent. Run risky job types on their own queue so one bad job can’t starve all the others. And write jobs to be idempotent so retries are safe — running twice shouldn’t pile up side effects.

Bottom line: capped retries + the failed_jobs DLQ + alerting + replay via queue:retry. Never let any job retry unbounded; unbounded retry turns a single poison message into a weapon that halts the whole system.