Contents
In brief
Slow work — email, webhooks, image resize, reports — should not block HTTP requests. Swig on Dev.to is a Go job queue stored in PostgreSQL, without a separate message broker.
What happened
Architecture sketch:
- jobs table in Postgres: JSONB payload, status, attempts, timestamps.
- Go workers claim jobs inside transactions using row locks so two processes never take the same row.
- Worker registry and a leader loop for maintenance: stale leases, retries, housekeeping.
- Go interfaces for handlers; goroutines for concurrency; context for cancel and timeouts.
- LISTEN/NOTIFY wakes workers on insert instead of constant polling.
If Postgres is already in production, you can embed the queue in the same database using transactions and locks rather than adding Redis “just for jobs.”
Why it matters
Teams often default to Redis/RabbitMQ when throughput is modest and operational simplicity matters more than extreme QPS. A Postgres-backed queue shares backups, monitoring, and migrations with the rest of the app.
At very high QPS or sub-second delivery SLAs, a dedicated broker may win — for typical web products, table + workers lasts years.
In practice
- Job schema:
id,kind,payload jsonb,status,attempts,run_at,locked_at,locked_by. - Claim in a transaction with
FOR UPDATE SKIP LOCKED(or equivalent). - Idempotent handlers — retries must not double-charge or double-send email.
- NOTIFY after commit on insert; periodic polling as a safety net.
- A leader process requeues stuck jobs and archives old rows.
- Metrics: queue depth, oldest pending age, failure rate.
Takeaway
Swig is a solid teaching skeleton for PostgreSQL job queues in Go. Read the original for schema and code before you buy a second data store.