Skip to content
PostgreSQL job queue Open source · Apache-2.0 · Postgres-backed

Durable job queues on PostgreSQL

Spooled stores every job in Postgres, retries failures with exponential backoff, and parks exhausted work in a dead-letter queue you can inspect and replay. Enqueue over REST or gRPC; run workers in any language. Idempotency keys stop duplicates.

See it in code ↓
A job moves through the queue: enqueued, claimed, running, done.
By the numbers
100
jobs / bulk enqueue

One request, up to 100 jobs

up to 100
attempts before DLQ

Exponential backoff, then dead-letter

6-field
cron, second precision

Timezone-aware, with history

REST + gRPC
APIs over HTTP/2

Scales to 100,000 jobs/day on Pro

Quickstart

Up and running in minutes

Three calls: enqueue, claim, complete. The same examples work in cURL, Node.js, Python, Go, and PHP.

Enqueue a job

POST a queue name and JSON payload. Pass an idempotency key and duplicate submissions collapse into one job. Bulk enqueue accepts up to 100 jobs per request.

Jobs & queues docs →
import { SpooledClient } from '@spooled/sdk';

const client = new SpooledClient({
  apiKey: process.env.SPOOLED_API_KEY!,
});

const userId = 'usr_123';

// Create a job
const { id } = await client.jobs.create({
  queueName: 'email-notifications',
  payload: {
    to: 'user@example.com',
    subject: 'Welcome!',
    template: 'welcome',
  },
  idempotencyKey: `welcome-${userId}`,
  maxRetries: 5,
});

console.log(`Created job: ${id}`);

Run a worker

Workers run on your infrastructure. They claim jobs, run your code, and report complete or fail. Failures retry with exponential backoff — up to 100 attempts — then land in the dead-letter queue.

Worker docs →
import { SpooledClient, SpooledWorker } from '@spooled/sdk';

const client = new SpooledClient({
  apiKey: process.env.SPOOLED_API_KEY!,
});

const worker = new SpooledWorker(client, {
  queueName: 'email-notifications',
  concurrency: 10,
});

worker.process(async (ctx) => {
  const { to, subject, body } = ctx.payload;
  await sendEmail({ to, subject, body });
  console.log(`Sent email to ${to}`);
  return { sent: true };
});

await worker.start();

Stream the results

Subscribe over Server-Sent Events or WebSocket instead of polling. The dashboard shows throughput, latency, failures, and DLQ replays from the same streams.

Real-time API docs →
import { SpooledClient } from '@spooled/sdk';

const client = new SpooledClient({
  apiKey: process.env.SPOOLED_API_KEY!,
});

// SSE realtime client (uses /api/v1/events with Authorization header under the hood)
const realtime = await client.realtime({ type: 'sse' });

realtime.on('job.created', (data) => {
  console.log('job.created:', data);
});

await realtime.connect();
await realtime.subscribe({ queueName: 'orders' });

Watch jobs flow through the queue

Spooled stores every job in Postgres. Your workers claim and process them in parallel, failures auto-retry, and the whole pipeline streams live. Here it is running.

Live pipeline

Live
Incoming 2 queued
job_128 pending
stripe.invoice.paid
job_129 pending
user.signup
Processing 2 active
job_126 processing
github.push
job_127 processing
order.created
Completed 2 done
job_124 completed
email.send
job_125 completed
webhook.deliver
0
jobs / sec
0ms
avg latency
100%
success
0
retries

Illustrative motion. Pro enqueues at 100 requests/sec (burst 200) and retains completed and failed jobs for 30 days.

The call that feeds the queue

Works with your stack

Any HTTP source in, workers in any language out.

  • Stripe
  • GitHub
  • Shopify
  • PostgreSQL
  • Redis
  • Node.js
  • Python
  • Go
  • PHP

Watch a job retry, live

Failed jobs retry on their own with exponential backoff. Press play and follow one job through its retries into the dead-letter queue — the same walk your jobs take in production.

job a1f9c… · queue payment-processing

Ready — press play to run the retry flow.

  1. Initial attempt

    t = 0s

    Worker claims the job and runs it

    Failed · HTTP 500
  2. 1

    Retry #1

    t + 1m

    First backoff delay — 1 minute

    Failed · Timeout
  3. 2

    Retry #2

    t + 3m

    Delay doubles — 2 minutes

    Failed · Connection refused
  4. 3

    Retry #3

    t + 7m

    Delay doubles again — 4 minutes

    Failed · HTTP 503
  5. Dead-letter queue

    exhausted

    Payload and error history preserved for inspect & replay

    Parked · inspect & replay

Backoff schedule

1m try 1
2m try 2
4m try 3
8m try 4
16m try 5
32m try 6
60m try 7+
Backoff formula
delay_minutes = min(2 ** retry_count, 60)

Retries are scheduled in minutes (1m, 2m, 4m …), doubling each time and capped at 60m, for up to 100 attempts. Exhausted jobs land in the dead-letter queue with their payload and error history intact — inspect and replay them from the dashboard or the API.

Inspect and replay the dead-letter queue

Exhausted jobs are not lost. List them with their payloads and errors, fix the cause, then replay them — one at a time or in bulk.

Inspect dead-letter jobs
Replay them
Workflows

Chain jobs into dependency graphs

Declare which jobs depend on which, and Spooled runs the graph for you. Roots go first, independent branches run in parallel, and a failed parent cancels everything downstream. Here is a workflow executing.

Workflow run

Live
completed
create-account
queue: users
completed
send-welcome-email
queue: emails
completed
setup-defaults
queue: users
waiting running completed
3 / 3 done

The workflow behind it

Automatic ordering

Declare depends_on and jobs run in topological order. No manual coordination.

Parallel branches

Independent jobs fan out and run at once across your workers, then rejoin downstream.

Failure cascades

If a parent exhausts its retries, its dependents are cancelled instead of running on bad state.

Illustrative motion. Workflows are a paid feature — Pro runs up to 25 concurrent workflows; the Free tier does not include them.

Urgent work jumps the queue

Give a job a priority and workers claim the highest first — high (10) before normal (0) before low (-10). Same enqueue call, one extra field.

Waiting to be claimed

6 jobs
  • VIP order High · +10
  • Pager alert High · +10
  • Send receipt Normal · 0
  • Send invite Normal · 0
  • Nightly cleanup Low · -10
  • Rebuild report Low · -10
worker · claiming

Processed in order

0 done
  1. VIP order #1
  2. Pager alert #2
  3. Send receipt #3
  4. Send invite #4
  5. Nightly cleanup #5
  6. Rebuild report #6

Priority only orders the claim — every job still runs. On the Free plan up to 10 jobs run at once, so higher priority simply gets there first.

The one extra field

Put jobs on a schedule

Daily reports, subscription renewals, cleanup sweeps — describe when with a 6-field cron expression and Spooled enqueues the job for you, second-precise and timezone-aware. Pick a pattern and watch the next runs fill in.

Pick a schedule

Cron expression Morning report
0 sec
0 min
9 hour
* day
* month
* wday

Six fields: second, minute, hour, day-of-month, month, day-of-week.

Next runs

Computed
  1. 1
    Tomorrow, 9:00 AM
    in 6 hours
  2. 2
    Fri, 9:00 AM
    in 1 day
  3. 3
    Sat, 9:00 AM
    in 2 days
  4. 4
    Sun, 9:00 AM
    in 3 days

Preview computed in your browser. Free includes 1 schedule, Pro allows 50 with 30-day job history.

The call that creates the schedule

Watch your workers claim jobs

Run your own workers wherever your code already lives. They claim jobs from Spooled, process them, and mark each complete or failed — in parallel, with no coordination. Row-level locking makes sure no two workers ever grab the same job.

Your worker pool

Claiming from queue emails

0
Claimed
0
Completed
0
Queued
worker-1
idle

on api-eu-1

polling for jobs…
worker-2
idle

on api-us-2

polling for jobs…
worker-3
idle

on worker-box

polling for jobs…

Incoming queue

The loop your worker runs

  • Claim a batch — Postgres row-level locking hands each job to exactly one worker.
  • Do the work in your own code, on your own servers.
  • Complete on success, or fail to retry with backoff, then dead-letter.

Scale out by starting more processes: Pro allows up to 25 workers across 50 queues. Spooled coordinates; you never write a scheduler.

Throughput

One worker, thousands of jobs a second

gRPC bidirectional streaming keeps a worker and Spooled talking over one open HTTP/2 connection, so a single worker can process thousands of jobs per second — no per-job round trip, no polling.

Publish once, stream everywhere

Enqueue a job over REST and Spooled fans it out in real time. Dashboards subscribe over Server-Sent Events, and clients that need two-way traffic connect over WebSocket. One publish, every subscriber updated. Here it is running.

Live pub / sub

Live
Publisher
Your application
POST /api/v1/jobs
{
"queue": "orders",
"payload": { … }
}
Spooled
queue orders
job_512 delivered
order.created
persisted to Postgres 512 published
SSE stream
GET /api/v1/events
job_512 job.completed
job_511 job.created
WebSocket dashboard
GET /api/v1/ws
job_512 queue.stats
job_511 job.created
512
published
2
subscribers
1,024
events fanned out
9ms
fan-out latency

Illustrative counters. Every publish is one enqueue against your 100 requests/sec Pro limit (burst 200); fan-out to subscribers is not rate-limited.

Subscribe over SSE

One-way stream, perfect for dashboards and live views. Reconnects on its own.

Subscribe over WebSocket

Two-way channel when you need to send filters and commands back up the wire.

Real-time

See your queues live

The hosted dashboard subscribes to the very streams you saw fan out above — so what you watch here is exactly what your own clients receive, with no polling.

In the dashboard

Job timeline Follow a job through retries to its outcome.
Dead-letter replay Inspect failures and replay once fixed.
Queue stats Throughput, pending counts, and latency at a glance.
Worker health Active workers, leases, and heartbeats in real time.
Open the dashboard →

Same streams, your clients

  • Server-Sent Events One-way stream for dashboards and live views. Reconnects on its own.
  • WebSocket Two-way channel when you need to send filters and commands back.
See the subscribe code ↑
Use cases

Common workloads, one queue

Webhook processing

Queue events from any source. For Stripe, GitHub, or Shopify, a small adapter verifies signatures and enqueues; idempotency keys absorb provider redeliveries.

In practice: Payment events, deploy-on-push, SMS replies

High-volume jobs

Bulk enqueue 100 jobs per request and check batch status in one call. Tag jobs to trace and filter millions of records later.

In practice: Mass email, data migrations, bulk notifications

Scheduled tasks

Cron with second precision (6-field expressions), timezone-aware execution, manual triggering, and per-schedule history.

In practice: Daily reports, subscription renewals, cleanup

Multi-step workflows

Chain jobs with dependencies, run steps in parallel or sequence, and retry a failed workflow end-to-end with full visibility.

In practice: Order processing, onboarding, approval flows

Callbacks

Outgoing webhooks

Get notified the moment a job finishes. Register a global endpoint that fires for every job, or attach a per-job callback URL to a single enqueue — the same durable queue delivers both.

A broadcast antenna streaming dots outward to several device tiles, illustrating Spooled webhooks delivering job-completion callbacks.

Global endpoint

One URL, called for every job your workers complete.

Per-job callback

A one-off URL attached to a single job at enqueue time.

How it fits

Division of labor

Spooled never runs your code. It stores jobs, schedules retries, and streams status; your workers do the work, wherever they already run.

A mint-green safety net catching falling task cards, with gold retry arrows curving back up and a checkmark shield — Spooled catches what fails and retries it.

Spooled handles

  • Durable job storage in Postgres
  • Retries with exponential backoff
  • Deduplication via idempotency keys
  • Dead-letter queues and replay
  • Schedules, workflows, priorities
  • Live status over SSE / WebSocket

You provide

  • Workers — your code, your servers
  • Email/SMS providers you already use
  • Storage (S3, R2, GCS…)
  • External APIs your jobs call
  • The business logic itself
In practice

See it in real code

Three patterns you can copy today. Each pairs the enqueue side with the worker side — the same durable queue sits between them. Tap a tab, then expand any block to read the full snippet.

Reliable Stripe checkout

Ack the Stripe webhook in milliseconds, then let a durable job fulfil the order — retried automatically if fulfilment fails.

1

Receive the event

Verify the signature, enqueue a job, return 200 fast.

2

Fulfil in a worker

Claim → do the work → complete or fail, on your own servers.

Architecture

Rust core, Postgres durability, open end to end

The API and workers are Rust. PostgreSQL is the source of truth — every job, retry, and dead-letter entry is a row you could query, with row-level security isolating tenants. Redis carries only real-time pub/sub for live dashboards and streams; losing it never loses a job.

The core is Apache-2.0: backend, queue engine, and all four SDKs. Self-host it against your own Postgres with no platform limits, or use the hosted cloud and skip the operating. Same API either way.

100
jobs per bulk enqueue
6-field
cron, second precision
SSE + WS
real-time job streams
REST + gRPC
APIs, HTTP/2 streaming

How work flows through Spooled

Data flow: GitHub, Stripe, and HTTP sources call the Spooled API over REST and gRPC; the API persists jobs to durable Postgres with row-level security and publishes to Redis pub/sub; Prometheus scrapes metrics for observability. GitHub Stripe HTTP Sources Spooled API REST · gRPC Postgres durable · RLS Redis pub / sub Datastores Prometheus metrics Observability
claimed completed
FAQ

Questions engineers ask

Friendly support: a headset, a chat bubble with a checkmark, docs and a question badge
What is Spooled Cloud?
Spooled is an open-source job queue backed by PostgreSQL. You enqueue jobs over REST or gRPC; your workers claim and run them; Spooled stores every job durably, retries failures with exponential backoff, and moves exhausted jobs to a dead-letter queue. The core is Apache-2.0 and self-hostable, with a hosted cloud option.
Is Spooled a job queue without Redis?
Yes. Jobs are stored in PostgreSQL, so durability does not depend on Redis. Redis is used only for real-time pub/sub (live dashboards and WebSocket/SSE streams), not as the source of truth for job state.
How do retries and the dead-letter queue work?
Failed jobs retry automatically with exponential backoff. You configure max attempts (up to 100) and delays. Jobs that exhaust retries land in the dead-letter queue, where you can inspect payloads, bulk retry, or purge with filters.
How does Spooled prevent duplicate jobs?
Every enqueue accepts an idempotency key. If the same key is submitted twice — for example, a webhook provider retrying delivery — Spooled deduplicates it, so the job runs once.
Does Spooled support cron and scheduled jobs?
Yes. Schedules use 6-field cron expressions with second precision and timezone-aware execution, plus manual triggering and execution history. One-off jobs can also be delayed to a future time.
Can I chain jobs into workflows?
Yes. Workflows chain jobs with dependencies and run steps in parallel or in sequence. A failed workflow can be retried end-to-end with full visibility into each step.
Which languages have official SDKs?
Node.js, Python, Go, and PHP — see the SDK docs. All SDKs cover the REST API; the gRPC API with bidirectional streaming is available for high-throughput workers. Any language that can make HTTP requests works without an SDK.
Where does my worker code run?
On your infrastructure. Workers are plain processes that claim jobs from Spooled, run your code, and report success or failure. Spooled never executes your code — it stores jobs, schedules retries, and streams status.
Is Spooled open source? Can I self-host it?
The core — the Rust backend, queue engine, and all SDKs — is Apache-2.0 on GitHub. You can self-host it with your own PostgreSQL with no platform limits, or use the hosted cloud with a free tier of 1,000 jobs per day.
How is Spooled different from BullMQ, Celery, or Sidekiq?
Those are language-specific libraries that require you to run and operate Redis. Spooled is a language-agnostic service with REST and gRPC APIs, PostgreSQL durability, a built-in dashboard, and webhooks — usable from Node.js, Python, Go, PHP, or anything that speaks HTTP. See the full comparison.
How do I watch jobs in real time?
Subscribe to job updates over WebSocket or Server-Sent Events instead of polling — see the real-time API. The dashboard uses the same streams, and Prometheus metrics are exported for your own monitoring.
Get started

Queue your first job today

The Free plan includes 1,000 jobs a day with retries, dead-letter queues, and real-time streams. No credit card.