ivo's home

Most services eventually need to do two things at once: save data and tell another system about it. A patient registers → store the visit and notify an external health API. A payment clears → write the record and push an event. The naive way is a “dual write”: write to the database, then publish to a message queue. That seam is where data quietly goes missing.

The outbox pattern removes the seam — and for a lot of systems it means you don’t need RabbitMQ, Kafka, or Redis at all.

The dual-write problem

Picture the unreliable version:

saveVisit(visit)        // committed to Postgres
publishToQueue(event)   // ...network blips here

If the process crashes between those two lines, or the queue is briefly unreachable, you now have a visit with no event. Or you reorder them and get an event for a visit that was never saved. There is no transaction spanning “my database” and “someone else’s broker”, so something can always slip through the gap.

What the outbox actually is

Instead of publishing to an external broker, you write the intent-to-send as a row in a table in the same database, inside the same transaction as your business data. Then a separate worker polls that table and does the actual sending.

CREATE TABLE outbox_event (
  id          text PRIMARY KEY DEFAULT gen_random_uuid()::text,
  type        text NOT NULL,          -- 'encounter', 'payment', ...
  payload     jsonb NOT NULL,
  status      text NOT NULL DEFAULT 'pending',   -- pending | sent
  attempts    int  NOT NULL DEFAULT 0,
  created_at  timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX idx_outbox_pending ON outbox_event (created_at) WHERE status = 'pending';

The write path — one transaction, nothing to desynchronize:

BEGIN;
  INSERT INTO visit (...) VALUES (...);
  INSERT INTO outbox_event (type, payload) VALUES ('encounter', $1);
COMMIT;

And a worker that drains it, on a timer:

for (const e of await listPending()) {
  try {
    await sendToExternalApi(e.payload);
    await markSent(e.id);
  } catch (err) {
    await markFailed(e.id, err);   // stays pending, retried next tick
  }
}

That’s the whole thing. A to-do list living in the same ledger as your real data, plus a janitor that works through it.

Why this replaces a message queue and Redis

For this job, a broker really only provides one essential service: hold messages that haven’t been processed yet, without losing them. A database table does that too.

1. Durability. Rows are on disk in Postgres. Restart the container, redeploy, crash — the pending rows are still there, same as Redis persistence or a durable MQ.
2. Consistency — this is where outbox actually wins. The business row and the “send this” row commit together, atomically. There is no window where one exists without the other. With a separate broker you are back to dual-write: the DB commit can succeed while the publish fails (or vice-versa), and now your systems disagree.
3. One less moving part. No extra server to install, secure, monitor, and back up. The database you already run stores both your data and your queue.

When a real broker still wins

Takeaway

If your volume is modest and what you truly care about is “never lose or duplicate the event,” the outbox pattern gives you durability and transactional consistency with zero extra infrastructure. Reach for Kafka/RabbitMQ/Redis when you outgrow it — high throughput, real-time latency, or sophisticated retry semantics — not before.

The mental model: an outbox is a to-do list in the same notebook as your transactions. A message queue is a separate post office — more powerful, but now you’re keeping two books in sync, and that’s exactly where the bugs live.