7 min left
Back to Series

Under the Hood > Article 11 | Intermediate | 7 min read

Article 11Intermediate7 min read

Co-locating workflow state with Postgres gives you exactly-once for free

When workflow checkpoints and application data live in the same Postgres database, a single ACID transaction can eliminate idempotency bookkeeping and simplify the transactional outbox pattern.


Database transaction boundary connecting workflow state and application data

Durable workflow engines promise fault tolerance: if a process crashes mid-run, it resumes from the last completed step instead of starting over. That promise breaks down at a specific seam — the moment between "the step finished its database write" and "the workflow engine recorded the checkpoint."

DBOS argues that the fix is not a better workflow engine sitting beside your database. It is co-locating workflow metadata inside the same Postgres instance as your application data, so both update in one ACID transaction.

This article explains why that co-location matters, how it simplifies idempotency and atomic enqueue patterns, and when the trade-off is worth taking.


The checkpoint gap

A durable workflow records progress step by step. Each step runs, produces a result, and the engine checkpoints that result before moving on. If the worker dies, the next worker picks up from the last checkpoint.

The failure mode is a timing window:

  1. Step executes a database update (credit $100 to an account).
  2. Worker crashes before the workflow engine writes the checkpoint.
  3. Recovery reruns the step.
  4. Account is credited $100 twice.

Workflow engines typically require steps to be idempotent — safe to run multiple times with the same effect — because they cannot close this window when workflow state lives in a separate system from application data.

The standard fix is application-level bookkeeping: an applied_payments table, a unique constraint on (workflow_id, step_name), or a conditional update that checks whether the operation already ran. Each pattern works. Each adds schema, query logic, and reconciliation code that exists only to paper over the separation between workflow state and business data.

Think of two clerks updating different ledgers in different buildings. To prevent double-counting, you add a third ledger tracking which clerk acted first. Co-location puts both clerks at the same desk with one shared notebook.


Exactly-once through one transaction

When workflow checkpoints and application rows live in the same Postgres database, the workflow engine can wrap both in a single transaction:

  1. Begin transaction.
  2. Execute the step's database updates.
  3. Write the step checkpoint.
  4. Commit.

Two outcomes, no gap:

OutcomeDatabase updateCheckpointRecovery behavior
Commit succeedsAppliedRecordedStep never runs again
Anything fails before commitRolled backRolled backStep reruns from scratch

The step's side effect and the proof that it happened are atomic. There is no window where money moved but the workflow forgot.

One transaction boundary replaces an entire class of idempotency tables.

This is not theoretical Postgres enthusiasm. It is the same reason payment systems have always preferred single-database transactions over distributed two-phase commit when they could avoid it. Co-locating workflow state removes a distributed coordination problem by refusing to distribute in the first place.


Simplifying the transactional outbox

The transactional outbox pattern solves another distributed-systems problem: atomically updating a database record and notifying a downstream system.

Classic approach:

  1. Begin transaction.
  2. Update the order row.
  3. Insert a message into an outbox table.
  4. Commit.
  5. A separate poller reads the outbox and delivers messages to the warehouse, email service, or workflow engine.

The outbox guarantees the message exists if and only if the database update committed. But it adds operational machinery: polling loops, delivery retries, poison-message handling, and reconciliation jobs when the poller and the database drift.

When workflow state is co-located, the outbox row is the workflow enqueue row. A Postgres user-defined function can create a workflow record — name, queue, input payload — in the same transaction as the application update. A worker dequeues and executes asynchronously.

The atomicity guarantee is identical. The separate outbox infrastructure is not.

PatternAtomicity guaranteeExtra infrastructure
Separate workflow engine + outbox tableTransactional write to outbox; async deliveryOutbox poller, retry logic, reconciliation
Co-located workflow in PostgresTransactional write to workflow queue row; async executionWorkflow worker (same as before, fewer moving parts)

When co-location is the wrong call

Co-location is a trade-off, not a universal upgrade.

Workflow state competes for database resources. Checkpoint writes on every step add write load to the same Postgres instance serving application queries. At high workflow volume, the workflow tables become a hot path — the subject of a different optimization conversation.

Operational blast radius expands. Workflow engine bugs, migration failures, or runaway queue depth now affect the same database as production data. Teams that separate concerns for isolation reasons may prefer the complexity of dual systems.

Cross-database workflows still need idempotency. If a step touches an external API, a second database, or an object store, the single-transaction guarantee ends at the Postgres boundary. Steps with external side effects still require idempotency keys or compensating transactions.

Not every workflow engine supports it. DBOS is built around this model. Temporal, Cadence, and similar engines store state in their own persistence layer by design. Migrating to co-located state is an architectural decision, not a configuration toggle.


Postgres as workflow runtime

The broader trend behind this argument: Postgres is absorbing workloads that previously required dedicated infrastructure.

Task queues, event sourcing, full-text search, and now workflow orchestration state — each migrated from specialized systems when teams discovered the operational cost of synchronizing two persistence layers exceeded the cost of pushing Postgres harder.

Co-located workflows are the logical endpoint of "just use Postgres" thinking. The database already provides durability, serializable isolation, and crash recovery. Workflow checkpointing is a write-heavy, consistency-critical workload that maps cleanly to those properties.

The question is not whether Postgres can store workflow state. It obviously can. The question is whether the simplification — eliminating idempotency bookkeeping and outbox pollers — justifies sharing the instance with application traffic.

For many mid-scale systems, the answer is yes. For systems where workflow volume dwarfs application queries, separation may still win.


What this means for builders

Four practical guidelines for teams evaluating workflow architecture.

Map your failure windows before choosing an engine. Draw the timeline for each step: where the database commit happens, where the checkpoint write happens, and what crashes in between. If those are separate systems, you need idempotency logic somewhere. If they are one transaction, you may not.

Prefer co-location when workflow steps are mostly database work. Pipelines that read, transform, and write Postgres rows — billing, provisioning, data migration — benefit most from shared transactions. Pipelines heavy on external API calls benefit less.

Measure before splitting databases. The default assumption in 2024–2025 was "workflow engine gets its own store." Co-location challenges that default for Postgres-centric applications. Profile write load before assuming separation is necessary.

Treat exactly-once as a property of the boundary, not the engine. No workflow framework delivers exactly-once execution across arbitrary side effects. Co-located Postgres transactions deliver it for transactional steps — steps whose effects are fully contained in the database. Label which steps are transactional and which require explicit idempotency design.

Conclusion

Separating workflow state from application data is a respectable architectural choice — but it imports distributed-systems problems that Postgres can solve locally. The checkpoint gap between "step completed" and "checkpoint recorded" is the root cause of most workflow idempotency boilerplate.

Co-locating both in one database and one transaction eliminates that gap for database-bound steps. The transactional outbox simplifies to a workflow enqueue in the same commit. The trade-off is shared operational blast radius and write load on a single instance.

For teams already running Postgres as their system of record, the question worth asking is not "which workflow engine should we adopt?" but "does our workflow state need to live anywhere else?"


Postgresdistributed systemsworkflowsACIDidempotencytransactional outboxDBOS

Up next in the series

Article 12Live

Multi-region architecture — going global without going broke

Adding a second region can improve latency and availability, but it introduces consistency problems that can make a system slower and harder to operate. This article walks through the progression from single-region to active-active.

multi-regionsystem designdistributed systemsavailability