← All posts

SQLite Is All You Need for Durable Workflows

DBOS and Obelisk: local SQLite plus Litestream to S3 for workflow state; Postgres when you need HA and shared scale.

SQLite Is All You Need for Durable Workflows
Contents

In brief

DBOS showed that durable execution does not require a separate orchestration tier — a database you already trust is enough. Obelisk pushes further: for a large class of systems, SQLite + Litestream is enough to keep workflow state for AI agents. Postgres remains the choice when you need HA and shared scalability.

What happened

Durable execution is often discussed as if it requires heavy infrastructure. In practice, what must survive is workflow state: an execution log, step history, replay, and activity retries. Compute can stay cheap and disposable.

Obelisk centers on an execution log: workflow progress survives restarts, activities can be retried, and state is easy to inspect. That does not require a network database.

SQLite gives transactional durable state without a separate database service: no network hop, no extra control plane. A local file is often the right level of machinery for experimental and agent systems.

Litestream handles portability: it asynchronously streams SQLite changes to S3-compatible object storage. The working database stays close to the runtime; copies go to backup, migration, and debugging. The caveat: replication is asynchronous — if the volume disappears before flush, the newest writes may be lost. That is fine for many AI experiments, not for a highly available shared database.

A typical setup: an Obelisk server with SQLite, Litestream to object storage, and an observer that pulls interesting databases for replay and understanding what an agent actually did.

Why it matters

AI agents and AI-generated workflows are often bursty, experimental, and easier to reason about when each tenant or agent has a small, self-contained unit of state. A fleet of micro VMs or containers, each with local SQLite and S3 backup, is often simpler, cheaper, and better isolated than one large always-on Postgres.

Takeaways for architects:

  • Do not start on Postgres if state does not yet require shared HA.
  • Durable ≠ distributed: workflow state in a SQLite file is already durable within one node.
  • Litestream turns SQLite from a local toy into a portable artifact for audit and debugging.

Postgres in Obelisk is the right choice when you need higher availability, broader shared scalability, or a durability model that async replication to object storage cannot provide.

In practice

If you design durable workflows for agents or internal automation:

  1. Start with SQLite on the node + Litestream to a bucket; do not pull in managed Postgres “just in case.”
  2. Define RPO/RTO explicitly: async backup fits dev/staging and research agents, not billing-critical state.
  3. Keep the execution log inspectable: one SQLite file is a convenient artifact for replay and post-mortems.
  4. Move to Postgres when you need multiple writers, strict uptime SLA, or cross-region access to one state.

Cheap workers around local SQLite give you a durable system with very little infrastructure — for AI agents, that may be the most sensible default.

Takeaway

“SQLite is all you need” is not a forever rejection of Postgres but the right complexity for your state on day one: local transactional DB, Litestream for portability, Postgres when HA and shared scale become real requirements. Read the original if you build orchestration for agents and do not want to overpay for infrastructure early.