std::db

Talking to a database, as a capability rather than a library.

The ecosystem design note §“Applying the rule” settles what belongs here and it is not an engine. SQLite has no middle layer at all — no framing, no handshake, nothing that fails at a packet boundary — so putting it in std would be the mistake std::net::http avoided, all top layer plus a quarter of a million lines of C. It belongs in a package rather than here. Postgres belongs in one for the ordinary reason: its wire protocol is versioned by somebody else.

No driver ships in std, and this interface is the whole of what is here. Db is a record of closures, so writing a handler over an existing client is a day’s work and a test double is a few lines. Said plainly because the shape of this module invites the opposite assumption: an interface this settled usually means implementations exist. For the drivers maintained alongside the compiler, see the Packages section of the documentation.

What the middle layer actually is

What a transaction does when its fiber is canceled. That is the part that fails in production and never in testing, the part every package would answer differently, and the part that is Khora’s to answer rather than an engine’s — because cancellation semantics belong to the language. A transaction that returns without rolling back, holding a connection and its locks, is the truncated-request-at-a-packet-boundary of this subject.

So std owns three things: the Db capability’s shape, the row and value types two packages must agree on to exchange a result, and the contract below. Everything that speaks a protocol is somebody else’s.

The contract

transaction commits when its body returns and rolls back when it does not — including when the body raises, and including when the fiber running it is canceled. A handler that cannot promise that is a broken handler, and the test beside this module is written against the promise rather than against any engine.

A rollback that itself fails is two facts, and they go to two places. What the caller is told is RolledBack with the body’s reason, because the engine’s complaint about a rollback is a worse thing to report than the reason the rollback was needed. What the handler is told is broken, because a connection whose ROLLBACK did not arrive may still be holding a transaction open and is not one to lend out again.

The second half is why broken is an operation rather than a return value: on the cancellation path there is no caller left to return anything to, and that is exactly the path where a connection goes back to a pool.

Database authority is carried by the capability row all the way through. Application code says with { db: Db }; transaction says the same thing. The concrete handler belongs at a composition boundary, not as a Db parameter threaded through domain functions.

All three ways out are covered: the rollback is registered with a region before BEGIN is sent, and a region’s finalizers run on every path out, a cancellation’s included. transaction’s own comment has the argument. What the runtime owes in return is that a finalizer running because of a cancellation is not itself canceled — khora-rt’s cancel::Shielded.

API at a glance

Types: Cell · Row · DbError

Effects: Db

Methods: Cell · Row · Cell

Trait implementations: Eq for Cell · Show for Cell · Show for DbError

Functions: transaction

Types

Cell

pub type Cell =
  | Null
  | Text(String)
  | Number(Int)
  | Money(Decimal)
  | Flag(Bool);

A value in a column, or bound to a parameter.

Decimal is here and Float is not, for money. A database NUMERIC read through a float is a number that has already lost, and The numbers design note is the long version. Floats can be added when something that is genuinely a measurement needs one.

A numeric column arrives as Text, not as Money, and the PostgreSQL driver is not alone in that. numeric is arbitrary-precision, so a driver that produced a Money would have to decide what to do with digits that will not parse — become a wrong number, or lose the value — and neither is a decision a driver may take on a caller’s behalf. The server’s own digits are kept instead, and turning them into a Decimal is the caller’s step: Decimal::of_string, whose Option::None arm means this is not a number I can represent and must not be answered with a zero.

What it costs is that Cell::money answers None for every money column a PostgreSQL server returns. Read those through Cell::text.

Row

pub type Row = {
  pub columns: List<String>,
  pub cells: List<Cell>,
};

One row: the columns the query named, and a cell under each.

The names are what let a row be read through a schema – Entry::schema() reads id, account, amount and memo by name – and a driver that has them has no reason to drop them. A driver that genuinely does not (a bare SELECT with no description) leaves them empty, and Row::to_raw then hands over a sequence rather than a record.

DbError

derive(Eq)
pub type DbError =
  | Rejected(String)
  | Disconnected(String)
  | RolledBack(String);

What went wrong, as far as std is willing to say.

Deliberately coarse, on the same reasoning std::fs gives for not knowing C’s error numbers: a code that means “unique violation” is a different integer in every engine, and a std that pretended otherwise would be wrong for the second engine anybody wrote. A package that knows its engine can offer a richer type; this is what everything can agree on.

Rejected

| Rejected(String)

The statement was not accepted.

Disconnected

| Disconnected(String)

The connection is gone.

RolledBack

| RolledBack(String)

The transaction was rolled back rather than committed.

Effects

Db

pub effect Db {
  query: (String, List<Cell>) -> Result<List<Row>, DbError>,
  execute: (String, List<Cell>) -> Result<Int, DbError>,
  depth: () -> Int,
  begin: () -> Result<(), DbError>,
  commit: () -> Result<(), DbError>,
  rollback: () -> Result<(), DbError>,
  savepoint: (Int) -> Result<(), DbError>,
  release: (Int) -> Result<(), DbError>,
  rollback_to: (Int) -> Result<(), DbError>,
  broken: () -> (),
  query_each: (String, List<List<Cell>>) -> List<Result<List<Row>, DbError>>,
}

Somewhere to run statements.

A record of closures, so a Postgres package, a SQLite package, D1 behind a Worker and an in-memory double are the same type to everything above.

Nesting

The handler keeps the depth, because the depth is the connection’s. Two fibers leasing two connections nest independently, and one db reached twice from one fiber is one connection with one depth, so a counter anywhere but beside the connection would be wrong for one of the two. The handler is the one thing there is exactly one of per connection.

transaction names the level; the handler does not guess it. The alternative, a handler that turns begin into SAVEPOINT when a transaction is already open, cannot tell a stray rollback from a real one: transaction registers its undo before anything is sent, so a cancel can deliver a rollback for an inner level that never opened, and a handler guessing the level would roll back the enclosing transaction; so transaction reads depth first, and every savepoint operation carries the level it belongs to.

What it costs a handler author is four more operations and one counter, and one more exchange with the connection per transaction.

query

query: (String, List) -> Result<List, DbError>

Runs a query and gives back every row.

execute

execute: (String, List) -> Result<Int, DbError>

Runs a statement and gives back how many rows it changed.

depth

depth: () -> Int

How many transactions are open on this connection: 0 outside one, 1 inside a transaction, and one more for each savepoint inside that.

Answers, and does not fail. A connection that is gone answers 0, and the begin that follows is what reports it.

begin

begin: () -> Result<(), DbError>

Begins a transaction, taking the depth from 0 to 1.

commit

commit: () -> Result<(), DbError>

Commits the transaction in progress, taking the depth to 0.

Ok means committed, and nothing weaker. An engine that ends a failed transaction with a rollback when asked to commit – PostgreSQL does, without an error – has not committed, and the handler answers RolledBack.

rollback

rollback: () -> Result<(), DbError>

Rolls the transaction in progress back.

May be called when no transaction is open, and must answer Ok then. transaction registers its rollback before it sends BEGIN, so a cancel that lands before the BEGIN went out still rolls back, and a handler that refused would be told broken about a healthy connection. PostgreSQL answers a stray ROLLBACK with a warning, not an error.

savepoint

savepoint: (Int) -> Result<(), DbError>

Opens a savepoint at level, which is the current depth, taking the depth to level + 1.

A handler refuses when the depth is not level: that means somebody else changed it between transaction reading it and asking, and a savepoint opened at the wrong level would be released by the wrong transaction.

release

release: (Int) -> Result<(), DbError>

Keeps what was done since the savepoint at level as part of the enclosing transaction, taking the depth back to level.

Ok means kept. An engine that refuses the release – PostgreSQL does inside a transaction a failed statement aborted – leaves the savepoint open, and transaction then rolls back to it.

rollback_to

rollback_to: (Int) -> Result<(), DbError>

Undoes everything since the savepoint at level and removes it, taking the depth back to level.

May be called for a savepoint that was never opened, when the depth is level or less, and must answer Ok without touching the engine then, for the reason rollback gives. PostgreSQL answers a ROLLBACK TO for a savepoint it does not have with an error that aborts the enclosing transaction, so a handler that sent it would undo work nobody asked it to.

broken

broken: () ->()

Says this connection’s state is no longer known.

Called when a rollback fails, and when a BEGIN or a COMMIT comes back Disconnected. A ROLLBACK that does not arrive leaves a connection that may still be holding a transaction open and rows locked, and there is no second thing to try: another rollback would fail the same way, and asking the server what state it is in needs a working connection to ask over.

The other two are the same shape from a different direction. A Disconnected from begin or commit says the exchange never completed, so nobody knows whether the statement reached the server; a Rejected says the engine answered and refused, which ends the transaction and leaves the connection perfectly usable, so that one does not call this.

So the handler is told, because the handler is the connection and is the only thing that can act. A pool stops lending it out. A single connection may close it. A test counts it.

Answers nothing and cannot fail, because it is called from a region finalizer on the way out — including on the way out of a cancellation, where there is no caller left to tell. A signal that could fail here would need a handler for its own failure, and that handler would be this one.

query_each

query_each: (String, List<List>) -> List<Result<List, DbError>>

Runs one statement once per set of values, in order, and gives back an answer per set: the same result as calling query once per set.

Each set is its own statement. Outside a transaction each runs in its own implicit one, so a set that fails fails alone and the sets after it still run; inside a transaction a failed set aborts it, and the sets after it are refused, exactly as the same query calls would be.

What this is for: a round trip per set. A handler may send every set before reading any reply, so twenty lookups cost one exchange with the server instead of twenty. That is safe because of the contract, not the handler: a caller cannot tell a pipelined batch from a loop. A handler that cannot pipeline calls its own query once per set, in order, and is correct.

The answer has one entry per set, in the order of sets, even when the connection is lost part-way: the sets it did not reach are answered with the reason, as a query on a lost connection would be.

Methods

Cell

impl Cell

text

pub fn text(self) -> Option<String>

The text in this cell, or None if it holds something else.

No coercion. A Number read as text is a schema misunderstanding, and a library that rendered it would hide the day somebody changed a column’s type.

number

pub fn number(self) -> Option<Int>

The number in this cell, or None.

money

pub fn money(self) -> Option<Decimal>

The decimal in this cell, or None.

None for a PostgreSQL numeric, which arrives as Cell::Text — see the note on Cell for why, and read it with Cell::text and Decimal::of_string. A money column that reads as None here is the normal case, not a null.

flag

pub fn flag(self) -> Option<Bool>

The boolean in this cell, or None.

is_null

pub fn is_null(self) -> Bool

Whether the cell is null, which is a question rather than a value.

Row

impl Row

cell

pub fn cell(self, index: Int) -> Option<Cell>

The cell at index, or None past the end.

named

pub fn named(self, name: String) -> Option<Cell>

The cell under name, or None when no column has it.

to_raw

pub fn to_raw(self) -> Raw

The row as a source hands it over, keyed by column, for a schema to read.

A Number and a Money become the token their digits spell, which both round-trip exactly, so Raw keeps one numeric representation. Text is text the column said was text, and is not read as a number.

sequence

pub fn sequence(rows: List<Row>) -> Raw

Every row, in order, for list(Entry::schema()) to read at once: a problem in the third row is reported as [2].amount.

Cell

impl Cell

to_raw

pub fn to_raw(self) -> Raw

The cell as a source hands it over.

Trait implementations

Eq for Cell

impl Eq for Cell

eq

fn eq(self, other: Cell) -> Bool

Show for Cell

impl Show for Cell

show

fn show(self) -> String

Show for DbError

impl Show for DbError

show

fn show(self) -> String

Written out rather than derived, unlike the Eq above.

A derived Show gives DbError::Rejected(...), which is right for the error types whose cases a reader has to tell apart. This one goes into a log line a person reads, and “rejected: duplicate key” is the sentence they want rather than a constructor.

Functions

transaction

pub fn transaction<A, 'ef, 'er>(body: () -> Result<A, DbError> with 'ef raises 'er) -> Result<A, DbError> with { 'ef | db: Db } raises 'er

Runs body in a transaction, committing on success and rolling back otherwise.

The database is a capability, not an ordinary argument. The caller must have db: Db in its capability row; the same binding is used for begin, the body, commit, and rollback.

This is the function the whole module exists for. A caller that wrote begin, body and commit by hand has written the correct thing only for the path where nothing goes wrong, and the paths where something does are the ones that leave a connection holding locks until somebody restarts the service.

The body’s failure is a DbError, not a type variable. A body doing database work produces one already, and a caller with an error of its own renders it into Rejected before getting here — which keeps this function concrete, and keeps the rendering where the type is still known rather than behind a Show the transaction would have to carry.

How the third way out is covered

A body can return, it can fail, and its fiber can be canceled — and the third one does not pass through any match written here. A cancellation travels the same tagged return an error does, straight out of this function, so a version that only inspected body()’s answer was correct for two paths out of three. The one it missed is the one that happens in production: a request times out, the nursery stops its children, and a connection goes back to the pool inside an open transaction holding its locks.

So the rollback is registered before BEGIN is sent rather than decided after the body, with a region: a region’s finalizers run on every path out, because every path out already releases the binding holding the region. Committing marks the transaction settled, and a settled transaction has nothing left to undo — so the finalizer costs a boolean on the ordinary path and is the whole contract on the extraordinary one.

Before BEGIN, not after it, because a fiber can be canceled while it waits for the server to answer BEGIN, and by then the server is inside the transaction. A rollback registered only once begin returns would miss that cancel, and the connection would go back to its pool with the transaction still open, where the next borrower’s writes are answered Ok and never committed. What registering first costs is a ROLLBACK sent to a server that never received the BEGIN, which it answers with a warning.

A cancel during the commit also rolls back. Whether a COMMIT cut off before its reply ran on the server is unknown, and a ROLLBACK is right either way: after a commit that ran it changes nothing, and after one that did not it ends the transaction. The connection goes back to its pool outside a transaction in both cases. A commit that loses its connection is answered Disconnected with a message saying the outcome is unknown, because the transaction may have committed, and a caller who reads “disconnected” as “it did not happen” and retries can apply it twice.

The region is this function’s own, for the reason postgres::pool gives at greater length: a transaction that borrowed its caller’s scope would end when the caller did, which is not what a transaction is.

Cancellation inside a transaction

A transaction is canceled like any other code: at a loop, at a blocking std call, or at a call to a function that can reach one, whatever the body’s raises row. A cancellation that lands in the body unwinds through here, releasing the region and running the rollback. The rollback itself runs as cleanup, so a plain cancel does not interrupt it. abort runs it too, and can stop it only at a cancellation point inside the handler’s rollback, such as the wait for the engine’s reply. A body that reaches no cancellation point runs to its commit.

raises 'er is there for the body’s own failures. The row is the caller’s, deliberately. A transaction over a body that can fail its own way keeps that failure’s type — folding it into DbError would be the loss Result<A, DbError> already declines to take.

Placeholder syntax belongs to the driver, not to Db. This example uses PostgreSQL’s $1, $2, which is what the postgres package takes; another engine’s handler would take that engine’s spelling. Db passes the string through untouched.

import std::db::{Cell, Db, DbError, transaction};

fn transfer(from: Int, to: Int, amount: Int) -> Result<(), DbError>
  with { db: Db }
{
  transaction(fn () =>
    match db.execute("update accounts set balance = balance - $1 where id = $2",
                     [Cell::Number(amount), Cell::Number(from)]) {
      Result::Err(problem) => Result::Err(problem),
      Result::Ok(_) => db.execute("update accounts set balance = balance + $1 where id = $2",
                                  [Cell::Number(amount), Cell::Number(to)]),
    })
}

There is no connection parameter. with { db: Db } is the function’s authority to run statements, and the caller supplies it at a boundary – which is what keeps a transaction from turning the capability back into plumbing threaded through every signature.

A transaction inside a transaction

Nested for real, with a savepoint. A transaction whose db is already inside one, however deep, opens a savepoint instead of a transaction. Its Ok releases the savepoint, so its writes commit or roll back with the enclosing transaction; its Err, a raise or a cancel rolls back to the savepoint, which undoes only its own writes, and the enclosing body is told RolledBack and can carry on. Nothing commits until the outermost transaction does.

What this prevents is an inner COMMIT or ROLLBACK ending the enclosing transaction, which an engine does to a second BEGIN sent inside one: the caller is then told one thing and something else is committed.

Fibers sharing one lease must not run transaction at the same time. The depth is the connection’s, so two fibers nesting on one connection would each release or roll back the other’s savepoint. A handler that refuses an operation for a level its connection is not at, as the postgres package’s does, turns that into an Err for one of them, and nothing that one wrote is kept; it does not make both work.

A refusal nested inside Ok commits

A is unconstrained, so Result<Result<(), Refusal>, DbError> is a legal body type and transaction commits it. A body has three outcomes more often than two: it worked, the database broke, or the business said no – an overdraft, a closed account, a mismatched currency. A refusal is not a DbError, so the obvious typing puts it in the Ok channel, where this function cannot see it. The rollback that the body’s author expected does not happen, and whatever the body wrote before the refusal is committed.

The failure is quiet and it is not the database’s: an idempotency key inserted at the top of the body outlives a refusal, so retrying the same request after the refusal’s cause is gone answers “already done” forever.

Roll back on a refusal by raising it, not by returning it. transaction carries a raises 'er row for exactly this: a raise leaves the body without an Ok, so the transaction rolls back, and the row carries the refusal out to a caller that can answer it.

fn transfer() -> Result<(), DbError> with { db: Db } raises Refusal {
  // ... a refusal is `raise Refusal(why)`, not `Result::Err(..)`
}

// At the boundary, where the refusal becomes the caller's answer:
let outcome = attempt(fn () => transaction(fn () => transfer()!)!);

The body’s Err channel is DbError and nothing else, so a refusal cannot travel that way — it does not typecheck. The rule is that anything which must undo the transaction leaves the body as a raise or as Result::Err(DbError); Ok means commit, whatever is inside it.