Delivery and idempotency¶
AAA delivers events at least once. Messages can arrive twice (after a crash, after a worker stalls past its lease, after a rebalance). Rather than promising exactly-once delivery, which no distributed system can honestly give, AAA makes duplicates harmless. This page lists the mechanisms so you can reason about your own code.
What the library guarantees¶
| Mechanism | Guarantee |
|---|---|
| Append before deliver | A handler never sees an event that is not already durable (with a durable store). |
| Idempotent append | An event id appears at most once per stream. Re-appending returns the existing event with dup=True. |
| Deterministic ids | Events derived from a parent get ids computed from (parent, type, data); re-deriving them after a crash produces the same ids. |
| One result per request | The result of request R has id R:result. Two workers racing on R yield one recorded result. |
| Level-triggered algorithms | An algorithm decides from folded state, not from "the event that just arrived", so seeing an old event again changes nothing. |
| Serial per algorithm | An algorithm handles its events one at a time, in log order. |
| Atomic enqueue | With the Postgres/Redis stores, the bus entry is written in the same transaction/script as the event, so a stored event cannot fail to be published. |
| Durable acknowledgement | Long-running work (a tool call) keeps its bus message unacknowledged until it finishes, so a crash redelivers it. |
What it cannot guarantee¶
External side effects are not exactly-once
If a tool sends an email and the worker dies before recording the result, the call will be run again (under
at_least_once) and the email goes out twice. No library can fix this from the outside, because the effect
happened in a system AAA does not control. Your options, from strongest to weakest:
- Make the effect idempotent using
ctx.idempotency_key(StripeIdempotency-Key, SES dedupe,INSERT ... ON CONFLICT). Then repeats are free. - Declare
delivery="at_most_once": a lost attempt is reported asOutcomeUnknownfor a human or the algorithm to reconcile, and is never re-run. - Gate it with human approval (
requires_approval=True).
The tool execution guard is the machinery behind 1 and 2.
Reading the guarantees as a recipe¶
- Write tools to be idempotent, or declare
at_most_once. - Write algorithms as
(event, projected state) -> events, with no hidden state and no I/O. - Never rely on the count of times a handler runs.
- When you add an event type, give any event that could be re-derived a deterministic id (use
event.caused(...)).
Ordering, precisely¶
| Scope | Order |
|---|---|
| Within a stream (a run) | Total, by seq. |
| An algorithm's view of its own events | Log order, one at a time. |
| Across runs | None. |
| Capability handlers (LLM, tools) | A handler may process several requests of one run concurrently (that is the point of parallel tool calls); each result is its own event. |
| In a cluster | A run's events are consumed in order by whichever worker owns its partition. Ownership can move; the cursor guarantees continuity, and duplicates around a move are absorbed by idempotent appends. |
In-process vs. durable¶
With the default in-memory store and in-process bus, nothing survives a crash and delivery is effectively once. Every
guarantee above that mentions durability starts to apply as soon as you use SQLiteEventStore, PostgresEventStore
or RedisEventStore. Redis durability additionally depends on your Redis persistence settings (see
Running a cluster).