Install

Stream routes ​

The wire contract of streams: an append-only log with consumer groups, with the behavior of Redis Streams and ndb names. Every route is under one prefix, /{s}/_streams/..., where {s} is the stream (a collection made with "stream": true).

Bodies and answers are JSON. Errors have the body of every route (see Errors).

Entry IDs ​

An entry ID is <ms>-<seq>: two unsigned 64-bit integers, as text, for example 1731160000000-0.

  • The server makes the ID from its clock. ms is the Unix time in milliseconds; seq counts the entries of the same millisecond, from 0.
  • IDs always grow. A clock that goes back keeps the last ms and adds 1 to seq.
  • The ID is the _id of the entry. On a stream, _id sorts and compares as an ID (ms, then seq), not as text: 1-9 comes before 1-10. This is true in sort, in range filters ($gt, $gte, $lt, $lte) and in cursors.
  • Special values in requests:
ValueMeaning
*append: the server makes the ID
<ms>-*append: this ms, the server makes seq
$the last ID of the stream (group start, setid)
0 or 0-0before the first entry
-, +the lowest and the highest ID (ranges)

An ID without -seq (1731160000000) means <ms>-0 as a start and <ms>-<max> as an end, as in Redis.

Entries ​

An entry is a JSON object with any fields (nested values are allowed, unlike Redis), plus:

  • _id: the entry ID.
  • _ts: the ms of the ID as an RFC 3339 time. It is the time field of the stream, so retention, the zone maps and time filters work.

A field name that starts with _ is reserved, as in every collection.

Stream ​

PUT /{s} with "stream": true ​

Makes a stream (or checks that it is one). The first append makes it too. The definition takes the keys of every collection (indexes, retention, durability, ...) except time (it is _ts) and unique.

A collection that exists and is not a stream: 400 not_a_stream. A stream cannot become a normal collection.

POST /{s}/_streams/append ​

Redis: XADD.

json
{ "doc": { "order": 42, "total": 99.9 }, "id": "*", "trim": { "max_len": 10000, "approx": true } }
json
{ "_id": "1731160000000-0" }
  • doc is the entry. docs (an array) appends many in order and answers { "ids": [...] }; one fsync covers them all.
  • id: * (default), <ms>-* or <ms>-<seq>. An ID that is not greater than the last one: 400 stream_id_too_small. 0-0: 400 invalid_stream_id.
  • After the highest ID (18446744073709551615-18446744073709551615) no entry can come: every append is 400 stream_id_too_small ("the stream has exhausted the last possible ID", as in Redis). <ms>-* when the last ID is <ms>-18446744073709551615 is 400 stream_id_too_small too.
  • trim: optional, as in trim below, run after the append.
  • no_create: true: do not make the stream (NOMKSTREAM); a stream that does not exist is 404 unknown_collection.

POST /{s}/_streams/trim ​

Redis: XTRIM.

json
{ "max_len": 10000, "approx": true }
json
{ "deleted": 1520 }
  • max_len: keep the newest N entries. min_id: delete the entries with an ID lower than this one. One of the two.
  • approx: true deletes only whole blocks of 1 024 entries (~ in Redis): the stream keeps up to 1 023 more entries than asked, never fewer. A stream trimmed on every append then deletes once per block, not once per append. Default false (exact).
  • limit: with approx, delete at most this many entries, for max_len and for min_id. 0 (default) is no limit. limit without approx is 400 invalid_request, as LIMIT without ~ in Redis; a negative limit too.
  • A trim does not change the groups: a pending entry that the trim deleted stays in the PEL, and claim gives its ID in deleted.

GET /{s}/_streams ​

Redis: XINFO STREAM, XLEN.

json
{
  "length": 128000,
  "first_id": "1731160000000-0",
  "last_id": "1731163600000-4",
  "entries_added": 130000,
  "max_deleted_id": "1731160300000-0",
  "groups": 2
}
  • entries_added counts every entry ever appended, deleted ones too.
  • max_deleted_id is the highest ID deleted one by one (DELETE /{s}/_doc/{id}, XDEL). A trim does not change it, as in Redis.

POST /{s}/_streams/read ​

Redis: XREAD.

A read without a group: the entries after an ID, in ID order. It changes no group and no PEL, so it is a read for a scoped token.

json
{ "after": "1731163600000-4", "count": 100, "block_ms": 5000 }
json
[{ "_id": "1731163600000-5", "_ts": "2026-11-09T15:00:00Z", "order": 43 }]
  • after: an ID (default 0, the start of the stream), or $: the last ID when the request starts, so only later entries come.
  • count: the most entries to give (default 10, at most 10 000).
  • block_ms: with no entry, wait up to this long for an append (BLOCK). 0 answers at once (the default). An empty answer is []. A stream that does not exist is waited for too, as XREAD BLOCK on a missing key: the first append makes it and the read gives its entries (after $ is 0-0 then). The server looks for the new stream every 50 ms. With no block_ms, a missing stream is 404 unknown_collection.
  • fields: give only these fields (and _id).
  • To read on, send the last _id of the answer as after. Do not send $ again: an entry appended between two requests would be lost.
  • One stream for each request. Redis reads many streams in one XREAD.

Read, count and delete entries ​

These are the routes of every collection, on the stream:

Redisndb
XRANGE s a b COUNT nPOST /{s}/_search {"query":{"_id":{"$gte":"a","$lte":"b"}},"sort":{"_id":1},"limit":n}
XREVRANGE s b a COUNT nthe same with "sort":{"_id":-1}
XLEN sPOST /{s}/_count (or length above)
XDEL s idDELETE /{s}/_doc/{id}
XREAD STREAMS s idPOST /{s}/_streams/read (above), or _search with {"_id":{"$gt":"id"}} and sort on _id

On disk, the _id of an entry has both numbers padded to 20 digits, so that the text order is the ID order. The read routes hide it:

  • _search, _count and GET /{s}/_doc/{id} take and give <ms>-<seq>.
  • _aggregate takes <ms>-<seq> in the $match stages that see the entries: the ones before the first stage that changes the documents ($group, $project, ...; $sort, $skip and $limit do not). After such a stage, _id is not an entry ID and is not changed. The output rows give <ms>-<seq>.
  • _watch takes <ms>-<seq> in its query, and an event gives <ms>-<seq> in _id and in doc.
  • A pattern route (/{pattern}/_search, _count, _aggregate) whose pattern matches a stream is 400 invalid_request: one query cannot be right for the padded _id of a stream and the _id of a collection. Read a stream by its name.
  • A lookup (or a $lookup stage) into a stream is 400 invalid_request. A lookup from a stream into a collection works: it sees the _id as <ms>-<seq>.

Groups ​

A group reads the whole stream. Each entry goes to one consumer of the group and stays in the group's PEL (pending entries list) until it is acked.

A group has: last_delivered_id, entries_read (for the lag), and the PEL: for each pending entry, its consumer, the time of its last delivery and its delivery count. All of it is on disk before the answer (durability of the stream; strict by default).

PUT /{s}/_streams/groups/{g} ​

Redis: XGROUP CREATE.

json
{ "start": "$" }
json
{ "group": "workers", "last_delivered_id": "1731163600000-4" }
  • start: $ (only new entries, default), 0 (every entry) or an ID.
  • entries_read: optional, for the lag (ENTRIESREAD). Without it, the entries_read of the group is null (unknown), whatever start is, as in Redis 7; the lag is then an estimate (see GET below).
  • The stream is made if it does not exist (MKSTREAM is the default); no_create: true turns it off.
  • A group that exists: 409 group_exists.

POST /{s}/_streams/groups/{g}/setid ​

Redis: XGROUP SETID.

json
{ "id": "0", "entries_read": 0 }

DELETE /{s}/_streams/groups/{g} ​

Redis: XGROUP DESTROY.

Deletes the group and its PEL. Answers { "deleted": true }.

PUT /{s}/_streams/groups/{g}/consumers/{c} ​

Redis: XGROUP CREATECONSUMER.

Answers { "created": true }, or false if it exists. A read makes the consumer too.

DELETE /{s}/_streams/groups/{g}/consumers/{c} ​

Redis: XGROUP DELCONSUMER.

Deletes the consumer and its pending entries. Answers { "pending": 3 }, the number of entries it held.

GET /{s}/_streams/groups ​

Redis: XINFO GROUPS.

json
[{ "name": "workers", "consumers": 3, "pending": 12, "last_delivered_id": "1731163600000-4", "entries_read": 129988, "lag": 12 }]

entries_read and lag follow Redis 7 (XINFO GROUPS):

  • With a known entries_read and no entry deleted one by one (DELETE /{s}/_doc/{id}) at or after last_delivered_id, lag is entries_added - entries_read.
  • Else the lag is an estimate from last_delivered_id: 0 at the last ID; the entries left (length) when it is before the first entry, so after a trim of entries the group did not read; null when it cannot be known (between the first and the last entry with a deleted entry ahead, or after the last ID).
  • A read updates entries_read one by one while it is known and no deleted entry is ahead; else it takes the estimate, which may be null until the group reads the last entry. A stream with no entry ever has lag 0.

GET /{s}/_streams/groups/{g}/consumers ​

Redis: XINFO CONSUMERS.

json
[{ "name": "w1", "pending": 4, "idle_ms": 1200, "inactive_ms": 1100 }]

idle_ms is the time since the last read or claim of the consumer; inactive_ms since the last read that gave entries.

POST /{s}/_streams/groups/{g}/read ​

Redis: XREADGROUP.

json
{ "consumer": "w1", "count": 100, "block_ms": 5000 }
json
[{ "_id": "1731163600000-5", "_ts": "2026-11-09T15:00:00Z", "order": 43 }]
  • New entries (> in Redis, the default): the entries after last_delivered_id, in ID order. Each one goes into the PEL of this consumer with a delivery count of 1, and last_delivered_id moves.
  • pending: true (0 in Redis): the entries of this consumer's PEL, with an ID greater than after (default 0). Each one counts as a new delivery, as in Redis: delivery count + 1 and a new delivery time. An entry deleted from the stream comes as {"_id": "...", "_deleted": true}. A nacked entry has no owner, so it is not in this read (see nack).
  • count: the most entries to give (default 10, at most 10 000).
  • block_ms: with no new entry, wait up to this long for one (BLOCK). 0 answers at once (the default). An empty answer is []. Not with pending.
  • no_ack: true: do not add the entries to the PEL (NOACK): at most once.
  • fields: give only these fields (and _id).
  • The answer is the full entries, as JSON or as NDJSON with Accept: application/x-ndjson, like _search.
  • The consumer is made on its first read.

POST /{s}/_streams/groups/{g}/ack ​

Redis: XACK.

json
{ "ids": ["1731163600000-5", "1731163600000-6"] }
json
{ "acked": 2 }

An ID that is not pending in the group counts 0; it is not an error. An ack of a nacked entry removes it: so an app can cancel a retry.

POST /{s}/_streams/groups/{g}/nack ​

ndb only: Redis has no delayed retry. Gives entries back to the group after a wait, for a retry.

json
{ "consumer": "w1", "ids": ["1731163600000-5"], "delay_ms": 60000 }
json
{ "nacked": 1 }
  • Only the IDs pending for this consumer count; other IDs count 0, not an error (as ack). So a worker that lost an entry to a claim cannot nack it.
  • The entry stays in the PEL with no owner ("consumer": "") and a retry time retry_at = now + delay_ms. Its delivery count does not change; the claim that takes it again adds 1.
  • delay_ms: 0 (the default) makes the entry due at once (a plain "give it back"). At most 2 592 000 000 (30 days). A negative or larger value is 400 invalid_request; so is a body with no consumer.
  • All the IDs of one call are one transaction: one fsync.
  • While it waits, no route gives the entry: not read (it is no one's), not claim by IDs or by a scan, whatever min_idle_ms or force. After retry_at, any consumer of the group takes it with claim, at once: min_idle_ms does not count, the wait was already chosen. A claim clears retry_at.
  • DELETE of the consumer keeps its nacked entries (they have no owner).
  • A trim that deletes a nacked entry: the entry leaves the PEL at its next claim, in deleted.
  • The retry time is on disk with the PEL, so it survives a restart, a backup and a restore. A restore after the retry time makes the entry due at once.

POST /{s}/_streams/groups/{g}/pending ​

Redis: XPENDING.

Summary (an empty body):

json
{ "count": 12, "lowest": "1731163600000-1", "highest": "1731163600000-12", "consumers": [{ "name": "w1", "count": 4 }], "waiting": 1 }

waiting (ndb only): the nacked entries whose retry time has not come. A nacked entry counts in count, and for no consumer in consumers.

List (with count):

json
{ "start": "-", "end": "+", "count": 100, "consumer": "w1", "idle_ms": 60000 }
json
[
  { "_id": "1731163600000-1", "consumer": "w1", "idle_ms": 61000, "deliveries": 2, "retry_at": null },
  { "_id": "1731163600000-2", "consumer": "", "idle_ms": 3000, "deliveries": 1, "retry_at": 1731163660000 }
]

consumer and idle_ms are optional filters.

  • retry_at (ndb only): the retry time of a nacked entry, Unix time in milliseconds (the unit of the ms of an ID); null when the entry is not nacked. A nacked entry has "consumer": "".
  • due (ndb only, optional): true gives only the nacked entries whose retry time passed (a claim takes them now); false only the ones that still wait. With no due, every entry.

POST /{s}/_streams/groups/{g}/claim ​

Redis: XCLAIM, XAUTOCLAIM.

By IDs (XCLAIM):

json
{ "consumer": "w2", "min_idle_ms": 60000, "ids": ["1731163600000-1"] }

With a scan (XAUTOCLAIM), instead of ids:

json
{ "consumer": "w2", "min_idle_ms": 60000, "cursor": "0-0", "count": 100 }
json
{ "cursor": "1731163600000-9", "entries": [{ "_id": "1731163600000-1", "order": 40 }], "deleted": [] }
  • Only the entries idle for at least min_idle_ms move to consumer. Each one gets a delivery count + 1 and a new delivery time.
  • cursor is 0-0 when the scan is complete (only with a scan).
  • deleted: pending IDs whose entry is no longer in the stream; they leave the PEL.
  • just_ids: true (JUSTID): entries are IDs only, and the delivery count does not change.
  • By IDs only: idle_ms, retry_count and force as in XCLAIM. idle_ms sets the delivery time to now − idle_ms. A time in the future (a negative idle_ms) or before 1970 is now, as Redis 7.4 does with IDLE and TIME (it clamps them to now): a claim never delays an entry. ndb has no time key. A negative min_idle_ms is 0, as in Redis.
  • A nacked entry is never claimed before its retry time; after it, it is claimed at once, whatever min_idle_ms (see nack).

Due scan (ndb only), instead of ids:

json
{ "consumer": "w2", "due": true, "cursor": "0-0", "count": 100 }
json
{ "cursor": "0-0", "entries": [{ "_id": "1731163600000-2", "order": 41 }], "deleted": [] }
  • It looks only at the nacked entries (an index of them), so it does not read the entries in flight at live workers: a worker can call it on a short tick. It claims the ones whose retry time passed, in ID order.
  • cursor is optional (default 0-0) and the answer has the next one, as a scan. min_idle_ms does not count; just_ids works. due with ids is 400 invalid_request.

Errors ​

CodeHTTPMeaning
not_a_stream400A _streams route, or "stream": true, on a collection that is not a stream.
stream_append_only400POST /{s}/_doc, _bulk, _update, _update_by_query, _delete_by_query, _expire or _persist on a stream. An entry is added only by an append, never changes, and is deleted with DELETE /{s}/_doc/{id} or a trim.
invalid_stream_id400An ID that is not <ms>-<seq>, a special value where it is not allowed, or 0-0 in an append.
stream_id_too_small400An append with an ID that is not greater than the last one, or any append after the highest ID.
unknown_group404The group does not exist.
group_exists409A group create with a name that exists.

Access ​

append, trim, every group change, read (it moves the PEL), ack, nack and claim are writes. GET /{s}/_streams, the GET of groups and consumers, pending and POST /{s}/_streams/read are reads.