Stream routes
The wire contract of streams: an append-only log with consumer groups, with the behavior of Redis Streams and ndb names. Every route is under one prefix, /{s}/_streams/..., where {s} is the stream (a collection made with "stream": true).
Bodies and answers are JSON. Errors have the body of every route (see Errors).
Entry IDs
An entry ID is <ms>-<seq>: two unsigned 64-bit integers, as text, for example 1731160000000-0.
- The server makes the ID from its clock.
msis the Unix time in milliseconds;seqcounts the entries of the same millisecond, from 0. - IDs always grow. A clock that goes back keeps the last
msand adds 1 toseq. - The ID is the
_idof the entry. On a stream,_idsorts and compares as an ID (ms, thenseq), not as text:1-9comes before1-10. This is true insort, in range filters ($gt,$gte,$lt,$lte) and in cursors. - Special values in requests:
| Value | Meaning |
|---|---|
* | append: the server makes the ID |
<ms>-* | append: this ms, the server makes seq |
$ | the last ID of the stream (group start, setid) |
0 or 0-0 | before the first entry |
-, + | the lowest and the highest ID (ranges) |
An ID without -seq (1731160000000) means <ms>-0 as a start and <ms>-<max> as an end, as in Redis.
Entries
An entry is a JSON object with any fields (nested values are allowed, unlike Redis), plus:
_id: the entry ID._ts: themsof the ID as an RFC 3339 time. It is thetimefield of the stream, soretention, the zone maps and time filters work.
A field name that starts with _ is reserved, as in every collection.
Stream
PUT /{s} with "stream": true
Makes a stream (or checks that it is one). The first append makes it too. The definition takes the keys of every collection (indexes, retention, durability, ...) except time (it is _ts) and unique.
A collection that exists and is not a stream: 400 not_a_stream. A stream cannot become a normal collection.
POST /{s}/_streams/append
Redis: XADD.
{ "doc": { "order": 42, "total": 99.9 }, "id": "*", "trim": { "max_len": 10000, "approx": true } }{ "_id": "1731160000000-0" }docis the entry.docs(an array) appends many in order and answers{ "ids": [...] }; one fsync covers them all.id:*(default),<ms>-*or<ms>-<seq>. An ID that is not greater than the last one:400 stream_id_too_small.0-0:400 invalid_stream_id.- After the highest ID (
18446744073709551615-18446744073709551615) no entry can come: every append is400 stream_id_too_small("the stream has exhausted the last possible ID", as in Redis).<ms>-*when the last ID is<ms>-18446744073709551615is400 stream_id_too_smalltoo. trim: optional, as intrimbelow, run after the append.no_create: true: do not make the stream (NOMKSTREAM); a stream that does not exist is404 unknown_collection.
POST /{s}/_streams/trim
Redis: XTRIM.
{ "max_len": 10000, "approx": true }{ "deleted": 1520 }max_len: keep the newest N entries.min_id: delete the entries with an ID lower than this one. One of the two.approx: truedeletes only whole blocks of 1 024 entries (~in Redis): the stream keeps up to 1 023 more entries than asked, never fewer. A stream trimmed on every append then deletes once per block, not once per append. Defaultfalse(exact).limit: withapprox, delete at most this many entries, formax_lenand formin_id.0(default) is no limit.limitwithoutapproxis400 invalid_request, asLIMITwithout~in Redis; a negativelimittoo.- A trim does not change the groups: a pending entry that the trim deleted stays in the PEL, and
claimgives its ID indeleted.
GET /{s}/_streams
Redis: XINFO STREAM, XLEN.
{
"length": 128000,
"first_id": "1731160000000-0",
"last_id": "1731163600000-4",
"entries_added": 130000,
"max_deleted_id": "1731160300000-0",
"groups": 2
}entries_addedcounts every entry ever appended, deleted ones too.max_deleted_idis the highest ID deleted one by one (DELETE /{s}/_doc/{id},XDEL). A trim does not change it, as in Redis.
POST /{s}/_streams/read
Redis: XREAD.
A read without a group: the entries after an ID, in ID order. It changes no group and no PEL, so it is a read for a scoped token.
{ "after": "1731163600000-4", "count": 100, "block_ms": 5000 }[{ "_id": "1731163600000-5", "_ts": "2026-11-09T15:00:00Z", "order": 43 }]after: an ID (default0, the start of the stream), or$: the last ID when the request starts, so only later entries come.count: the most entries to give (default 10, at most 10 000).block_ms: with no entry, wait up to this long for an append (BLOCK).0answers at once (the default). An empty answer is[]. A stream that does not exist is waited for too, asXREAD BLOCKon a missing key: the first append makes it and the read gives its entries (after$is0-0then). The server looks for the new stream every 50 ms. With noblock_ms, a missing stream is404 unknown_collection.fields: give only these fields (and_id).- To read on, send the last
_idof the answer asafter. Do not send$again: an entry appended between two requests would be lost. - One stream for each request. Redis reads many streams in one
XREAD.
Read, count and delete entries
These are the routes of every collection, on the stream:
| Redis | ndb |
|---|---|
XRANGE s a b COUNT n | POST /{s}/_search {"query":{"_id":{"$gte":"a","$lte":"b"}},"sort":{"_id":1},"limit":n} |
XREVRANGE s b a COUNT n | the same with "sort":{"_id":-1} |
XLEN s | POST /{s}/_count (or length above) |
XDEL s id | DELETE /{s}/_doc/{id} |
XREAD STREAMS s id | POST /{s}/_streams/read (above), or _search with {"_id":{"$gt":"id"}} and sort on _id |
On disk, the _id of an entry has both numbers padded to 20 digits, so that the text order is the ID order. The read routes hide it:
_search,_countandGET /{s}/_doc/{id}take and give<ms>-<seq>._aggregatetakes<ms>-<seq>in the$matchstages that see the entries: the ones before the first stage that changes the documents ($group,$project, ...;$sort,$skipand$limitdo not). After such a stage,_idis not an entry ID and is not changed. The output rows give<ms>-<seq>._watchtakes<ms>-<seq>in itsquery, and an event gives<ms>-<seq>in_idand indoc.- A pattern route (
/{pattern}/_search,_count,_aggregate) whose pattern matches a stream is400 invalid_request: one query cannot be right for the padded_idof a stream and the_idof a collection. Read a stream by its name. - A
lookup(or a$lookupstage) into a stream is400 invalid_request. A lookup from a stream into a collection works: it sees the_idas<ms>-<seq>.
Groups
A group reads the whole stream. Each entry goes to one consumer of the group and stays in the group's PEL (pending entries list) until it is acked.
A group has: last_delivered_id, entries_read (for the lag), and the PEL: for each pending entry, its consumer, the time of its last delivery and its delivery count. All of it is on disk before the answer (durability of the stream; strict by default).
PUT /{s}/_streams/groups/{g}
Redis: XGROUP CREATE.
{ "start": "$" }{ "group": "workers", "last_delivered_id": "1731163600000-4" }start:$(only new entries, default),0(every entry) or an ID.entries_read: optional, for the lag (ENTRIESREAD). Without it, theentries_readof the group isnull(unknown), whateverstartis, as in Redis 7; the lag is then an estimate (seeGETbelow).- The stream is made if it does not exist (
MKSTREAMis the default);no_create: trueturns it off. - A group that exists:
409 group_exists.
POST /{s}/_streams/groups/{g}/setid
Redis: XGROUP SETID.
{ "id": "0", "entries_read": 0 }DELETE /{s}/_streams/groups/{g}
Redis: XGROUP DESTROY.
Deletes the group and its PEL. Answers { "deleted": true }.
PUT /{s}/_streams/groups/{g}/consumers/{c}
Redis: XGROUP CREATECONSUMER.
Answers { "created": true }, or false if it exists. A read makes the consumer too.
DELETE /{s}/_streams/groups/{g}/consumers/{c}
Redis: XGROUP DELCONSUMER.
Deletes the consumer and its pending entries. Answers { "pending": 3 }, the number of entries it held.
GET /{s}/_streams/groups
Redis: XINFO GROUPS.
[{ "name": "workers", "consumers": 3, "pending": 12, "last_delivered_id": "1731163600000-4", "entries_read": 129988, "lag": 12 }]entries_read and lag follow Redis 7 (XINFO GROUPS):
- With a known
entries_readand no entry deleted one by one (DELETE /{s}/_doc/{id}) at or afterlast_delivered_id,lagisentries_added - entries_read. - Else the lag is an estimate from
last_delivered_id:0at the last ID; the entries left (length) when it is before the first entry, so after a trim of entries the group did not read;nullwhen it cannot be known (between the first and the last entry with a deleted entry ahead, or after the last ID). - A read updates
entries_readone by one while it is known and no deleted entry is ahead; else it takes the estimate, which may benulluntil the group reads the last entry. A stream with no entry ever has lag0.
GET /{s}/_streams/groups/{g}/consumers
Redis: XINFO CONSUMERS.
[{ "name": "w1", "pending": 4, "idle_ms": 1200, "inactive_ms": 1100 }]idle_ms is the time since the last read or claim of the consumer; inactive_ms since the last read that gave entries.
POST /{s}/_streams/groups/{g}/read
Redis: XREADGROUP.
{ "consumer": "w1", "count": 100, "block_ms": 5000 }[{ "_id": "1731163600000-5", "_ts": "2026-11-09T15:00:00Z", "order": 43 }]- New entries (
>in Redis, the default): the entries afterlast_delivered_id, in ID order. Each one goes into the PEL of this consumer with a delivery count of 1, andlast_delivered_idmoves. pending: true(0in Redis): the entries of this consumer's PEL, with an ID greater thanafter(default0). Each one counts as a new delivery, as in Redis: delivery count + 1 and a new delivery time. An entry deleted from the stream comes as{"_id": "...", "_deleted": true}. A nacked entry has no owner, so it is not in this read (see nack).count: the most entries to give (default 10, at most 10 000).block_ms: with no new entry, wait up to this long for one (BLOCK).0answers at once (the default). An empty answer is[]. Not withpending.no_ack: true: do not add the entries to the PEL (NOACK): at most once.fields: give only these fields (and_id).- The answer is the full entries, as JSON or as NDJSON with
Accept: application/x-ndjson, like_search. - The consumer is made on its first read.
POST /{s}/_streams/groups/{g}/ack
Redis: XACK.
{ "ids": ["1731163600000-5", "1731163600000-6"] }{ "acked": 2 }An ID that is not pending in the group counts 0; it is not an error. An ack of a nacked entry removes it: so an app can cancel a retry.
POST /{s}/_streams/groups/{g}/nack
ndb only: Redis has no delayed retry. Gives entries back to the group after a wait, for a retry.
{ "consumer": "w1", "ids": ["1731163600000-5"], "delay_ms": 60000 }{ "nacked": 1 }- Only the IDs pending for this consumer count; other IDs count 0, not an error (as
ack). So a worker that lost an entry to a claim cannot nack it. - The entry stays in the PEL with no owner (
"consumer": "") and a retry timeretry_at= now +delay_ms. Its delivery count does not change; the claim that takes it again adds 1. delay_ms:0(the default) makes the entry due at once (a plain "give it back"). At most 2 592 000 000 (30 days). A negative or larger value is400 invalid_request; so is a body with noconsumer.- All the IDs of one call are one transaction: one fsync.
- While it waits, no route gives the entry: not
read(it is no one's), notclaimby IDs or by a scan, whatevermin_idle_msorforce. Afterretry_at, any consumer of the group takes it withclaim, at once:min_idle_msdoes not count, the wait was already chosen. A claim clearsretry_at. DELETEof the consumer keeps its nacked entries (they have no owner).- A trim that deletes a nacked entry: the entry leaves the PEL at its next claim, in
deleted. - The retry time is on disk with the PEL, so it survives a restart, a backup and a restore. A restore after the retry time makes the entry due at once.
POST /{s}/_streams/groups/{g}/pending
Redis: XPENDING.
Summary (an empty body):
{ "count": 12, "lowest": "1731163600000-1", "highest": "1731163600000-12", "consumers": [{ "name": "w1", "count": 4 }], "waiting": 1 }waiting (ndb only): the nacked entries whose retry time has not come. A nacked entry counts in count, and for no consumer in consumers.
List (with count):
{ "start": "-", "end": "+", "count": 100, "consumer": "w1", "idle_ms": 60000 }[
{ "_id": "1731163600000-1", "consumer": "w1", "idle_ms": 61000, "deliveries": 2, "retry_at": null },
{ "_id": "1731163600000-2", "consumer": "", "idle_ms": 3000, "deliveries": 1, "retry_at": 1731163660000 }
]consumer and idle_ms are optional filters.
retry_at(ndb only): the retry time of a nacked entry, Unix time in milliseconds (the unit of themsof an ID);nullwhen the entry is not nacked. A nacked entry has"consumer": "".due(ndb only, optional):truegives only the nacked entries whose retry time passed (a claim takes them now);falseonly the ones that still wait. With nodue, every entry.
POST /{s}/_streams/groups/{g}/claim
Redis: XCLAIM, XAUTOCLAIM.
By IDs (XCLAIM):
{ "consumer": "w2", "min_idle_ms": 60000, "ids": ["1731163600000-1"] }With a scan (XAUTOCLAIM), instead of ids:
{ "consumer": "w2", "min_idle_ms": 60000, "cursor": "0-0", "count": 100 }{ "cursor": "1731163600000-9", "entries": [{ "_id": "1731163600000-1", "order": 40 }], "deleted": [] }- Only the entries idle for at least
min_idle_msmove toconsumer. Each one gets a delivery count + 1 and a new delivery time. cursoris0-0when the scan is complete (only with a scan).deleted: pending IDs whose entry is no longer in the stream; they leave the PEL.just_ids: true(JUSTID):entriesare IDs only, and the delivery count does not change.- By IDs only:
idle_ms,retry_countandforceas inXCLAIM.idle_mssets the delivery time to now −idle_ms. A time in the future (a negativeidle_ms) or before 1970 is now, as Redis 7.4 does withIDLEandTIME(it clamps them to now): a claim never delays an entry. ndb has notimekey. A negativemin_idle_msis 0, as in Redis. - A nacked entry is never claimed before its retry time; after it, it is claimed at once, whatever
min_idle_ms(see nack).
Due scan (ndb only), instead of ids:
{ "consumer": "w2", "due": true, "cursor": "0-0", "count": 100 }{ "cursor": "0-0", "entries": [{ "_id": "1731163600000-2", "order": 41 }], "deleted": [] }- It looks only at the nacked entries (an index of them), so it does not read the entries in flight at live workers: a worker can call it on a short tick. It claims the ones whose retry time passed, in ID order.
cursoris optional (default0-0) and the answer has the next one, as a scan.min_idle_msdoes not count;just_idsworks.duewithidsis400 invalid_request.
Errors
| Code | HTTP | Meaning |
|---|---|---|
not_a_stream | 400 | A _streams route, or "stream": true, on a collection that is not a stream. |
stream_append_only | 400 | POST /{s}/_doc, _bulk, _update, _update_by_query, _delete_by_query, _expire or _persist on a stream. An entry is added only by an append, never changes, and is deleted with DELETE /{s}/_doc/{id} or a trim. |
invalid_stream_id | 400 | An ID that is not <ms>-<seq>, a special value where it is not allowed, or 0-0 in an append. |
stream_id_too_small | 400 | An append with an ID that is not greater than the last one, or any append after the highest ID. |
unknown_group | 404 | The group does not exist. |
group_exists | 409 | A group create with a name that exists. |
Access
append, trim, every group change, read (it moves the PEL), ack, nack and claim are writes. GET /{s}/_streams, the GET of groups and consumers, pending and POST /{s}/_streams/read are reads.