Architecture

StreamStack primarily has a write and read path, joined by a feed. Producers append to an open stream on the feed. And while the stream is open, its records are ongoing unsealed history. When the producer seals the stream, it becomes a commit in the sealed feed history. Consumers read from both as one or by commit.

Write Path

A producer opens a stream on a feed and appends records in batches. Each batch is sequenced, written to a write-ahead log, and acknowledged once durable. While the stream is open, a background flush copies log data into packs, a manifest, and optionally a key index.

The producer seals the stream when that ingest boundary should close. Seal writes a commit into the feed history and trims the log for that stream. What belongs in one stream and when to seal are producer decisions.

Read Path

Consumers read sealed commits and ongoing unsealed records on the feed. Feed-scoped reads advance by cursor and can include unsealed data from an open stream. Stream-scoped reads address one stream by sequence, sealed or open. A read returns records from the chosen range with an updated cursor for the next page.

Reads can follow the feed continuously or target a single commit or stream. RAW, deduplicated, and latest-per-key modes apply at read time. A subscription is a named cursor in metadata. Polling reads from the saved position to the current head.