NATS & JetStream Distributed Messaging Architecture Studio
Design high-performance distributed streaming architectures on NATS 2.10+. Dissect subject routing wildcards, calculate JetStream storage footprints, model consumer backpressure, and synthesize production code in browser memory.
.).
* matches exactly one token at that specific position.
> matches one or more trailing tokens and must only appear at the end of the subject.
orders.*, orders.>, events.user.*.login, telemetry.>
Subject Matcher Verification Matrix
Recommended JetStream Stream JSON Configuration
Synthesized Consumer Definition (JSON)
Nats-Msg-Id header prompts the JetStream storage engine to record an 8-byte hash of the message ID in an uncompressed memory window. If the same ID arrives within duplicate_window, the message is acknowledged to the sender (preventing retry storms) but dropped from the stream log.
Idempotent Publisher Example with Backoff Retry
Production Architecture Showdowns
- Single 25MB Go binary with zero external dependencies (no JVM, no ZooKeeper, no KRaft).
- Sub-millisecond P99 latency (< 250 microseconds on NVMe).
- Subject-based routing with wildcard hierarchies (tens of thousands of subjects per stream).
- Linear scaling with RAFT consensus per stream (independent leader per stream).
- Dynamic consumer groups without stream partitioning bottlenecks.
- Heavy JVM memory footprint (requires 16GB-32GB RAM + OS page cache per broker).
- Topic partition bound: maximum consumer parallelism is strictly bounded by partition count.
- High-latency coordination (P99 latencies 5-20ms under heavy consumer rebalances).
- Excels at massive petabyte-scale batch log storage and long-term analytical tiering.
- Rigid topic hierarchies: no dynamic runtime subject routing or wildcards.
- Client pulls exactly what it can process (e.g.,
fetch(50)). - Zero risk of slow-consumer disconnects or memory exhaustion.
- Graceful auto-scaling: new Kubernetes pods pull from the same durable stream immediately.
- Batching minimizes TCP roundtrips across high-latency networks.
- Server pushes messages unrequested to a delivery subject.
- If consumer stalls or suffers GC pause, server disconnects client with
ErrSlowConsumer. - Requires complex client-side buffering and flow control heartbeats.
- Only suitable for low-frequency firehose metrics or ephemeral UI push notifications.
- Limits: Keeps messages until age/bytes limit is reached. Standard append-only replay log.
- Interest: Deletes messages automatically once all subscribed consumer groups have acknowledged them.
- WorkQueue: Deletes a message the moment the FIRST worker acks it. True competing-consumer task distribution.
- Core NATS: In-memory, at-most-once pub/sub. Millions of msgs/sec. If no subscriber is listening, message is dropped instantly.
- JetStream: Built-in RAFT-replicated persistent engine providing at-least-once, exactly-once deduplication, and historic replay.
5 Fatal NATS & JetStream Engineering Traps
Engineers coming from RabbitMQ or MQTT frequently attempt to write subject filters like orders.>.processed. In NATS, > represents a multi-token wildcard and MUST be the final token in the subject string. Writing orders.>.processed is an illegal subject error that will cause stream creation or subscription registration to fail immediately. To match a single token in the middle, use orders.*.processed.
When using Push consumers in microservices, the NATS server pushes messages directly across the TCP socket. If a downstream consumer experiences database lock contention or CPU throttling, its client socket buffer fills up. When max_pending_msgs (default 65,536) is exceeded, the server aggressively terminates the TCP connection. When the pod reconnects, the entire backlog is pushed again, creating an infinite crash loop. Always use Pull Consumers in production backend services.
In a WorkQueue stream, unacknowledged messages are automatically redelivered to available workers when AckWait expires. If a slow task takes 35 seconds to process, but AckWait is set to the default 30 seconds, NATS will redeliver that same task to a second worker while the first worker is still finishing it. Both workers will process the task simultaneously and compete to ack. Always ensure AckWait generously exceeds your worst-case processing duration, or have workers send periodic msg.in_progress() heartbeats.
When configuring stream capacity limits (max_bytes or max_msgs), NATS offers two discard policies: DiscardOld (FIFO drop oldest message) and DiscardNew (reject incoming publishes). If you configure DiscardNew and a burst of traffic fills the stream, NATS returns error code 10053. If your publisher does not catch this error and route to a fallback DLQ, critical business transactions are dropped permanently.
NATS JetStream deduplication stores hashes of Nats-Msg-Id in uncompressed memory for the duration of duplicate_window. Setting a 7-day duplicate window on a stream ingesting 10,000 msgs/sec forces the NATS server to maintain an in-memory hash table of 6 billion entries, consuming over 45 gigabytes of non-reclaimable server RAM per cluster node. Limit your duplicate_window to between 2 minutes and 10 minutes, which is more than sufficient to catch network retry bursts.