Menu

Pub sub explained: topics, partitions, replicas, and failover
📰
0

Pub sub explained: topics, partitions, replicas, and failover

Reddit r/webdev·u/CorrectHornet4939·about 1 month ago
#SZut1o1P
#once#partitions#leader#topics#replicas#photo
Reading 0:00
15s threshold

Pub sub explained: topics, partitions, replicas, and failover Most web apps start out with everything talking to the database directly, then hit a wall once you need background jobs, email sending, webhooks, analytics or anything async. That is usually [when pub sub shows up](https://www.youtube.com/watch?v=5vwxu5Q3DHU). Covers: Producer, broker, consumer and how the three roles fit together Inside the broker: a small control plane and a huge append only log Why a log beats a queue: offsets, replay, fan out Topics split into N partitions for throughput Hash the key, modulo N, pick the partition, with a worked example Replication with 3 replicas across 3 zones, one leader and two followers Leader failover when the leader dies Consumer groups and how partitions get split across consumers At most once vs at least once vs exactly once delivery The part that trips up most web devs moving from a job queue to something like Kafka is that the log model is fundamentally different.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More