Menu

Post image 1
Post image 2
1 / 2
0

A senior engineer spotted a bug in my pipeline. I fixed it the same day. Here is what I learned.

DEV Community·Aftab Bashir·23 days ago
#eJEAflQU
#dotnet#kafka#order#outbox#article#video
Reading 0:00
15s threshold

A few weeks ago I published an article about an event-driven order pipeline I built in .NET with Kafka and Azure Service Bus. Someone left a comment that stopped me in my tracks. Andrew Tan wrote: "One thing I'd watch: you now have two sources of truth in flight, PostgreSQL and Kafka. If the API crashes after writing to Postgres but before publishing, you've got an order that never gets processed. Have you considered using an outbox pattern or transactional writes to close that gap?" He was right. I had not thought about it properly. The gap he spotted Here is what the original code did when a new order came in: Save the order to PostgreSQL Publish an event to Kafka Two separate operations. No transaction between them. If the API crashed, ran out of memory, got killed by Kubernetes, or just had a bad moment between step 1 and step 2, the order would exist in the database with a Pending status and never move forward. Nobody would know. No error. No alert. The order would just sit there.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More