Menu

Post image 1
Post image 2
Post image 3
Post image 4
1 / 4
0

5-Minute AI Jobs and Closed Tabs — Why We Built Replay-Then-Tail SSE

DEV Community·Akshat Soni·about 1 month ago
#tEjtugdp
#how#backend#event#redis#session_id#queue
Reading 0:00
15s threshold

We had a feature in production where a single user request could run for five-plus minutes — fetch documents, chunk them, hit an LLM per chunk, synthesize a final answer. We did the obvious thing first: a FastAPI handler that ran the pipeline and streamed progress back to the browser over Server-Sent Events . It looked like this: # Naive in-handler version — what we wrote first, before learning the # work inside an SSE generator dies with the connection. The helpers # (fetch_text, chunk_text, summarize_chunk) are real — they live in # app/pipeline.py and survived the refactor. What got replaced is the # in-handler shape of the orchestrator below.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More