Menu

Post image 1
Post image 2
1 / 2
0

The Accordion Pattern: Why I stopped writing one fat LLM prompt

DEV Community·Hideki Mori·about 1 month ago
#OsJ4o0jR
#why#llm#ai#stage#prompt#segment
Reading 0:00
15s threshold

Most structured-extraction tutorials look the same. Take a document, write one big prompt that says "extract A, B, C, D, E, F", get JSON back. Done. This works on short inputs. It quietly breaks on long ones. After running this in production for a while, I stopped doing it. Here's what I switched to and why. The fat prompt problem Say you have a 50-page report and you want a structured summary out of it. The natural first move is something like: Extract: - title - sections (with headings) - purpose - mentioned services - acceptance criteria - ... Return JSON in this shape: { ... } Enter fullscreen mode Exit fullscreen mode You hand the whole document to the model. It returns JSON. It looks fine on the first try. Then you scale it up and three things happen: Quality drifts. The model "forgets" mid-document. Later sections are summarized worse than earlier ones, or fields go missing. One bad field poisons the whole call.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More