Or, at least imho, the architectural mistake quietly inflating AI infrastructure budgets. Press enter or click to view image in full size “Image generated using OpenAI’s DALL·E.” There is a category of architectural decision that seems sensible at the time it is made and indefensible, perhaps even somewhat embarrassing, a few years later, once the constraints that produced it have lifted. Ok, I know you’re thinking this is all about agile but, that’s apparently a “ process ” rather than an architectural decision. Besides, and unfortunately, management hasn’t realised its embarrassment yet (even though pretty much all developers have), but let’s not get too sidetracked in the first few paragraphs. I’m actually talking about Retrieval Augmented Generation, or RAG to its friends, and it’s one of these. It was, in its time, the only way to make a useful conversational AI work over a real corpus when context windows were four thousand tokens, and it has been a reasonable patch ever since.…