If you work with RAG pipelines, agent tools, or LLM APIs, you’ve probably noticed something frustrating: sometimes the biggest cost in a prompt is not the data itself — it’s the repeated JSON structure wrapped around it. That is exactly the problem TOON tries to solve. TOON (Token-Oriented Object Notation) is a compact, human-readable encoding of the JSON data model designed for LLM prompts. It keeps the same logical structure as JSON, but reduces token overhead by declaring structure once and streaming the data in a denser format. In this post, we’ll break down the anatomy of the TOON format, explain where it fits in modern AI pipelines, and compare it with JSON, Arrow, and Parquet so you know when it is a smart choice — and when it is not. Why TOON matters ⚡ In many LLM workflows, especially RAG, the bottleneck is not storage size on disk. It is prompt size , token cost, and how much useful context you can fit into the model window.…