Menu

Post image 1
Post image 2
1 / 2
0

I Built an llms.txt Generator, Showed It to the Creator of the Standard, and Had to Rewrite Everything

DEV Community·David Evdoshchenko·27 days ago
#NrvcnqcO
#llms#ai#webdev#pages#problem#everything
Reading 0:00
15s threshold

This is a follow-up to my previous post — I shipped v1, got feedback from the creator of the standard, and had to rethink everything. I'm not going to pretend I had a plan. I didn't. There's a new standard called llms.txt — a file you place in your site root to help AI agents navigate your content. Think of it as robots.txt for AI agents: instead of telling crawlers what to ignore, it tells agents how to understand your site. I saw other generators producing it. I built one too, shipped it, and thought I was done. Then I talked to Jeremy Howard — the creator of the standard — and discovered I had fundamentally misunderstood what llms.txt is supposed to be. And then things got complicated. Version 1: Copying What Everyone Else Was Doing The pattern every generator was producing: - [ Page Title ]( https://example.com/page ) : AI-generated summary. - [ Another Page ]( https://example.com/other ) : AI-generated summary. Enter fullscreen mode Exit fullscreen mode One URL → one summary. I did the same.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More