Menu

Post image 1
Post image 2
Post image 3
Post image 4
Post image 5
Post image 6
Post image 7
Post image 8
Post image 9
Post image 10
Post image 11
1 / 11
0

LLM Summarizers Skip the Identification Step | Towards Data Science

Towards Data Science·William Gieng·23 days ago
#MfWV8V5z
Reading 0:00
15s threshold

takes a five-minute exchange and returns eight clean sections. Decisions. Action items. Risks. Open questions. Each section reads like it was written by someone who was paying attention. Read the underlying transcript, though, and you find that two of those sections were inferred from a single ambiguous sentence, one was invented entirely, and three were pattern-matched from the model’s prior on what a meeting summary should contain. Confident, formatted, structurally indistinguishable from a summary of a meeting where those things actually happened. This is not a hallucination problem in the usual sense. The model is not making up a fact about the world. It is making up a fact about the meeting. And the failure mode is not visible in the output. It is just confident-sounding text that the reader cannot easily verify against the source. There is a name for this failure mode in another field, and it is older than language models. It is what happens when you do estimation without identification.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More