Menu

📰
0

I built a hallucination detector for LLMs in 6 days as someone with zero MLOps experience. Honest take.

Reddit r/learnmachinelearning·u/SithEmperorX·about 1 month ago
#b3xQl5UC
Reading 0:00
15s threshold

I built a hallucination detector for LLMs in 6 days as someone with zero MLOps experience. Honest take. I'm a Data Science Master's student at RWTH Aachen. My university teaches theory and math but nothing about actually shipping ML systems. No Docker, no deployment, no HuggingFace, nothing. I wanted to fix that so I built a project with the goal of not just training a model but actually shipping it. I used Claude to guide me through the process and I'll be upfront about that. It took 6 days. The project: fine-tune Meta's Llama 3.2 3B to detect hallucinations in LLM responses. Given a question and an answer, it predicts TRUTHFUL or HALLUCINATED. Trained on TruthfulQA and HaluEval, 15,918 labeled pairs, using LoRA so only 0.14% of the 3 billion parameters were actually trained. Result: F1 score of 0.90. Honestly did not expect that. The hardest part was Docker. My first time using it and I kept thinking I'd broken something permanently or that my setup wasn't good enough.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More