Menu

Post image 1
Post image 2
1 / 2
0

I Fine-Tuned a Compliance Judge and Beat the Stock Model by +29.6pp F1

DEV Community·Akhona Eland·18 days ago
#fhrEuLIG
Reading 0:00
15s threshold

I Fine-Tuned a Compliance Judge and Beat the Stock Model by +29.6pp F1 The problem: if your LLM-powered product touches personal information in South Africa, POPIA sits over it. The regulator doesn't ask "is your model good?" — they ask "can you demonstrate the output was validated against the clause, and can you show me the validation?" The uncomfortable answer most teams give today: "we call GPT-4 as a judge with a prompt that mentions POPIA." That's not a defence. It's non-deterministic, sends personal information cross-border, and produces no receipt. What I built instead: a local NLI cross-encoder fine-tuned on 7 POPIA clauses, released under Apache 2.0, shipped as a quantized ONNX model, scored and gated on every CI run.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More