How to Test LLM-Powered Applications Effectively Testing a CRUD app is deterministic. You input X, you expect Y, you assert equality. Testing an LLM-powered application is different in a way that breaks most of your existing instincts. The model's output is probabilistic . The same prompt can return different phrasing across runs. "Correct" is often subjective. Traditional assertEqual doesn't work here. Here's how to think about testing LLM apps properly. The Three Layers of an LLM App Before writing a single test, map out what you're actually testing: [ User Input ] ↓ [ Prompt Construction ] ← Layer 1: Deterministic. Testable normally. ↓ [ LLM API Call ] ← Layer 2: Non-deterministic. Mock in unit tests. ↓ [ Output Parsing ] ← Layer 3: Deterministic. Testable normally. ↓ [ App Response ] Enter fullscreen mode Exit fullscreen mode Most bugs aren't in the LLM — they're in layers 1 and 3. Start there. Layer 1: Test Prompt Construction Your prompt builder is plain code. Test it like code.…