http://akjamie.github.io/tags/evaluation/ Recent content in Evaluation on Jamie's Blog Hugo en-us Sun, 23 Aug 2026 00:00:00 +0000 http://akjamie.github.io/post/2026-08-23-eval-architecture-and-red-teaming-guide/ Sun, 23 Aug 2026 00:00:00 +0000 http://akjamie.github.io/post/2026-08-23-eval-architecture-and-red-teaming-guide/ <h1 id="-building-eval-literacy--red-teaming-for-multi-agent-ai-code-review">🧠 Building Eval Literacy &amp; Red-Teaming for Multi-Agent AI Code Review</h1> <blockquote> <p><strong>How to upgrade an AI Agent from a &ldquo;working prototype&rdquo; to an empirical, production-grade review system with OWASP 2026 Agentic Security defenses.</strong></p> </blockquote> <h2 id="-bluf">🎯 BLUF</h2> <p>Multi-agent AI code reviewers are easy to demo and dangerous to trust blindly. This project closes that gap: a 22-case empirical benchmark (<strong>Recall 91.2%</strong>, <strong>Precision 94.7%</strong>, <strong>F1 0.93</strong>) plus OWASP ASI 2026 red-team defenses that cut a measured <strong>~80% prompt-injection attack success rate</strong> down to near-zero — turning an LLM wrapper into an auditable, CI-gated production system where every prompt change is verified against both <em>capability</em> (does it still catch real bugs?) and <em>security</em> (can it still be manipulated?) before it ships.</p>