AI Agent Evals: The 4 Layers Most Teams Skip
Most teams evaluate AI agents by vibes. Here are the four layers of evals you actually need to ship agents with confidence.
I walk through the eval stack I use on real agent projects — from unit-level prompt checks up through end-to-end trajectory scoring — and explain where each layer catches different classes of failure. If you’re building agents for production and wondering why regressions keep slipping through, this is the framework to borrow.
Building an AI agent?
I help teams design and ship agentic systems — from architecture to production.
See how I can help
I Gave My AI Agent Access to My Second Brain
What happens when you wire an AI agent directly into your Obsidian vault? Here's the setup I use to turn notes into real leverage.
Build Your Own AI Agent from Scratch (Mastra + TypeScript)
Learn to build your own AI agent that actually does work for you, not just answers questions.
How AI Agents Search Their Memory
In my last video, I covered how AI agents store memory. But storing it is only half the problem. In this one, we dig into how agents retrieve the right memory
2. Building CreatorSignal: AI Agent Teams Build Features in Parallel (LIVE)
I used Claude Code's new Agent Teams to have multiple AI agents build features for my Rails app in parallel -- live and unedited.
Get new videos and posts by email
Weekly videos on AI engineering, plus deeper dives in the newsletter.
Occasional emails, no fluff.