AI tester over 11 weeks: 77 bugs from the agent, 16 from the human
A Claude Code session diary: 277 agent hours, 94 confirmed finds, where a reference pays off, and why an agent hour does not replace a human hour.
Stuzhuk Lab — Chemistry of Code
Chemistry of Code
← All posts
Tag
testing
All blog posts with this tag.
Language Model Quality Testing: What the Tests Are and What They Actually Measure
LLM quality checks: public task sets, arenas, model judges, safety, product regression, and online signals — what each test measures and what it cannot see.
Why cheap testing gets expensive: the economics of software failure cost
A feature is an investment that does not end at release. How to account for build, verification, support, and failure cost — and why you should test in proportion to blast radius, not coverage percentage.
AI Evaluation Harness in 2026: Golden Sets, Regression Gates, and Release Criteria That Survive Production
Build an AI evaluation harness that connects golden datasets, retrieval and generation metrics, CI regression gates, shadow traffic, release controls, observability, and LLM gateway FinOps.
Fable 5 and qa-mcp: reverse-engineering the 1C TestClient protocol with AI
Habr case study: from Vanessa Automation to native TestClient in Python — where Codex stalled for a week, Fable 5 mapped the protocol in a day for qa-mcp.