Why one metric is almost never enough

Neural network quality is not a property of the weights alone. It is a property of the chain task → error definition → sample → decision threshold → operating environment. One metric collapses that chain into a scalar and inevitably drops dimensions that matter for the product.

On a test set you measure generalization under a fixed distribution and fixed labels. In the product you get input drift, a different class balance, operator edits, timeouts, scanner degradation, and policy changes. On individual ML pipeline stages an error can be locally small and globally fatal: the detector found the field, OCR mistyped one character in the account number — for the business the document is entirely wrong.

Metric choice depends on the task not as fashion but as a contract. In binary classification the contract often reads “do not miss the dangerous class” or “do not bury the operator in false alarms.” In computer vision — “boxes are accurate enough at the required IoU.” In OCR — “the line or field matched entirely.” In search — “the needed document is in the top-K.” In LLM and RAG — “the answer is correct, complete, grounded in context, and not invented.” These are different oracles; mixing them into “quality 0.93” yields a number with no explanation for a rollback.

A practical antifraud example. Model A reports Accuracy 99.2%, model B — 97.5%. If fraudulent operations are 0.5%, model A may be a constant “legitimate,” while model B is a real system with 80% Recall on the rare class. Without Precision/Recall and without FP/FN costs, the comparison is meaningless. The same plot appears in line defect control: “everything is fine” almost always looks strong on Accuracy and is useless for product quality.

Train, validation, test, and what counts as an error

Why three splits

The training set is for fitting parameters. Validation is for choosing architecture, hyperparameters, threshold, and stopping time without peeking at the final exam. The test set is for evaluating generalization once, after all decisions are locked. Judging “model quality” only on train measures memory, not work.

Data leakage destroys metrics more quietly than overt overfitting. The same document in train and test after different augmentations, frames from one video in different splits, rows from one table after random shuffling on a temporal task, features computed with the target label — all of this makes the report optimistic. Build the test set to imitate future input: by time, device, source, client — not only by a random percentage of rows.

A separate benchmark that you do not touch just to “raise the number” is a mandatory artifact. How to prepare and version data for training and evaluation is covered in dataset engineering.

What exactly we count as an error

Before choosing a formula, agree on the unit of accounting. Object recognized correctly — one type of success. Object found but wrong class — another error (often FP for one class and FN for another). Object missed — FN. Extra object — FP. In OCR, an error in one character can be mild by CER and a full catastrophe by Exact Match for a “amount” or “passport number” field.

In detection, a “match” is usually defined by an IoU threshold between predicted and reference boxes. In segmentation — by mask overlap. In ranking — by a relevant item landing in the first K positions. In RAG — by the needed fragment being in context and used correctly in the answer. While the error definition floats, metrics from different teams cannot be compared.

Classification: Accuracy, Precision, Recall, F1, and the confusion matrix

On a payment review or a defect line, the person on shift does not see “model quality.” They see a queue: how many honest cases were stopped, and how many dangerous ones slipped through. Those are different prices, and a single average of correct answers blends them into one number.

Until those prices are named, comparing models by one percentage is empty. The four formulas below score the same set of answers from different sides. Whichever error a formula hides is the error you must not treat as the only reason to ship.

Accuracy

If 99% of transactions are legitimate, a model that answers “legitimate” to every one of them is almost never wrong on average. The fraud still passes in full, and the report stays green.

Accuracy = (TP + TN) / (TP + TN + FP + FN)

The share of correct answers is convenient when classes are relatively balanced, error costs are symmetric, and the question is literally “how often is the model right on average.” On a rare positive class Accuracy misleads: the dominant class masks failure. If 99% of transactions are legitimate, the constant “legitimate” scores 99% Accuracy and 0% Recall on fraud.

Accuracy decides whether one average percentage is enough to ship. When the dangerous class is rare, it is not, and misses are read on their own.

Precision

The operator does not review every answer, only the cases the model raised. If most of those cards are honest customers, the shift stops trusting the queue, even when a rare dangerous case is sometimes caught.

Precision = TP / (TP + FP)

Precision answers: among all model triggers, what share is true. High cost of false alarms is the classic Precision case: blocking an honest customer, stopping the line for a false defect, escalating a safe request. Low Precision kills operator trust faster than mediocre Accuracy.

Precision decides whether a trigger will flood a live shift. If a false alarm is expensive, do not ship an automatic block on a low share of true alarms.

Recall

On a scan or on the line, the expensive case is the one the model stayed silent about. Normal frames vastly outnumber it, so an average percentage does not show that silence.

Recall = TP / (TP + FN)

Recall answers: what share of real positive cases the model found. Where a miss costs more than a false alarm — oncology on a scan, a secret leak, a critical defect, fraud — Recall is often more important than Accuracy. Raising Recall usually means lowering the threshold and accepting more FP; that is not “model degradation,” it is a shift of the operating point.

Recall decides how many dangerous cases you can afford to miss. If a miss costs more than a false alarm, lower the threshold even when the average on the report falls.

F1-score

One model almost never raises a false alarm and misses nine dangerous cases out of ten — the same skew where Precision sits near 1.0 and Recall near 0.1. Another catches almost everything and buries the shift. Neither the share of true alarms nor the share of cases found, taken alone, says which model is closer to a working compromise.

F1 = 2 × Precision × Recall / (Precision + Recall)

F1 is the harmonic mean of Precision and Recall. It penalizes strong skew: Precision 1.0 with Recall 0.1 yields a low F1. Macro F1 averages F1 across classes equally; Micro — through pooled TP/FP/FN; Weighted — with class support. Macro helps when rare classes matter on their own; Micro is closer to overall error share; Weighted can hide the rare class again.

F1 decides whether a failure on one side is hiding behind a perfect score on the other. When rare classes matter on their own, read the average across classes, not the overall share of errors.

Confusion matrix

A single F1 is already computed, and the model still swaps neighboring decision classes on live payments. Without a table of who is confused with whom, you cannot tell whether to collect more “approved / rejected” examples or noise in other labels.

The confusion matrix shows which classes get confused. A single F1 number will not tell you that the model systematically swaps “3” for “8” or “approved” for “rejected.” For multiclass, the matrix and per-class Precision/Recall are often more useful than any average: you see where to invest data and augmentations.

The confusion matrix decides where labeling and extra examples go: into the pair of classes the model mixes up, not into the overall average.

Class imbalance, ROC-AUC, and PR-AUC

On a defect line almost every item is good. A model that never calls the inspector looks strong on the average share of correct answers and is useless for product quality.

Class imbalance is normal in industrial tasks: defects, fraud, spam, rare diagnoses, rare document types. Balanced Accuracy averages Recall across classes and does not let the dominant class buy a high score. Macro F1 does a similar service for F1. For ranking probabilistic models you also look at curves.

ROC curve and ROC-AUC

The threshold is not chosen yet. You need to see how the share of fraud found and the share of false alarms move together as the cutoff shifts. The area under that curve compares models before anyone locks an operating point.

ROC is built from TPR (the same as Recall) and FPR as the threshold changes. The area under the curve (ROC-AUC) estimates the model’s ability to rank positive examples above negative ones, averaged over thresholds. That is useful for comparing scorers. Weak spot: with a very rare positive class, FPR can look small even with a large absolute number of false alarms, because negatives are huge. ROC-AUC then looks “green” while the operator drowns in FP.

ROC-AUC decides which model ranks dangerous cases above safe ones. You still cannot ship on that area alone.

Precision-Recall and PR-AUC

Every false alarm in the queue directly spoils the share of true triggers. When dangerous operations are few, that shows up more clearly than the share of false alarms in a sea of ordinary cases.

The PR curve looks at Precision and Recall across thresholds. PR-AUC is more sensitive to quality on the rare class: every FP hits Precision directly. For fraud, defects, and rare events, PR-AUC and the operating point on the PR curve are usually more informative than ROC-AUC.

PR-AUC and a point on the curve choose the working threshold for a rare event, rather than a “green” area that tolerates a crowd of false alarms.

Why AUC is not enough to ship

The same AUC admits different thresholds with different Precision/Recall. The product needs an operating point: chosen threshold, expected FP/FN per day, review queue, SLA. Comparing models by AUC alone compares ranking potential, not system behavior after the “fired / did not fire” decision.

Regression: MAE, MSE, RMSE, MAPE, and R²

Procurement does not ask how clever the model is. They ask how far it misses on a deadline and a price. The share of correct answers is not even defined here: the target is a number, not a class.

Until the miss is back in the same units as the loss, the report compares benches, not orders. The formulas below look at one shift with different strictness toward large errors.

A delivery date slipped, and the model named a different number of days. The planner needs that shift in days, not its square. A short slip and a long one must both be readable on the original scale.

MAE = mean(|y - ŷ|) — mean absolute error, easy to read in original units and relatively robust to outliers.

MAE decides whether a typical miss is acceptable in money, days, or degrees, without translating out of a squared error.

A large miss on an expensive order hurts more than a dozen small ones. If every error is averaged the same way, the tail of blow-ups hides behind a mass of nearly exact small lines.

MSE = mean((y - ŷ)²) — penalizes large misses more strongly; convenient mathematically, harder to interpret directly.

MSE decides whether training should punish rare large misses harder than frequent small ones. Do not put it alone in a business report.

Buyers compare models in days and money, not in squares of those units. The square root of the mean squared error puts the tail penalty back on the same scale as the mean absolute error.

RMSE = sqrt(MSE) — returns the error to the original scale while keeping sensitivity to tails.

RMSE decides which model is calmer on large orders when the candidates’ mean absolute misses are almost the same.

A percentage error sounds familiar to the business: “8% on average.” It falls apart when price or duration is near zero, and it punishes over- and under-prediction differently.

MAPE computes mean percentage error. It is clear to business (“8% on average”) but breaks near zero and is asymmetric to over- vs under-prediction. If the target can be zero or tiny, MAPE is a poor sole criterion.

Do not ship on MAPE when the target can be zero or tiny: the percentage then lies harder than a miss in the original units.

A high share of variance explained against a naive mean is easy to get on a convenient sample. On expensive orders and at the edges of the scale the same model can fail systematically, and the report will not shout about it.

R² shows the share of explained variance relative to a naive mean. High R² does not guarantee usefulness: the model may explain noise well on a convenient sample and systematically fail in a critical range — expensive orders or scale edges.

Do not accept a release on R²: a high share of explained spread does not replace a tolerance band or behavior on the expensive edge of the scale.

For product regression, average RMSE often matters less than error quantiles, the share of predictions within tolerance, and behavior on strata (region, customer segment, magnitude range). As in classification, an average without error structure deceives.

Computer vision: classification, detection, and segmentation

On a label line the model owes one hard tag: hazardous freight or an ordinary box. The average share of correct frames goes green easily, because ordinary boxes vastly outnumber the hazardous ones.

A detector can “find” an account-number field with a box that also swallows the next column. For text recognition that crop is already someone else’s field, and the document as a whole is wrong. Until you know whether the box or the mask matches the annotation, a percentage of “objects found” does not say whether the next step can ship.

Image classification

A catalog has thousands of easy classes and one expensive one. Until errors are cut by class, optimization flows into frequent frames, and “flammable” or “personal data in the image” stays a hole in the average.

The base layer is the same: Accuracy, Precision, Recall, F1, confusion matrix, per-class metrics. On large sets with thousands of classes you also use Top-1 and Top-5 Accuracy: whether the true class landed in the first or top-five hypotheses. Top-5 is useful for research comparison and useless if the product needs one hard label without a candidate list.

An average across classes easily hides failure on a rare but expensive class (“flammable,” “personal data in the image,” “critical defect”). Publish per-class Recall/Precision alongside the average — otherwise optimization will flow into frequent easy classes.

Object detection and IoU

The detector marked the field, but the box also took the neighboring column. To a person the number was “found.” For text recognition the crop is already wrong: extra characters land in the line. Until you measure how the box overlaps the annotation, a high rate of found fields does not say the document is usable.

IoU = Area(Intersection) / Area(Union)

IoU measures overlap between predicted and reference boxes. The IoU threshold (often 0.5, sometimes higher) decides whether a detection counts as a true match. At a low threshold, a model with “approximate” frames looks strong; at a high one, precise geometry is required — important for OCR crops or robotics.

IoU decides whether the box is accurate enough for the next step. A low overlap threshold lets coarse boxes through; a high one demands geometry fit for a crop or for a robot motion.

TP/FP/FN in detection are counted by box matching, not by whole-image pixels. One extra box — FP; a missed object — FN; a box with the right class but weak IoU — usually FP plus FN relative to the reference.

The model finds the fields, but the boxes are coarse: on a lenient overlap threshold the report looks strong, on a strict one it does not. One percentage of “detection counted” is not enough to decide whether the geometry is fit for a crop.

Average Precision (AP) integrates Precision-Recall for a class; mAP averages AP across classes. [email protected] counts a match at IoU ≥ 0.5. [email protected]:0.95 averages over several IoU thresholds and is much stricter about box quality. A model can be strong on [email protected] and noticeably weaker on [email protected]:0.95 — a signal of “finds, but localizes coarsely.”

The pair [email protected] and [email protected]:0.95 decides whether to ship a detector into a document pipeline. Strength only at the lenient threshold means the boxes are still too coarse for a crop.

Segmentation

The defect in the image is a thin scratch on a large clean field. If you count the share of correct pixels, the background yields an almost perfect percentage, and the scratch is absent from the report. A mask needs overlap of the region, not agreement with empty background.

Pixel Accuracy is dangerous again with small objects on a large background: “all background” yields a high pixel percentage. IoU / Jaccard and mean IoU (mIoU) look at mask overlap per class. Dice / F1 for masks:

Dice = 2|A ∩ B| / (|A| + |B|)

Dice is monotonically related to IoU but weights the intersection differently; in medicine and small structures Dice is often the more familiar training and reporting target. For thin boundaries and fine defects, look at class-wise Dice/IoU, not only the average pixel score.

Dice decides whether the mask is fit to measure a small structure. On thin defects, read it by class, not as an average percentage of pixels.

OCR and document recognition

OCR is a domain where naive “character accuracy” especially often lies to the business. A practical treatment of handwritten digits and the CNN/TrOCR stack is in the note on digit OCR.

Character accuracy and CER

Accounting checks the number that will go into a payment, not the share of guessed glyphs. A report can show that most characters matched and still hide one swapped digit in a long identifier.

Character accuracy is the share of correctly guessed characters after alignment. Example from a report: 158 / 223 = 70.9%. Useful for model diagnosis, weak as the sole product KPI.

CER = (S + D + I) / N, where S is substitutions, D deletions, I insertions, N the number of characters in the reference. CER accounts for edit distance and is usually more informative than a “raw” share of matched positions without careful alignment. Lower CER is better; compare CER only under the same normalization definition (spaces, case, Unicode).

CER decides which model edits the reference less under the same normalization. It does not decide whether a field can go to payment without a check.

WER and Exact Match

A typo inside a word in a letter is often still readable. The same single error in an account number sends the payment to the wrong place. “Wrong by a word” and “the whole string matched” are different acceptances, and for codes the second one is the one that matters.

WER = (S + D + I) / N at the word level. WER is coarser toward local typos inside a word and closer to phrase readability. For numbers, codes, and amounts, WER is less natural than CER and Exact Match.

WER decides whether a phrase is readable. For a number, a code, or an amount, ship on a full string match, not on a word-level error.

Exact Match Accuracy requires a full string match. A vignette worth remembering for anyone shipping OCR:

• reference: 48291037;
• prediction: 48291057 (one-character error);
• character accuracy ≈ 87.5–90% depending on length and alignment;
• Exact Match = 0%.

If the field is an account number, “almost right” equals “wrong.” For forms with several fields, introduce field-level exact match and document-level exact match: the share of documents where all critical fields are correct at once.

Structured OCR and geometry

In a document pipeline, error can sit in the text, field binding, box coordinates, table parsing, or postprocessing. Count field accuracy, zone detection quality, robustness to empty cells and damaged fragments separately. End-to-end document accuracy is the main business oracle; CER on crops is the diagnostic layer.

Example report for a numeric recognition model on a fixed benchmark:

Read it this way: characters are almost right on average, but on long and hard inputs whole strings fail more often — that is what accounting will see, not CER.

NLP, LLM, and RAG: what to measure separately

The assistant answers smoothly and off the contract: the needed paragraph never surfaced in search, and the model filled in a return period from generic wording. One figure called “answer quality” mixes a search failure with a generation failure, and the team will fix the wrong stage.

Keep three layers apart: a label on the text, properties of the answer itself, and the join of search with generation. Blend them into an average and you get a report that cannot say whether to change the index or the model.

Classical NLP

Support wants to know whether a message is a complaint or a question. The cost of a false escalation and the cost of a missed complaint are the same contract as in tabular classification. The average share of correct letters stays quiet when a rare request type drowns in a mass of calm ones.

A separate break appears when you pull an object out of the text, not a label for the whole letter. The model can tag almost every fragment of a name and still cut the organization on the wrong boundary. For a counterparty card that is already a different object.

Text classification uses the same Accuracy/Precision/Recall/F1 set. For sequence labeling (NER and similar), distinguish token-level F1 and entity-level F1: an entity with “almost the same span” may be right by tokens and wrong as a business object. Entity-level is usually closer to the product.

A translation can look close to the sample word for word and still flip a negation: a ban becomes a permission. Overlap with the reference does not catch that, and on a contract that overlap is more dangerous than a low score on a neutral sentence.

In machine translation, BLEU, chrF, and COMET are automatic and scalable but limited: they do not replace human evaluation on critical domains and poorly catch semantic negation. In generation, ROUGE/BLEU/BERTScore measure similarity to a reference, not truth; perplexity is the language model’s confidence in the text, not answer usefulness. LLM-as-a-Judge scales labeling and needs calibration. A detailed test map is in LLM quality testing; why surface metrics miss contradiction with a reference is in the MATCHA breakdown.

Do not ship contract text on these automatic scores. They decide where to call a person, not whether a negation in the document survived.

LLM answer quality

For a product assistant it helps to decompose properties: correctness, relevance, completeness, faithfulness / groundedness (reliance on sources, absence of invention). One average “answer quality” again mixes different failure modes.

RAG: retrieval and generation

A user asks for the return period on a specific contract. If the needed paragraph never lands in the context, a strong model still answers with confidence — and not from the document. If the paragraph is there and the answer swaps the period, generation is at fault, and replacing the index changes nothing.

Typical pipeline: Question → Retrieval → Context → LLM → Answer. Stage errors are independent. Weak search with a strong model yields a confident answer on the wrong fragments. Strong search with weak generation — correct context and a distorted conclusion.

That chain decides where to stop the investigation. An empty search and a swapped fact with the context present are different jobs.

When the index is silent, more questions arrive whose context lacks the needed fragment. When the paragraph is found and the answer rewrites the period, reliance on sources falls while search is alive. One end-to-end score does not show these two failures apart.

Retrieval metrics: Context Precision, Context Recall, Retrieval Recall, Hit Rate, MRR. Answer metrics: Answer Relevance, Faithfulness, groundedness, exact fact/field match where possible. End-to-end success is needed, but without decomposition you will swap the LLM when the index is guilty, or vice versa. Building references is covered in the RAG golden set; chain engineering — in production RAG.

This split decides whether to change the index or the model. An end-to-end failure with no decomposition treats the wrong stage.

Vignette. Query: “what is the return period under contract 14-A?” Retrieval does not place the needed paragraph in the top context — Context Recall drops; generation answers from “general knowledge,” honestly or not. Another case: the needed paragraph is in context, the model changes “14 days” to “30” — Faithfulness drops while retrieval is alive. Treating both failures with one fine-tune “on answers” is an expensive confusion of causes.

Ranking: Precision@K, MRR, and NDCG

Someone looking for a policy reads the first rows of the result list. A document buried far down does not exist for them, even if it sits in the archive and formally matches the query. Ordinary accuracy over the whole corpus is the wrong question: unsuitable documents are almost endless.

Until you agree how many positions a person actually reads, and what counts as a useful hit, there is nothing to compare search models on. The names below look at the head of the list, not at the archive as a whole.

Precision@K — share of relevant items among the first K. Recall@K — what share of all relevant items landed in the top-K. Hit Rate@K — whether at least one relevant item is in the top-K. MRR (Mean Reciprocal Rank) looks at the position of the first relevant item: higher is better. MAP averages precision over relevant finds. NDCG accounts for both position and graded usefulness: a document that is “perfectly relevant” in first place is worth more than in tenth.

These numbers decide which search to ship in the interface. A gain deep in a list nobody scrolls does not count for the product.

Why this is not ordinary classification: negative documents are almost infinite, and you do not need “accuracy over the whole corpus.” You need usefulness of the head of the list. Comparing ranking by Accuracy on a random subsample of pairs is a common methodological mistake.

Calibration, decision threshold, and robustness

The model writes “0.9” on an application, and a rule accepts it on its own. If success among such applications is clearly rarer than the figure promises, the automation is lying, even when the model separates classes well. Confidence and the decision are different things, and the average share of correct answers does not tell them apart.

The next three checks answer different shipping questions: whether the number can be trusted as a frequency, where to place the cutoff, and whether the report is green only on convenient examples.

Confidence calibration

A model can separate classes well and still speak probability poorly. A calibration curve compares predicted probability with observed frequency. Brier Score and Expected Calibration Error (ECE) quantify the mismatch. High Accuracy with poor calibration is dangerous where score decides “auto-accept / send to a human / reject”: “0.9” should mean roughly 90% success frequency, or automation lies.

Threshold — not “magic 0.5”

The decision threshold turns a score into an action. Moving the threshold pushes Precision and Recall in opposite directions. Choose under a business constraint: maximum Recall at Precision ≥ X, minimum FP at Recall ≥ Y, cost of the review queue. There is no universal 0.5: it only makes sense if the score is calibrated and classes/costs are symmetric — a rare case.

Robustness and OOD

Average quality on a clean test does not describe behavior under noise, blur, lighting change, rotation, JPEG compression, a cropped field, another capture device, or another text domain. Stress sets compare degradation relative to a baseline. In-distribution test, out-of-distribution test, cross-domain / cross-device / cross-source evaluation show whether you bought a metric with a random split inside one source. Random train/test on frames from one camera is often optimistic relative to a new camera on the shop floor.

Fairness across groups

Overall F1 can hide failure on a subgroup: language, region, scanner type, demographic stratum, traffic source. Compute per-group Accuracy/F1/Recall and separate FPR/FNR. Separate validation by data source is minimal hygiene even when a formal fairness audit is not required by a regulator.

Production metrics: latency, resources, and cost

A nightly document batch has to finish before the morning shift, and a chat reply has to land inside the pause a person still treats as instant. A model with a better share of exact fields that misses both of those windows is worse for the product than a simpler one.

Until the comparison includes time, queue, and the price of one document, you are comparing benches, not a service.

Model quality in production is not only ML numbers. Latency: mean, P50, P95, P99. The P99 tail often matters more than the mean: it breaks UX and decides whether the answer fits an interactive scenario. Throughput — requests/sec, images/sec, documents/hour — decides whether a nightly batch scales. CPU/GPU/RAM/VRAM and storage utilization limit deploy density. Cost — cost per inference, per document, per 1000 predictions — decides whether the “better” model survives product economics.

A model with +1% Exact Match at triple cost and doubled P95 can be a product regression. Comparing candidates without latency and cost compares lab benches, not services.

End-to-end quality versus component metrics

On a stack of invoices the detector almost always finds the field, recognition almost always guesses the characters, and a rule almost always strips a stray space. Accounting still sends back a noticeable share of documents: one wrong critical field makes the document wrong as a whole, and the local percentages do not add up to that.

Until there is a figure for “the document is entirely usable,” the team fixes the stage with the worst report and can miss the field where the business actually loses money.

A document pipeline is typical:

Image → Detection → Crop → OCR → Postprocessing → Structured JSON

That chain decides where to look when you fix, and whether you can ship. Fix components by their own metrics; accept the release by the share of documents that are correct as a whole.

Suppose detection mAP = 95%, OCR character accuracy = 97%, postprocessing “almost always” fixes spaces. The document as a whole is correct in only 82% of cases: stage errors correlate weakly and multiply on critical fields. Component metrics are needed to know where to fix. An end-to-end metric is needed to know whether you can ship.

The same principle applies to RAG and agents: search hit rate, correct tool call, and correct final answer are different layers. A management frame for end-to-end evaluation is in evaluating enterprise AI; barrier automation is in the ai-eval-harness.

Evaluation pipeline and error analysis

The number in the report is already computed, and a week later the run cannot be repeated: the test set was edited, the model’s answers were not saved, the threshold in the notebook is different. Without a frozen set and saved answers, “it got better” cannot be told apart from a different sample.

The investigation does not start with a new architecture. It starts with where the system is actually wrong. The average only says how many errors there are; which classes get mixed up, and on which sources, shows up in the saved predictions.

A working evaluation pipeline looks like this:

Dataset → Validation split / fixed benchmark → Model inference → Predictions → Metrics → Error analysis → Report → Regression tests

That chain decides whether a run can be compared with the last one. No frozen set and no error analysis means no shipping decision.

Automation includes an evaluation script, saving predictions and metrics, run comparison, experiment tracking (MLflow, Weights & Biases, or your own loop). Version, freeze, and do not edit the evaluation dataset just to improve the report. Keep predictions: without them you cannot run error analysis after the number is already computed.

A metric shows how many errors. Next, sort errors by confidence, break down FP and FN, frequent confusion pairs, slices by class, source, and difficulty. For OCR, look separately at handwritten digits, pairs 1/7, 3/8, 5/6, noise, tilt, empty cells, damaged fields. For RAG — queries with empty retrieval, partial context, conflicting fragments, required refusal.

After metrics, a model card helps: dataset and its version, model and preprocessing version, main metrics and threshold, limitations, known failure cases, latency, hardware, evaluation pipeline version, run date. That makes the result reproducible a month later, not “someone’s notebook.”

How to choose metrics and compare models

Two teams each bring “plus one percent” and argue who is better. Until you say one percent of what — share of correct answers, missed defects, or a field correct as a whole — the argument is empty. Task contract first, then a row of the table, then a comparison on the same set.

The matrix below is a lookup for which instrument usually matches the task. It does not replace a threshold, the cost of error, or a check that both models computed the same thing.

Task → metrics matrix

Why 97% can be worse than 90%

Scenarios from practice. Imbalance: 97% Accuracy of a constant is worse than 90% Accuracy of a model with high Recall on defects. FP/FN cost: a model with a lower average but half as many expensive misses is better. Exact Match: 97% character accuracy is worse than 90% Exact Match if the business counts whole fields. Critical class: overall F1 is higher while Recall on the “dangerous” class is lower. Production distribution: the lab test is cleaner than the field. Poor calibration: high Accuracy but auto-thresholds lie. High latency: the “best” model misses the SLA.

Core idea: a good metric is not necessarily a high number. A good metric measures what matters to the system.

Correctly comparing two models

Do not stop at Model A = 95% versus Model B = 96%. Check the same test set, same preprocessing, same threshold, same metrics, comparable confidence intervals, latency, memory, cost, robustness, and quality by class/stratum. Bootstrap and confidence intervals remind you: on a small test, a one-point difference often sits inside noise. Repeated evaluation and sample size are part of honesty, not bureaucracy.

Common mistakes and checklist

The model “suddenly” got better after test documents landed in training, or after awkward examples were pulled out of the test. The report is green; the field is not. Most of those stories are about how the number was computed, not only about the architecture.

Before the list, check your own last report: did you score only the training set, did you move the test after every run, did you look only at the average? If yes, the checklist at the end of the section is cheaper than an incident review.

Typical mistakes in neural network evaluation:

1. Evaluating on the train dataset.
2. Data leakage between splits.
3. Test set too small.
4. Changing the test set after every experiment.
5. Using only Accuracy.
6. Ignoring class imbalance.
7. Ignoring the threshold.
8. Evaluating only the average metric without per-class / per-slice.
9. No error analysis.
10. Ignoring production latency.
11. Ignoring inference cost.
12. Comparing models on different data.
13. No fixed benchmark.

Universal checklist before publishing a model

• Train/validation/test are split correctly.
• No data leakage.
• Test set / benchmark is frozen and versioned.
• Task-specific metrics are chosen for the cost of error.
• Baseline metrics exist.
• Per-class / per-slice metrics exist.
• Class imbalance is accounted for (if present).
• Error analysis was performed.
• Robustness / OOD checked on realistic distortions.
• Calibration checked, threshold chosen.
• Latency measured (including tails).
• Resource consumption measured.
• Inference cost estimated.
• Results are reproducible with the same evaluation script.

Frequently asked questions

Further reading

• Language model quality testing — which LLM tests exist and what they actually measure.
• AI evaluation harness — how to turn metrics into CI, regression, and release criteria.
• Golden set for RAG evaluation — retrieval and answer references, not an exam of a “bare” model.
• Production RAG engineering — where in the chain the error arises that generation metrics do not see.
• Handwritten digits: CNN and TrOCR — a practical OCR loop and why Exact Match is stricter than character accuracy.
• How to evaluate enterprise AI — linking quality, process, and risk at the enterprise level.
• Dataset engineering for ML — how data and labeling set the ceiling of any metrics.

Conclusion

You cannot tell that a neural network “actually works” from one pretty validation number. You need an error contract, an honest split, task-fit metrics, a working threshold, analysis of failure structure, robustness checks, and production constraints. Accuracy, F1, ROC-AUC, IoU, mAP, CER, WER, NDCG, Faithfulness are instruments; meaning appears only when it is clear which claim about the system they confirm.

For complex pipelines, keep two layers of truth: component-level to fix, end-to-end to ship. Make the evaluation dataset and evaluation script first-class artifacts — on par with weights. Then the question “how good is the model?” is joined by “where does it fail?”, “why?”, and “can we trust the result in production?” — and that is engineering, not report optimism.

On the next model, write down which error costs money and which end-to-end number forbids a release, and only then pick the instrument.