Contents
The note on a compact CRNN left a conflict open: one head should not jointly answer “is there a number?” and “which number?” On empty cells the recognizer paints a plausible thickness. This sequel from the same ai-vision pilot is about what already sits before OCR in the computer-vision pipeline, which cell kinds we distinguish, and why a tiny empty-versus-value classifier is still ahead even though the heuristic already runs.
This is not a finished guide and not a claim that rules closed the problem. The project is in development; the repository is private — the outline without raw forms is on the portfolio page. The sheet-level loop (vision model, confidence, operator report) is in why one model is not enough.
The cost of error is the same as on any ultrasonic thickness (UT) sheet: an empty cell turned into 8,0 looks like a confident measurement and can ride into the plant ERP if the operator misses the grey highlight. A mangled ditto sits next to that failure: a “same as above” column becomes a minus or a one, and later rows inherit a thickness nobody wrote.
Key takeaways
“Is there a measurement?” is not an OCR task. Kind first, reading second. Otherwise the net must say something even on a grid with no ink.
Five kinds in the pilot: empty, ditto, digit, text, crossed. That is a contract between the Python vision pass and the TypeScript result builder, not “five more models.”
We look for ink as blue pen and dark pencil, trying to ignore thin grid lines. The pixel-fraction threshold is small: a typical cell is almost empty, ink is sparse strokes.
A ditto (horizontal dash) is not read by OCR. We take the value up the same column. If there is no source — a review status, not an invented thickness.
Strike-through is its own kind: mask the stroke, then try to read the last valid number; otherwise the cell stays crossed, not “yet another measurement.”
The heuristic already cuts OCR calls, but it does not replace a trainable classifier. Grid, dust, and a neighbour stroke still impersonate a digit.
After OCR you still need deterministic shields: an unrealistic thickness above the cap (80 mm here) is dropped; a narrow column with no ink must not keep a model hallucination.
Why OCR must not decide emptiness
A recognizer is trained to emit a string. Input — an image. Output — alphabet symbols. If the image is only a table line, the net cannot “stay silent” unless you explicitly taught it an empty class. In the previous note mixing negatives cut ghosts and hit filled cells. The engineering conclusion: split the tasks.
On a UT form this is not academic. Empty cells are common: a point was not measured, an element is missing, the operator has not reached that zone yet. Feed every such rectangle to the measurement head and you get a tidy thickness table, part of which nobody wrote. The operator then compares the report with paper and loses trust in the loop faster than you can fine-tune the net.
In the pipeline it looks like this.
flowchart TB
crop[Cell_crop]
crop --> ink[Ink_and_kind]
ink -->|empty| skip[Skip_OCR]
ink -->|ditto| up[Resolve_up_column]
ink -->|crossed| strike[Mask_strike_then_OCR]
ink -->|digit_or_text| ocr[Handwriting_OCR]
ocr --> parse[Parse_and_range]
up --> parse
strike --> parse
parse --> field[Structured_field]
Until the crop has a kind, any engine — PaddleOCR or the compact CRNN — is a candidate to write a thickness from the grid. Classification is cheaper than that error. A cloud vision model on the crop is not exempt: without a kind it also has to “read” an empty rectangle.
Five cell kinds as a contract
The vision pass sets the kind explicitly: empty | ditto | digit | text | crossed. The TypeScript builder reads that kind instead of re-guessing geometry from raw text.
| Kind | What we see on the form | What we do |
|---|---|---|
empty |
no ink, or below threshold | skip handwriting OCR, value null |
ditto |
horizontal “same as above” | disable OCR, look up the column |
digit |
looks like thickness | read, normalize the comma, check range |
text |
“no element”, “heating”, a short note | do not force it into a digit-comma CRNN |
crossed |
a strike over writing | remove the line, read what remains, or leave crossed |
That is the “split by visual contract” table from the previous article, now in code: a cell-content classifier plus column-wise ditto resolution.
Text in a measurement cell is not a form bug. Operators write short notes. The parser handles not only 12,3 but dictionary labels such as “no element” and “heating”, and strips Roman section numerals that OCR sometimes glues onto a number. Those are rules, not a second network. Kind text tells the builder: do not squeeze the string into millimetres at any cost.
For the operator, kinds must look different in the report. Empty is one behaviour. A ditto with no source is another: a person has to see why the column broke. Strike-through is a third: not “no number”, but “there was one and it was cancelled.” Collapse all three into empty and sheet review becomes a lottery.
Ink: blue pen, pencil, grid
The ink detector looks at HSV for the blue pen band and at grey intensity for dark pencil. Thin horizontals and verticals of the grid are supposed not to count as writing: morphological opening with an ellipse. The fraction of “ink” pixels is compared with a threshold around 0.8% of the cell — enough for handwriting, easy to exceed for a fat grid if you do not subtract lines.
The kind classifier looks at the ink mask and connected-blob geometry: a wide, low blob with a large width-to-height ratio is closer to a ditto than to a digit. Fewer than four ink pixels — the cell is empty, no philosophy.
The threshold is a trade-off. Too high: a faint pencil becomes empty, the measurement vanishes, the verify loop later has to recover hasInk=true with a null value. Too low: the grid becomes “there is ink,” OCR paints 11,1. In the pilot we keep the threshold low and clean the kind in a second pass: ditto, strike-through, text.
A phone photo and a flatbed scan do not share contrast. Blue ink on yellowish paper and pencil on a grey photocopy are not the same HSV. So the threshold is not a product magic constant; it is a setting for film and light. Changing it without counting missed filled cells is how a thin 8,2 quietly disappears.
Ditto up the column
A horizontal dash on a UT form means “same as the row above at this point.” It is not a minus thickness and not “no data.” Feed the dash to OCR and you get -, 1, or a random digit tail.
Vision marks that crop ditto and clears OCR text. Column assembly runs top down: for each ditto row we look for the nearest upper cell in the same column whose kind is not ditto and whose value is not empty. Found — copy the value and record the source. Not found — review status, raw text like “ditto with no source row.” A silent null is worse: the operator will not see why the column broke.
A chain of dittos is normal: three dashes in a row should all point at one source thickness, not at each other as emptiness. Hence the top-down walk, not “take the neighbour and stop.” If the source cell is itself in review, the ditto inherits the doubt: you must not copy a confident ok from a shaky source.
This is ordinary code. You can unit-test it on a synthetic column. A neural net to “find dittos” would pay off only if blob geometry systematically confused a dash with a one or a strike. For now the error is more often the other way — grid as ditto — so tweaking blob geometry is cheaper than training a detector.
A separate trap is a dash in header utility fields. On design or reject thickness a horizontal often means “no number in this field,” not “same as the row above.” One glyph - on paper, two contracts in code. Mix them and you copy a diameter down a column the operator never wrote.
Strike-through: mask the stroke first
A crossed cell is a common field gesture: wrote, changed mind, struck a line, sometimes wrote a new number below. Feed the crop to OCR as-is and the net reads a mash of the old number and the stroke.
The detector looks for a strong or weak strike signal on the ink mask: an elongated blob, horizontal morphology, sometimes Hough lines. If a strike is present, we paint it white and read what remains; among candidates we prefer the lowest valid thickness — usually the correction, not the struck-out old value. If that fails, kind stays crossed, the value can go to review rather than ok.
This is brittle. A weak strike on a thin nine looks like the grid; a fat strike eats useful strokes. So strike-through is merged with neither empty nor ditto: the HTML report needs a different highlight than “simply no number.”
Not every strike means “read the number underneath.” In some procedures a crossed-out field is empty no matter what else is visible. Then the loop should stop at kind crossed and skip OCR: otherwise you “recover” a cancelled measurement and send it to ERP as live. In the pilot we try the lowest number because on our forms that is how thickness is usually corrected. That is a domain rule; write it down instead of hiding it in a blob threshold.
When we do not call handwriting OCR
On the measurement grid the recognizer runs only if there is ink and the preliminary kind is not ditto. Empty — save the crop and cellKind=empty. Ditto — crop exists, OCR does not. Strike-through — a separate read branch. Otherwise ocr_handwriting_cell: default printed/handwritten cascade, optionally the compact CRNN.
That saves time and, more important, gives an empty cell no chance to become a thickness. A second shield sits after the models: if a narrow column has a value but vision sees no ink, the cell is cleared — a typical hallucination of a “grid” model on empty IV.a. A third shield is a thickness cap of 80 mm: a merge of neighbouring digits (96 instead of empty) fails the domain check.
Narrow column IV.a in the pilot is empty more often and feeds ghosts more easily: the frame takes a larger share of the crop, and handwriting, when present, is tighter. You do not need a second net for that column — you need a stricter “no ink → drop value” bind. Relax the shield “so we do not miss a rare measurement” and the mass of 11,1 on empties comes back.
Printed utility fields (zone, diameter, type) go through another OCR with grid-border trim and upscaling. The header date is not stuffed into MeasurementCrnn: day, month, year are ordinary code. A measurement head whose alphabet is ten digits and a comma must not “just a bit” become a universal form reader — that was already a thesis of the previous note; here it is a fork in run_cv.
Why the heuristic is not the end
Blob rules are cheap, explainable, testable on synthetics. Their limit is predictable: new handwriting, a fatter grid from another scanner, a ditto that is a tick not a dash, a cross-shaped strike, faint gel ink. The hard-negative set for the CRNN already showed it: the model and the heuristic learn different mistakes.
The next step from the plan is a tiny empty-versus-value classifier (and, logically, a wider kind set) before the measurement head. The heuristic stays as a weak filter and as features: ink fraction, blob aspect, strike flag. The classifier trains on crops the heuristic systematically confuses. Until it exists, we honestly keep the tail in review and do not raise the ok threshold on doubtful tenths — that is already the discipline of the system note.
We have not trained it yet, not because “a convolution is too much work.” We need labels on the cells where the heuristic lies with confidence: a fat grid as a digit, faint pencil as empty, a short ditto as a one. Without that queue the classifier learns the same thing the 0.8% threshold already counts. Building the queue is more expensive than fine-tuning the measurement head on empties again — which is exactly why the temptation to mix the tasks stays strong.
We do not train a net per column. Kinds are shared across grid I–IV. Only the geometry of column IV.a changes (narrower, more often empty) — hence a separate shield “no ink → drop value,” not a second model.
Typical mistakes
The most expensive one is calling OCR “just in case” because the crop is already cut. A rectangle does not have to contain a number.
The second is merging ditto and empty into one status. Then the operator cannot tell “we did not measure here” from “this should match the row above, but the source is gone.”
The third is raising the ink threshold after a batch of ghosts and not counting how many filled cells became empty. A valve that “cures” empties easily strangles faint handwriting.
The fourth is fine-tuning the measurement head on empties until the ghosts die. In the previous note that already cost points of exact match on digits. A valve is cheaper.
The fifth is sending a utility-field dash through the same column resolver as a measurement. The meaning of - depends on the field, not on the shape of the stroke.
What to try today
If you slice a form into cells and call OCR immediately — stop the call on empties.
Put an explicit cell kind in the vision result, even three values: empty, ditto, other. Resolve dittos with code up the column, not with an OCR string.
Count separately: how many empties got a false thickness; how many filled cells were lost to a high ink threshold; how many dittos became - or 1.
Do not fine-tune the measurement head on empties “until victory.” Valve first, narrow OCR second, a human on review.
FAQ
Can we just add an “empty” class to the CRNN?
Yes, and we tried mixing negatives. Ghosts fall, digits too. An “empty” class inside a CTC head is a compromise in the same weights that must read a thin tail of a nine. A valve is cheaper.
How is a measurement ditto different from a dash in meta fields?
In measurements the dash is a reference to the row above. In design/reject thickness a dash often means “no number in this field.” Do not mix the semantics: one glyph - on the form, two contracts.
Must strike-through always be read?
No. Sometimes the stroke means “ignore the cell.” We try to read the lowest valid number because on our forms that is how a measurement is usually corrected. If your procedure treats crossed-out as always empty — keep kind crossed and skip OCR.
Do we need a neural net to find ink?
Not on day one. A blue/dark threshold plus grid morphology covers the mass. A net pays off when false hasInk on a given film/scanner cannot be healed by the threshold without growing misses.
How does this meet the cloud vision model?
Vision emits kind and crop. The cloud pass must not overwrite empty with a confident hallucination: the “no ink → drop value” shield is exactly about that. Low confidence and hasInk=true with a null value is a reason to verify, not a silent ok.
Further reading
The measurement head, 640 thousand parameters, and the empty-cell conflict inside CTC — compact CRNN. The whole-form loop and the operator report — one model is not enough. Sequence mechanics — from convolution to Transformer. Dataset as a product — dataset engineering. Cost of a miss — testing economics. The one-lever experiment loop — quality does not grow from hitting train.
Conclusion
Industrial OCR on a UT grid breaks not where “there are too few layers,” but where the cell was given no kind. Empty, ditto, strike-through, a number, and a short note are different contracts. In the pilot the cascade already exists: ink, kind heuristic, OCR banned on empties and dittos, column resolution in code, shields on thickness cap and on hallucinations without ink.
What remains open is what the heuristic cannot do: confident errors on a fat grid and faint handwriting. That is where a tiny classifier goes, not a second universal model. A practical step this week is to count empty ghosts before and after the hasInk / cellKind valve. If the difference is small, you are still feeding OCR the grid.