Why OCR can't fix rare Chinese characters

27 Sep 2026

I’ve been structuring the tables of contents of a series of village gazetteers from Guangdong: for each book, turn the contents pages into a “town → administrative village → natural village → page” hierarchy, so the body text can later be linked back to it. The contents pages are two-column lists set in a plain Song typeface, which looks like the easiest possible layout for OCR. In practice the structure and page numbers came out fine. Almost every problem was in the characters.

A first human review of three volumes (986 lines) found 30 real errors, and nearly all of them were the same kind: rare characters used in Cantonese place names. Worse, many of those errors came with a confidence of 1.0, so no confidence threshold could catch them.

Three root causes §

The dictionary ceiling. The OCR engine I used (PP-OCRv5) is a classifier over a fixed character set of 18,383 characters. 㘵, 乪, 揾 and 𣿀 are not in it. For a character outside the set, the probability of emitting it is exactly zero. Upscaling the image or tuning parameters cannot change that.

Rare characters lose to the prior. 冚, 迳, 塅, 坣 and 窿 are in the character set, but barely appear in the training data. The result: 冚 gets dropped entirely (水冚村 becomes 水村), 迳 becomes 泾, 坣 becomes 堂, 窿 becomes 隆. The model is very confident about these mistakes, because from its point of view the common character is the “reasonable” answer.

Large vision-language models fail differently. Their vocabulary comes from an LLM tokenizer, which in principle covers all of Unicode, so the dictionary ceiling is gone. But they carry a strong language prior. I rendered 水冚村 cleanly from a font and asked a 7B vision model to transcribe it. It wrote 水田村. Scoring the candidate strings gives a log-probability of −0.64 for 水田村 and −8.58 for 水冚村. However clear the image, the model prefers a common word.

Four dead ends §

Crop the line, upscale it 2×, and run the same model again. Of 28 errors, 6 were fixed, 5 changed into something else, and 17 came back unchanged. The errors come from the character set and the prior, not from resolution, so a model re-checking itself doesn’t help.
Use a vision-language model as a second opinion. It did read a few characters the original model physically cannot output (乪, 揾). But on single-line crops it hallucinated heavily: of 30 lines that were already correct, it turned 13 into dots, underscores or Japanese. Usable only with output cleaning, and it cannot help when both models get the same character wrong.
Candidate scoring. With a self-hosted model you can skip free generation and just compare the probabilities of a few candidate strings. The −0.64 vs −8.58 above shows that with a strong enough prior, scoring picks the wrong one too.
Zero-shot character recognition. There is a line of research that decomposes Chinese characters into stroke or component sequences, so unseen characters can still be recognised. I trained an open-source implementation from scratch, deliberately excluding the target rare characters from training. On real crops it reached 76% on the target characters with an open candidate set and 86% with a restricted one; 㘵 7/7, 冚 10/10 and 迳 5/5 were all correct. But on ordinary characters it only reached 97.3%. Useful as a screen for characters outside a fixed list, not as the main tool.

What worked §

Back to the plainest idea: the contents are monospaced Song type, so every character sits in a square cell. Cut out each character and classify it on its own.

Segmentation. Character pitch is roughly equal to line height. Take the vertical ink projection along each line and snap the cut points to the pitch grid, so left-right characters like 朋 or 州 are not split in half. On the 986 human-checked lines, the number of segmented characters matched the length of the correct name 100% of the time.
Training data is entirely synthetic. The classes are common characters plus every character that appears in the books and in place names: 6,788 classes in the first version. Each is rendered from 27 Song and sans-serif font faces, then degraded like a scan: blur, binarisation jitter, JPEG compression, slight rotation, stroke thickening and thinning. Not a single real crop was used in training.
The model is small. A ResNet-18 with 64×64 input, trained in about 40 minutes on one A4000 with 2.2 GB of memory.

On 3,294 human-verified real character crops, per-character accuracy was 99.76%. Confusable pairs such as 迳/泾, 窿/隆 and 坣/堂 had zero confusion in either direction, and all 11 characters outside the OCR dictionary were correct.

The classifier doesn’t replace OCR. It acts as a second reader: if the string the classifier reads differs from the string OCR reads, a human looks at that line. On those three volumes, this rule caught all 30 errors found by the human reviewers, with only 6 false alarms among 940 correct lines. It also found 16 errors that the human review had missed.

Two later problems are worth noting:

Two pages had been photographed by hand with motion blur, and the classifier dropped to 44% on them. The reason was simple: motion blur wasn’t among the training degradations. After retraining with motion blur, defocus and ghosting augmentation, the blurred pages came back to 96.3%, and sharp pages didn’t get worse.
When the class list was expanded to every character used in place names across the province, the first attempt simply turned all 7,741 code points into classes, and accuracy went down. The place-name data mixed in traditional and variant code points, so 高 and 髙 became two classes competing for the same printed glyph. After folding 189 variant code points back into their standard forms, the list settled at 7,552 classes and the problem disappeared.

Takeaways §

The key to catching confident errors is not a stronger model but a second reader that fails in a different way. The single-character classifier has no language prior; to it, 冚 and 田 are just two equal classes. That covers exactly the blind spot OCR and large models share.
Synthetic data works better than I expected, as long as the degradations resemble the real material. Looking at a contact sheet of real and synthetic crops side by side is worth more than an afternoon of hyperparameter tuning.
Human review is noisy too. Of the 46 real errors in those three volumes, the first human pass missed 16. Before using human labels as ground truth, it’s worth knowing their recall.