Processing "clean" text to simulate "old-fashioned" printing

Question

I am hoping to start a project to train Tesseract for a certain kind of image that is a very common problem for open-source OCR-assisted transcription of old texts: ~1700s printing presses and paper were not very good and, combined with the use of long-s, OCR badly.

Thus, I intend to attempt to train Tesseract 4 (or 5) on a corpus of text. However, I do not heavy ready access to thousands and thousands of line-by-line ground-trust image/transcription pairs and I don't really want to start a whole website just for collecting such transcriptions (I will if I have to, but, oh, I so really don't want to)!

So, my current plan is to use a similar font (most text of the period uses the original Caslon, or something like it, so that's easy enough to do). However, I think the variable impression quality, degradation of the text during 300+ years and the scanning process is going to be the major issue here.

So, my question is: given a block of "perfect" text produced by a computer, how can I degrade it to look like this, close enough that a Tesseract model trained on it is at least not worse than using the "clean" text:

Features like the lines and drop caps are not important - the degradation of the body text is what's important. And it has to be automate-able (I can use OpenCV, Pillow, GIMP+Python or whatever, but doing it by hand is not possible).

I have tried a lot of messing about in OpenCV, but I am struggling to find any handle on the problem in a way that produces something similar.

Answer 1

To make a convincing simulation of a scanned page from a 1700s printed book, you'll need to study and analyze each step of the process it has went through. Since there is a variety in how scanned pages look, you'll have to determine some parameters you can randomize to create diverse samples.

I thought the whole idea with machine learning was to be able to skip (part of) such an analysis and instead let the computer find patterns by itself based on the real-life data presented to it.

To make a simulation of the world and then let a computer analyze your interpretation seems less than ideal to me. No matter how thorough you are, you are bound to make some simplifications. Aren't you risking miseducating your computer? Wouldn't it be much better to use real-life samples?

To me it sounds a bit like building a walking robot and letting a computer study it to analyze how people walk.

(I only know the general principle of machine learning so forgive me if I sound naive.)

---

If I were to create a copy of that single page you show here, I would simply start by setting up the text in InDesign, manually adjust everything, export an image and deteriorate it in Photoshop. But that's a designer's approach. You need a programmed way of producing varied samples.

Here is a list of subjects I would explore and try to turn into variables. Some of them might turn out to be disregardable and there might be additional subjects I'm not thinking of. This is just a quick brainstorm. Each subject is a rabbit hole of its own.

Text

The raw text used for samples should ideally be from the correct time period or at least follow all the grammatical rules from the time. It should use the correct glyphs (all the different kinds of s's, ligatures etc.).

Font

You might be right that most books were set in Caslon, but you would have to research that. If you find examples with other fonts with distinctively different features, you'll have to include them.

You need to both use the regular and the italic version of the fonts.

It's important that the fonts you use contain all the needed glyphs.

The digital fonts we have today differ a bit from the lead types used at the time which had less sharp details and were designed differently at different font sizes. Bear in mind that many different versions of fonts exist. Some might look more like the original than others.

Layout

You need to somehow establish some rules for the layout of the pages which you can randomize. You might need to look at a selection of pages and systematically measure: page sizes, margins, font sizes, leading, positioning of page number, headers, footers, decorations etc.

Typesetting

When we use a digital layout application to set up text, a myriad of typographical rules are automatically applied. Most of these are of course inherited from the typesetting craft from the past, but there might have been some different practices back then. In your example, I notice the extra wide space after punctuation characters. There might be many of these rules to account for. And different rulesets might have been used by different typesetters.

The manually set type have some wonkiness to it. This might be possible to imitate by applying random changes to the tracking and also applying tiny random rotation to each individual letter. But of course make sure letters don't overlap which wouldn't be possible in real life.

Ink

When printing letters on paper, the ink will bleed slightly into the paper. What we call dot gain today. Sometimes there will be too much ink applied and the letters become very blobby, perhaps even with small additional blobs of ink close to the letters. Sometimes too little ink is applied resulting in lighter letters which might even have dots of white paper with no ink. This might vary from page to page and even from one side of a page to the other.

This can be simulated in numerous ways. For example by applying Gaussian Blur followed by Threshold. Using a mask with some random gradient to create variation. Here is a quick example:

Paper texture

It might be a good idea to be able to add different realistic paper textures. Also look into adding some wear in the form of folds, dots, water damage and whatever discoloration that could occur.

Paper distortion

The paper could have some bumps and dents. It might be a good idea simulate this. For example by applying some subtle, random displacement map.

Furthermore the binding of the book makes it impossible to place the book completely flat in the scanner. Some perspective distortion will occur close to the spine. This could happen in either side of the page. You also need to take this into account.

Scanning

The example you show looks like a black and white (1-bit) scan (which have been saved as a compressed JPEG). Perhaps most scans are like this, but there could also be grayscale and color images. Some heavily compressed files also have this strange kind of mix between color and b&w:

As mentioned, your example has some quite visible JPEG artifacts:

You might need to take into account that input images could be heavily compressed.

Answer 2

This is the basic technique I have used so far:

Image degradation

• Dilate the text a little (because the text is black on white it's actually an erode, but it makes the text "fatter")
• Add Gaussian noise (with various different amplitudes)
• Clip the image values
• Blur the images
• Threshold the images
• Erode the result, with various different values (simulating lighter/heavier printing)

I plan to also sprinkle in some "blobs", but the result so far has been good, and I think blobs will need a lot more training, since the model will need to see blobs in lots of positions to be able to make sense of them.

# add noise to the images for train and test cases
def distort_image(input_imgs, noise_factor=0.5):
    noisy_imgs = input_imgs + noise_factor * np.random.normal(loc=0, scale=1, size=input_imgs.shape) 
    noisy_imgs = np.clip(noisy_imgs, 0, 1.0)
    return noisy_imgs


def process_image(args):
    img = cv.imread(imgf, cv.IMREAD_GRAYSCALE)

    erode_kern_size = 2
    noise_strength = args.noise
    blur_size = 3
    threshold = 0.6
    dilate_kern_size = args.erode  # yes it's backwards because the image is black on white

    kernel = cv.getStructuringElement(cv.MORPH_ELLIPSE, (erode_kern_size, erode_kern_size))
    img = cv.erode(img, kernel, iterations=1)

    img = img / 255.0

    img = add_noise(img, noise_strength)

    img = cv.GaussianBlur(img, (blur_size, blur_size), 1.0)

    img = img * 255

    ret, img = cv.threshold(img, threshold * 255, 255, cv.THRESH_BINARY)

    kernel = cv.getStructuringElement(cv.MORPH_ELLIPSE, (dilate_kern_size, dilate_kern_size))
    img = cv.dilate(img, kernel, iterations=1)

Font

Nearly all the texts I have looked at are either in Caslon or something very like it. Texts that are newer generally don't have the two major issues that destroy the OCR accuracy: long-s and rough paper and printing. By the early 19th century, printing and paper was much improved and generally the existing Tesseract eng model (which is trained on a ground-truth corpus that includes a range of fonts) works.

However, there are some differences between the original Caslon (and it's contemporaneous imitators) and modern computer fonts. I took Adobe Caslon Pro and made a few changes in FontForge to make it emulate the text more closely:

• Reduced the kerning after long-s (ſ) a lot - this is a bit cause of errors when the next letter is nestled under the "ſ", like in "ſo"
• Adjusted some serifs to more closely match
• Lowered the "rise" of lower case "r"
• Added some substitution forms with "errors" like missing dots on i and j, truncated bars on t (which causes t → r/i/c errors)
• Added a substitution form with a much higher "bar" in the e (not all texts have this, but lots do, and it causes e → c errors)

The substitution forms are sprinkled in randomly though the training data to simulate occasional damaged letters.

I also wanted to reduce the spacing, but doing that in Pango disabled ligatures, which are important to include, so I haven't done that yet. A small horizontal "squeeze" might be enough, but it also doesn't seem to be needed at all.

I turned on the following font features as well:

'dlig', 'hlig', 'liga', 'onum', 'swsh', 'cswh', 'calt', 'kern'

Here are a few of the images. I found that models trained with the "clean" images were less accurate, so I left those out in the training.

"Clean" (not degraded):

Noise 0.2

Noise 0.3

Noise 0.4

So this looks plausibly degraded, and the fact that including "clean" text makes the OCR model worse when applied to a real image indicates that, at least to some extent, the model "considers" this degradation to be representative of the real image.

Result

After generating 30,000 images of lines of text and the corresponding ground truths, and with 20,000 training iterations I can go from this:

Zeft thou forget the things which thine Eyes bave feen

to this:

leſt thou forget the things which thine Eyes have ſeen

It's still not perfect, but it's much, much better. Work continues...

Answer 3

I would suggest a combination of blurring, resharpening and texture overlays. You should find some suitable textures if you search "grunch texture" (probably then reverse them and overlay with blending mode "screen")

Here an example original text on the left text with filters on the right...