tesseract ... lstm.train can produce defective *.lstmf files

[adam-funk]

tess-4c-examples.zip
The attached lstmf files contain 4 bytes of nulls and, even when mixed in with a batch of valid lstmf files, cause the lstmtraining command to fail. They were generated with a script looping through the *.tif files generated by another script (along with matching *.box files):
tesseract 20190930-125338-000000007.tif 20190930-125338-000000007 lstm.train

I admit that these examples of image files are atrocious for OCR, but I'm doing experiments trying to train tesseract to deal with text superimposed on images. I'm not surprised that it rejects some of the tif/box combinations I've produced, but I don't think it should produce defective files that crash a later stage of training.

I generated 90000 matching *.box and *.tif files; the tesseract lstm.train command produced 89676 *.lstmf files, many of which were 4 bytes long. Once I ran find -name '*lstmf' -size 4c -delete, which left me with 26320 *.lstmf files, and then lstmtraining worked.

[wrznr]

This is a Tesseract-related issue. Pls. report in the corresponding repository. (tesstrain is simply an ease-of-use wrapper for training Tesseract.)

[stweil]

I can transfer the issue to Tesseract, so no need to create a new issue there.

[stweil]

I can confirm that even the latest code produces unusable lstmf files for this example. It should either fail with an error message or produce working lstmf files.

[stweil]

This problem will occur with any image where Tesseract does not detect text.

[adam-funk]

Thanks for confirming that. I'm not surprised that it can't detect text in some of the horrible examples I'm using, and I now have a workaround (using find ... -delete to clean up before running the next command) for it. If the documentation warns users about this, I'm sorry I didn't see that. If it doesn't, it would be helpful to include it.

[stweil]

Pull request #2744 fixes this issue. Tesseract no longer creates an lstmf file which just contains four 0 bytes. It now prints a message and fails for this case.

[adam-funk]

Thanks!

[stweil]

The previous fix created a new problem. Pull request #2751 addresses this.