Tesseract strikes again

[Sklifosoffski]

Both Audiveris versions 5.2 and 5.3beta seem to ignore the TESSDATA_PREFIX variable.

Steps taken as per instructions: language training data files for languages of choice (en, de, fra, ita) downloaded from Tesseract Git (v. 3.x) and placed into a directory made for them; TESSDATA_PREFIX is set to the parent directory;

Other things I've tried: completely removed Tesseract v.5.0 that was installed on the system (it was confirmed working prior to removal); pointed TESSDATA_PREFIX to the actual tessdata directory (not the parent one) as suggested on one of the Tesseract-related discussions elsewhere... still no joy.

Not a super-critical issue, but would appreciate if someone can point me in the right direction... what am I missing?

OS: Kubuntu 23.04
Java: OpenJDK 11 (for Audiveris 5.2) and 17 for the beta -- program compiles and runs no problem, just the OCR is missing.

[hbitteur]

@Sklifosoffski

1/ Use "development" branch, which means 5.3beta (and Java 17)
2/ Since last week, this branch now uses Tesseract 5.2.0 with 4.0+ standard language files (available from https://tesseract-ocr.github.io/tessdoc/Data-Files.html) instead of the old 3.x ones.
3/ TESSDATA_PREFIX must now point to tessdata itself (no longer to tessdata parent). This change occurred between Tesseract 3 to 4.

I'm working on documentation update for the coming 5.3 release, sorry for the delay.

[Sklifosoffski]

thank you for the prompt response! it is not working still, however...
Audiveris 5.3-beta:2f3f0d4bb
the language files (deu fra ita eng .traineddata) tagged 4.00
TESSDATA_PREFIX=/usr/share/tesseract-ocr/4/tessdata/

[hbitteur]

Could you tell more about the observed behavior?

0/ Especially, could you share the .log file?

1/ Near the beginning of log file, you should have a line like:

OCR Engine: Tesseract OCR, version 5.2.0

2/ Launch Audiveris in interactive mode and select the pull down menu Book | Set Book Parameters.
Check the box facing the item OCR language(s) on the left and observe the languages in a combo box on the right.
There you should read all the languages available. Note this list is retrieved via the Tesseract library, which implies that Tesseract is able to read these files.
Does this work?

3/ I suppose the OCR step does not work in your case. Any error message?

[Sklifosoffski]

no, TesseractOCR still reports 338...

alex@alex-desktop:~/Soft/audiveris$ ./gradlew run

Configure project :
targetOS=linux-x86_64

Task :compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

Task :run
INFO [] CLI 282 | CLI args: []
WARN [] TesseractOCR 338 | Tesseract data could not be found. Try setting TESSDATA_PREFIX environment variable to point to tessdata folder.
INFO [] Main 402 | Environment:

• Audiveris: 5.3-beta:95a1eb3b9
• OS: Linux 6.2.0-20-generic
• Architecture: amd64
• Java VM: OpenJDK 64-Bit Server VM (build 17.0.6+10-Ubuntu-1ubuntu2, mixed mode, sharing)
• OCR Engine: No OCR is available!
  INFO [] AliasPatterns 134 | Alias patterns: [(IMSLP[0-9])-.]
  INFO [] MainGui 573 | Audiveris version 5.3-beta
  INFO [] MainGui 574 |
  LogUtil. Property logback.configurationFile not defined, skipped.
  LogUtil. No /home/alex/.config/AudiverisLtd/audiveris/logback.xml, skipped.
  LogUtil. Configuration found file:///home/alex/Soft/audiveris/res/logback.xml
  LogUtil. Logging to /home/alex/.cache/AudiverisLtd/audiveris/log/20230508T121026.log

WARN [] MainGui 577 | No OCR is available! Check log file for more details.
INFO [] PluginsManager 210 | No /home/alex/.config/AudiverisLtd/audiveris/plugins.xml file found
INFO [] BasicClassifier 277 | Classifier loaded XML norms.
INFO [] AbstractClassifier 396 | Classifier data loaded from default uri file:///home/alex/Soft/audiveris/res/basic-classifier.zip
INFO [] Versions 301 | Versions. Poll frequency: Always, next poll on: 07-May-2023
INFO [] Versions 397 | Software version is up-to-date
INFO [] BookParameters 438 | No language pane for lack of OCR.
INFO [] ConstantBasedParam 110 | Default Default font family for music symbols set to MusicalSymbols
INFO [] BookParameters 386 | Default parameters committed
<==========---> 83% EXECUTING [3m 1s]

:run

[hbitteur]

@Sklifosoffski
I have made a very basic test:

1. Discard all *.traineddata files from my tessdata folder (the one pointed to by TESSDATA_PREFIX)
2. Download 5 languages from https://github.com/tesseract-ocr/tessdata into tessdata folder
3. Run Audiveris again: OCR still works

Here are the SHA signatures of the 5 files I downloaded and used:
sha-signatures.txt
Could you make sure that yours have the same SHA signatures?

FYI, I used sha256sum utility as follows, from within tessdata folder:

$ sha256sum *.traineddata > sha-signatures.txt

To check them, you can use (assuming you have downloaded sha-signatures.txt file to tessdata folder as well):

$ sha256sum --check sha-signatures.txt
deu.traineddata: OK
eng.traineddata: OK
fra.traineddata: OK
ita.traineddata: OK
spa.traineddata: OK

[Sklifosoffski]

alex@alex-desktop:~/Downloads/tessdata.4+$ sha256sum *.traineddata --check sha-signatures.txt
sha256sum: deu.traineddata: no properly formatted checksum lines found
sha256sum: eng.traineddata: no properly formatted checksum lines found
sha256sum: fra.traineddata: no properly formatted checksum lines found
sha256sum: ita.traineddata: no properly formatted checksum lines found
sha256sum: lat.traineddata: no properly formatted checksum lines found
sha256sum: osd.traineddata: no properly formatted checksum lines found
deu.traineddata: OK
eng.traineddata: OK
fra.traineddata: OK
ita.traineddata: OK
sha256sum: spa.traineddata: No such file or directory
spa.traineddata: FAILED open or read
sha256sum: WARNING: 1 listed file could not be read

[Sklifosoffski]

@hbitteur I FOUND IT!!! (it being what I did wrong):
one has to export the TESSDATA_PREFIX in the same terminal window as the subsequent process run!!! (and I did it in a separate terminal...)

now everything looks as it should!

alex@alex-desktop:/Soft/audiveris$ export TESSDATA_PREFIX=/usr/share/tesseract-ocr/4+/tessdata/
alex@alex-desktop:/Soft/audiveris$ ./gradlew run

Configure project :
targetOS=linux-x86_64

Task :run
INFO [] CLI 282 | CLI args: []
INFO [] Main 402 | Environment:

• Audiveris: 5.3-beta:95a1eb3b9
• OS: Linux 6.2.0-20-generic
• Architecture: amd64
• Java VM: OpenJDK 64-Bit Server VM (build 17.0.6+10-Ubuntu-1ubuntu2, mixed mode, sharing)
• OCR Engine: Tesseract OCR, version 5.2.0
  INFO [] AliasPatterns 134 | Alias patterns: [(IMSLP[0-9])-.]
  INFO [] MainGui 573 | Audiveris version 5.3-beta
  INFO [] MainGui 574 |
  LogUtil. Property logback.configurationFile not defined, skipped.
  LogUtil. No /home/alex/.config/AudiverisLtd/audiveris/logback.xml, skipped.
  LogUtil. Configuration found file:///home/alex/Soft/audiveris/res/logback.xml
  LogUtil. Logging to /home/alex/.cache/AudiverisLtd/audiveris/log/20230508T124026.log

INFO [] PluginsManager 210 | No /home/alex/.config/AudiverisLtd/audiveris/plugins.xml file found
INFO [] BasicClassifier 277 | Classifier loaded XML norms.
INFO [] AbstractClassifier 396 | Classifier data loaded from default uri file:///home/alex/Soft/audiveris/res/basic-classifier.zip
INFO [] Versions 301 | Versions. Poll frequency: Always, next poll on: 08-May-2023
INFO [] Versions 397 | Software version is up-to-date
INFO [] ConstantBasedParam 110 | Default OCR language(s) set to deu+eng+fra+ita+lat
INFO [] BookParameters 386 | Default parameters committed

BUILD SUCCESSFUL in 2m 52s
5 actionable tasks: 3 executed, 2 up-to-date

will test the actual text recognition later today.

THANK YOU for your patience, and for the unique soft in the FOSS space!

[hbitteur]

@Sklifosoffski
You can't be serious! :-)
I'm glad you found the mistake.
And I just modified Audiveris to always print out itself the value of TESSDATA_PREFIX environment variable. Just in case.

[Sklifosoffski]

Tested the OCR part -- it does extract lyrics etc.

Thank you so very much again! Please keep up the good work, it helps tremendously the amateur classical vocalists like myself! I'm marking the issue "solved".