Commit Graph

620 Commits

Author SHA1 Message Date
Stefan Weil
9adf6e442b Revert 59fb3370bb (-ffast-math)
It breaks intsimdmatrix_test.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-14 17:56:35 +01:00
Nick White
ebbf907c56 Fix typo in hocr character box output 2019-01-13 16:28:31 +00:00
Nick White
4ce797b6f6 Fix hocr character box info to use new hocr renderer correctly 2019-01-13 13:01:14 +00:00
Nick White
c43e4501e3 Merge remote-tracking branch 'origin/master' into hocrcharboxes 2019-01-13 12:41:42 +00:00
zdenop
238cb219d5
Merge pull request #2152 from stweil/clean
Remove opencl_device_selection.h
2019-01-09 15:02:59 +01:00
Stefan Weil
a0e6586e63 Fix documentation for page segmentation mode 2
It never worked, so add a comment that the implementation is missing.
Add also a to-do comment.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-09 13:51:44 +01:00
Stefan Weil
0fae848b58 OpenCL: Add comments to users of openclwrapper.h
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-09 12:11:00 +01:00
Stefan Weil
e0fc4f2945 Remove opencl_device_selection.h
Always use OpenCL device selection if OpenCL is enabled.

This fixes a regression which was introduced by commit
5c6a57b727 which removed
the definition for USE_DEVICE_SELECTION.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-09 12:09:56 +01:00
Stefan Weil
595bb7df16 OpenCL: Remove unused code
The OpenCL kernel pixSubtract is never used, so remove it.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-05 16:41:20 +01:00
Nick White
b8de06430d Ensure baseapi.h header is used by commontraining.h regardless of autotools usage 2019-01-04 20:20:00 +00:00
Nick White
cd34ee55ec Add necessary intproto.h header to protos.cpp 2019-01-04 20:19:54 +00:00
Stefan Weil
62b635a74e Remove unused functions from cluster.cpp
Add also missing static attributes.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-03 13:16:31 +01:00
Stefan Weil
f76d8a14cd Remove unused code from oldlist
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-03 12:27:10 +01:00
Stefan Weil
7719f80155 Add missing std namespace in tensorflow code
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-03 11:15:36 +01:00
Stefan Weil
8a6fa452dc Fix build for architectures without CPUID
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-03 09:32:36 +01:00
Stefan Weil
91af010200 Fix compiler warning
gcc warning:

    src/training/text2image.cpp:694:35: warning:
        ISO C++ forbids converting a string constant to ‘char*’
        [-Wwrite-strings]

putenv expects a string which can be modified.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-01 22:49:04 +01:00
Stefan Weil
5dd606c631 Replace NULL by nullptr
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-01 22:45:49 +01:00
Stefan Weil
d9600cd82e Fix and simplify SIMD tests
The tests for SSE and AVX must only be done if the correct compiler
flags were used.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-01-01 11:19:17 +01:00
zdenop
d3065520fa fix 2 clang warnings 2018-12-30 20:25:24 +01:00
Stefan Weil
cb049133cd Fix compiler warning
clang warning:

    tesseractmain.cpp(512,21): warning: '&&' within '||' [-Wlogical-op-parentheses]

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-29 22:17:33 +01:00
zdenop
420fb0ced0 Merge branch 'master' of https://github.com/tesseract-ocr/tesseract 2018-12-29 10:31:33 +01:00
zdenop
8885fe2ccb provide info about compiled openmp version 2018-12-29 10:18:27 +01:00
Stefan Weil
993e56ffde Don't try to create text output if other renderers failed (fix regression)
Commit 49d7df6dc3 added error handling,
but since that commit Tesseract used the text fallback if the user
selected output failed.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-27 10:23:28 +01:00
zdenop
cc997b53c7 add missing the implementation for TessBaseAPIGetAltoText method in C-API 2018-12-26 21:35:47 +01:00
Stefan Weil
db9c7e0312 Use std::stringstream to generate hOCR output
Using std::stringstream simplifies the code and allows conversion of
double to string independant of the current locale setting.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-16 20:14:11 +01:00
zdenop
72d8df581b
Merge pull request #2121 from stweil/hocr
Move code for hOCR renderer to new file
2018-12-16 16:26:27 +01:00
Stefan Weil
c7e8d30280 Fix value for PHYSICAL_IMG_NR in ALTO output
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-16 15:07:02 +01:00
Stefan Weil
457c53026d Fix indentation of hOCR output
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-15 17:51:59 +01:00
Stefan Weil
5de3fc47bb Format code in new file hocrrenderer.cpp
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-15 15:35:21 +01:00
Stefan Weil
48713f7df2 Move code for hOCR renderer to new file
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-15 15:33:47 +01:00
zdenop
1f5fb15af3 remove setting constant resolution from ImageThresholder::SetImage.
Credible resolution with be set afterward. Fixes #2080.
2018-12-14 19:23:22 +01:00
zdenop
6d06d39bf4
Merge pull request #2118 from stweil/clean
protos: Remove several unused macros, functions and global variables
2018-12-14 09:20:53 +01:00
Stefan Weil
b8c4f1b9fc protos: Remove unused config variable
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-13 21:37:33 +01:00
Stefan Weil
f35eeb3b4a protos: Remove several unused macros, functions and global variables
The unused global variable TrainingData used a lot of runtime memory.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-13 21:32:56 +01:00
Stefan Weil
fbbbdb4565 Use std::stringstream to generate ALTO output and add <SP> element
Using std::stringstream simplifies the code.
The <SP> element is needed between two >String> elements.
Remove also some unneeded spaces in the ALTO output.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-12 22:29:35 +01:00
Stefan Weil
7ebd3153ae Fix several typos (most of them found by codespell)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-10 18:59:58 +01:00
Stefan Weil
81ab302d52 FPRow: Remove three unused methods
This fixes warnings from the Intel compiler:

    src/textord/cjkpitch.cpp(319): warning #177:
      function "<unnamed>::FPRow::good_gaps" was declared but never referenced
    src/textord/cjkpitch.cpp(383): warning #177:
      function "<unnamed>::FPRow::is_bad" was declared but never referenced
    src/textord/cjkpitch.cpp(387): warning #177:
      function "<unnamed>::FPRow::is_unknown" was declared but never referenced

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-08 16:43:52 +01:00
Stefan Weil
404f9cd147 SimpleStats: Remove unused method
This fixes a warning from the Intel compiler:

    src/textord/cjkpitch.cpp(79): warning #177:
      function "<unnamed>::SimpleStats::maximum" was declared
      but never referenced

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-08 16:39:46 +01:00
Stefan Weil
a9121d28f3
Merge pull request #2107 from stweil/march
Add check whether compiler supports -march=native flag
2018-12-08 10:53:09 +01:00
Stefan Weil
2c044df959 Fix wrong x_fsize in hOCR output (regression)
The regression was caused by the latest commit
c9e85ab78f.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-08 10:39:31 +01:00
Stefan Weil
2ccc5810f3 Add check whether compiler supports -march=native flag
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-05 20:13:28 +01:00
Stefan Weil
c9e85ab78f Fix wrong font attributes in hOCR output
Instrumented code throws this runtime error during OCR:

    ../../src/api/baseapi.cpp:1616:5: runtime error: load of value 128,
      which is not a valid value for type 'bool'
    ../../src/api/baseapi.cpp:1627:5: runtime error: load of value 128,
      which is not a valid value for type 'bool'

If there is no font information (typical for Tesseract with a LSTM model),
the font attributes got random values resulting in wrong hOCR output.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-04 10:52:46 +01:00
Stefan Weil
0bdae8f8bf GENERIC_2D_ARRAY: Fix runtime error in assignment operator
Instrumented code throws this runtime error during OCR:

    ../../src/ccstruct/matrix.h:84:11: runtime error:
      null pointer passed as argument 2, which is declared to never be null

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-04 10:48:46 +01:00
Stefan Weil
f0a4d04187 Add config variable for selection of dot product function
All also a C++ implementation with more aggressive compiler options
which is optimized for the CPU where the software was built.

It is now possible to select the function used for the dot product
with -c dotproduct=FUNCTION where FUNCTION can be one of those values:

* auto      selection based on detected hardware (default)
* generic   C++ code with default compiler options
* native    C++ code optimized for build host
* avx       optimized code for AVX
* sse       optimized code for SSE

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-12-01 00:19:28 +01:00
zdenop
b527b37825
Merge pull request #2097 from stweil/namespace
SIMDDetect: Use tesseract namespace and format code
2018-12-01 00:02:18 +01:00
Stefan Weil
1910b1a72b SIMDDetect: Use tesseract namespace and format code
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 23:36:39 +01:00
Stefan Weil
66d3275d0b IntSimdMatrixSSE: Remove unused include statement and simplify code
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 23:14:11 +01:00
Stefan Weil
048eb34934 Add missing static attribute to local inline functions
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 23:14:11 +01:00
Stefan Weil
b73370aac9 Remove unneeded test for nullptr
IntSimdMatrix::GetFastestMultiplier never returns a nullptr.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 23:14:11 +01:00
Stefan Weil
e2419b1968 Fix potential crash in tprintf
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 23:14:11 +01:00
Stefan Weil
6b6d9de497 Fix potential crash in STRING class
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 23:14:11 +01:00
Stefan Weil
59fb3370bb Use -ffast-math for calculation of dot product
This reduces the code size for intsimdmatrixavx2 from 2700 to 2668
and slightly improves the performance for fast models with AVX2.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 22:52:04 +01:00
Stefan Weil
fda3ba9009 IntSimdMatrixSSE: Fix comment
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 22:13:32 +01:00
zdenop
07b140364f
Merge pull request #2093 from stweil/python
Updates for Python scripts
2018-11-30 08:10:20 +01:00
zdenop
53600c677e
Merge pull request #2092 from stweil/format
Format new ALTO code with clang-format
2018-11-30 08:08:52 +01:00
zdenop
f6493dd5e8
Merge pull request #2090 from stweil/inline
Optimize performance by using inline functions
2018-11-30 08:07:45 +01:00
Stefan Weil
c59c45fb3e Fix Amharic font list
This was reported for the Python code by LGTM.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 08:00:22 +01:00
Stefan Weil
b148644c1b Make Python script executable
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 07:08:45 +01:00
Stefan Weil
ed48b2a8f5 Format new ALTO code with clang-format
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-30 06:37:25 +01:00
Jake Sebright
d7cee03a94 Add support for ALTO output 2018-11-30 06:09:36 +01:00
Stefan Weil
3c047f0ac8 Optimize performance by using inline function DotProduct
This improves performace for the "best" models because it
avoids function calls.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-29 21:43:41 +01:00
Stefan Weil
e161501df6 Optimize performance by using inline MatrixDotVectorInternal
This improves performace for the "best" models because it
avoids function calls.

The compiler also knows the passed values for the parameters
add_bias_fwd and skip_bias_back.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-29 21:37:32 +01:00
Egor Pugin
685b136d89
Fix incorrect condition. 2018-11-29 19:02:54 +03:00
Egor Pugin
267b79982d
Merge pull request #2076 from jbarlow83/pythonize-training
RFC: Pythonize tesstrain.sh and friends
2018-11-25 13:31:48 +03:00
James R. Barlow
8aa25239ae Fix some of Codacy's complaints 2018-11-24 16:59:01 -08:00
James R. Barlow
9122e6249e Autoreformat code
This increases the deviation from the bash scripts so is done separately.
2018-11-24 00:50:29 -08:00
James R. Barlow
d9ae7ecc49 Pythonize tesstrain.sh -> tesstrain.py
This is a lightweight, semi-Pythonic conversion of tesstrain.sh that currently
supports only LSTM and not the Tesseract 3 training mode.

I attempted to keep source changes minimal so it would be easy to compare
bash to Python in code review and confirm equivalence.

Python 3.6+ is required.  Ubuntu 18.04 ships Python 3.6 and it is a mandatory
package (the package manager is also written in Python), so it is available
in the baseline Tesseract 4.0 system.

There are minor output and behavioral changes, and advantages.  Python's loggingis used.  Temporary files are only deleted on success, so they can be inspected
if training files.  Console output is more terse and the log file is more
verbose.  And there are progress bars!  (The python3-tqdm package is required.)
Where tesstrain.sh would sometimes fail without explanation and return an error
code of 1, it is much easier to find the point of failure in this version.
That was also the main motivation for this work.

Argument checking is also more comprehensive.
2018-11-24 00:45:35 -08:00
Stefan Weil
9b783822a0 Remove unused include statements for tprintf.h
Format also a call of tprintf and add a missing explicit include statement.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-18 17:25:01 +01:00
Stefan Weil
a93426c9ff Fix wrong results from function streamtofloat
The local variable k should be 10 ^ (number of digits after comma),
but will overflow when there are more than 9 digits after the comma
because an int value cannot store 10000000000.

This results in wrong double values read from .tr files for example
(or in a runtime exception if Tesseract was compiled with -ftrapv).

Using uint64_t does not fix the general problem but allows more digits
which should be sufficient for the data read by Tesseract.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-17 20:02:21 +01:00
Stefan Weil
acca4fb999 Fix some unbound variables and other small issues in training shell scripts
Fix also the logging helper functions to work without log file.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-16 11:13:46 +01:00
Stefan Weil
a4b03fbb27 Fix warning from shellcheck
shellcheck warning:

    In /tesseract/src/training/tesstrain_utils.sh line 209:
        TIMESTAMP=`date +%Y-%m-%d`
                  ^-- SC2006: Use $(..) instead of legacy `..`.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-15 17:45:20 +01:00
John Lin
bfe58aa56f Fix unbound variable $FONTS 2018-11-15 17:43:15 +01:00
Stefan Weil
0915cbd535 Simplify shell script using mktemp
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-11-15 13:36:52 +01:00
John Lin
edb76e281a Simplify MKTEMP_DT logic 2018-11-15 10:38:40 +08:00
John Lin
dbfc89f9af Fix mktemp in tesstrain_utils.sh
The commit 10f2c45c00 unified the usage of mktemp, but with a
incorrect bash syntax and unnecessary definition of LANG_CODE
and TIMESTAMP. This patch fixes the above problems.
2018-11-14 09:04:34 +08:00
Ray Smith
ce88adbf32 fix issue #1192 2018-11-12 12:53:12 +01:00
zdenop
724957167e fix typo in non VS build 2018-11-08 23:10:14 +01:00
zdenop
eb104f9fe4 VS build: fix warning C4996: The POSIX name for this item is deprecated. Instead, use the ISO C and C++ conformant name. 2018-11-08 22:55:04 +01:00
zdenop
cbef2ebe12 implement patches vcpkg tesseract 2018-11-08 21:37:47 +01:00
zdenop
7a7f226228 ocrclass: Remove unused macros
Signed-off-by: Stefan Weil <sw@weilnetz.de>

# Conflicts:
#	src/ccutil/ocrclass.h
2018-11-08 20:23:36 +01:00
Zdenko Podobný
2dd753ee4c replace VS implementation of gettimeofday with std::chrono::steady_clock::now(); fixes #2038 2018-11-08 19:43:46 +01:00
chrismamo1
439dfaaf8b un-fix one of the warnings 2018-10-30 18:10:48 -06:00
chrismamo1
30be5aaaac fix a couple minor compiler warnings 2018-10-30 18:00:32 -06:00
Stefan Weil
6f8bd340d9 Remove chopper.h
It is no longer needed after some reordering of code in chopper.cpp.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-29 19:51:44 +01:00
Stefan Weil
286dfb031a Remove unused include statements
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-29 19:46:58 +01:00
Stefan Weil
2098bb6daf Remove unused function ComputeOrientation
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-29 19:43:56 +01:00
Stefan Weil
cad6ebb5ff LIST: Remove old comments
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-29 19:43:56 +01:00
zdenop
99054f10c7
Merge pull request #2027 from stweil/warn
Fix compiler warning
2018-10-24 07:31:15 +02:00
Stefan Weil
eefb8348f7 Fix compiler warning
Compiler warning on macOS:

    tesscallback.h:29:7: warning:
      'TessClosure' has no out-of-line virtual method definitions;
      its vtable will be emitted in every translation unit [-Wweak-vtables]

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-23 17:01:53 +02:00
Noah Metzger
f7f5f41073 Fixed a mac compiler warning in recodebeam.cpp
Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2018-10-23 16:57:39 +02:00
zdenop
e60318f9c0 set PANGOCAIRO_BACKEND=fc to avoid crash; fixes #736 2018-10-23 13:22:38 +02:00
Zdenko Podobný
3d508a65a7 set unlv_tilde_crunching to false; fixes #1449 #948 2018-10-23 09:26:32 +02:00
Stefan Weil
7ebbb7370a ColPartition: Fix CID 1164543 (Division or modulo by float zero)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-22 22:14:15 +02:00
Stefan Weil
eaabe4a3ce ErrorCounter: Fix CID 1164538 (Division or modulo by float zero)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-22 22:14:15 +02:00
Stefan Weil
8f615d44f1 osdetect: Fix CID 1164539 (Division or modulo by float zero)
Avoid also a conversion from int16_t to double to float.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-22 22:14:15 +02:00
Stefan Weil
be0cf03778 tesseractmain: Fix memory leak
Commit 49d7df6dc3 introduced a memory leak
when the output file could not be created.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-22 18:50:47 +02:00
Stefan Weil
9c0799314e Add parenthesis in boolean expression
This fixes a compiler warning:

    scanutils.cpp:444:32: warning:
        '&&' within '||' [-Wlogical-op-parentheses]

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-22 17:48:17 +02:00
Stefan Weil
0f973e1d62 Add missing 'static' keyword
This fixes a compiler warning:

    globaloc.cpp:33:6: warning: no previous extern declaration for
      non-static variable 'global_crash_pixes'
      [-Wmissing-variable-declarations]

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-22 17:48:17 +02:00
Stefan Weil
a71ad455be Remove unused macros
This fixes some compiler warnings:

    mainblk.cpp:28:9: warning: macro is not used [-Wunused-macros]
    mainblk.cpp:29:9: warning: macro is not used [-Wunused-macros]
    [...]

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-10-22 17:48:17 +02:00
zdenop
dba7f456d5
Merge pull request #2018 from stweil/sort
Get sorted list of available languages
2018-10-22 16:06:42 +02:00