Commit Graph

343 Commits

Author SHA1 Message Date
Stefan Weil
f3c7a17df3 Fix CID 1395108 (Dereference after null check)
Let's hope that word->best_choice is never NULL.
Overwise both the old and the new code would abort with SIGSEGV.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-22 13:55:37 +02:00
Stefan Weil
6092a8f865 Fix CID 1395109 (Logically dead code)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-22 13:55:37 +02:00
Stefan Weil
ac17663015 Fix CID 1395113 ('Constant' variable guards dead code)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-22 13:55:37 +02:00
Stefan Weil
7e9dfefc5c Fix CID 1395114 ('Constant' variable guards dead code)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-22 13:55:37 +02:00
Stefan Weil
99efc13de8 Fix CID 1395116 ('Constant' variable guards dead code)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-22 13:55:37 +02:00
Egor Pugin
621a8cd29d
Merge pull request #1851 from noahmetzger/winfix
Added the option for character accumulated glyph confidences.
2018-08-20 16:35:14 +03:00
Egor Pugin
1f3acca03a
Merge pull request #1850 from Shreeshrii/new-branch-name
add option --save_box_tiff to save box/tiff pairs with lstmf files
2018-08-20 12:39:52 +03:00
Noah Metzger
663be426f6 Added the option for character accumulated glyph confidences.
The parameter glyph_confidences is changed from bool to int.
An execution with value 1 outputs the hOCR file enriched with glyph confidences
for every timestep like before. An execution with value 2 outputs the timesteps
accumulated over the recognized characters.

Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2018-08-20 10:43:58 +02:00
Shree Devi Kumar
43e3f24bb0 add variable --save_box_tiff to Save box/tiff pairs along with lstmf files. 2018-08-20 08:24:09 +00:00
Egor Pugin
115fe7662c
Merge pull request #1844 from Shreeshrii/new-branch-name
Updates to Javanese Script Validation and Training
2018-08-17 13:24:28 +03:00
zdenop
debe3da36d remove duplicate include 2018-08-16 20:50:28 +02:00
Shree Devi Kumar
b34cf9d424 Javanese script training 2018-08-16 12:15:10 +00:00
Stefan Weil
e1c387c9b3 Fix typo in comments and variable name
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-16 11:38:36 +00:00
Stefan Weil
bf33301114 Fix typo in function name
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-16 11:38:36 +00:00
Stefan Weil
641237495a Fix typo in comments and variable name
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-14 16:20:27 +02:00
Stefan Weil
95ed924d81 Fix typo in function name
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-14 16:20:27 +02:00
Stefan Weil
ce135de37c scrollview: Clean include statements
cstring was included twice (reported by Martin Strunz).
Use C++ header files and sort them alphabetically.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-14 13:12:51 +02:00
Zdenko Podobný
296309c1f1 remove duplicate include. Fixes #1837 2018-08-14 13:06:14 +02:00
Atsuyoshi Suzuki
4cda775d73 Revert Makefile.am to beta.2
thesserocr needs `osdetect.h'.
2018-08-06 23:21:20 +09:00
Shree Devi Kumar
7957288fd5 chamge validate javanese similar to indic 2018-08-04 09:43:53 +00:00
Shree Devi Kumar
f93f9e8a09 fix typo re Javanese 2018-08-03 14:33:24 +00:00
Shree Devi Kumar
0eb7be1cd1 Initial COmmit to add Aksara Jawa - Javanese script 2018-08-03 13:59:27 +00:00
Stefan Weil
6a0f8e8c07 ColPartition: Rename median_size_ -> median_height_
This implements a TODO. Rename also some related items.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-03 08:46:38 +02:00
Stefan Weil
8af80b7ba6 Fix ImageThresholder::OtsuThresholdRectToPix for OpenCL
The ThresholdRectToPix OpenCL kernel only supports 4 channels.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-01 22:49:28 +02:00
zdenop
c044b8c916
Merge pull request #1818 from stweil/psm
Fix potential crash with --psm 0 and use osd.traineddata automatically
2018-08-01 16:56:56 +02:00
zdenop
d22ca6bb06
Merge pull request #1817 from noahmetzger/winfix
Fix issue detected by Coverity Scan
2018-08-01 16:55:56 +02:00
Stefan Weil
27ce472666 Fix potential crash with --psm 0 and use osd.traineddata automatically
Page segmentation mode "OSD only" requires osd.traineddata,
so use it automatically.

Report a warning if the user specified a different language.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-01 16:52:37 +02:00
Noah Metzger
65997bed16 Fix issue detected by Coverity Scan
CID: 1340285 (Division or modulo by zero)

Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2018-08-01 15:56:19 +02:00
zdenop
b23568f3d1
Merge pull request #1816 from noahmetzger/winfix
Fix issues detected by Coverity Scan
2018-08-01 14:45:00 +02:00
Noah Metzger
d28631a274 Fix issues detected by Coverity Scan
CID: 1164604 (Nesting level does not match indentation)

Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2018-08-01 14:30:13 +02:00
Stefan Weil
6a28cce96b Fix whitespace issues
* Remove whitespace (blanks, tabs, cr) at line endings

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-01 13:19:52 +02:00
zdenop
3af2773d0e
Merge pull request #1814 from noahmetzger/winfix
Fix issue detected by Coverity Scan
2018-08-01 11:20:13 +02:00
Noah Metzger
2d96c66126 Fix issue detected by Coverity Scan
CID: 1164533 (Logically dead code)

Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2018-08-01 10:30:52 +02:00
Stefan Weil
eb69dd0201 TessPDFRenderer: Improve robustness of API (issue #1804)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-08-01 09:11:04 +02:00
Egor Pugin
9ce4d05188
Merge pull request #1812 from noahmetzger/winfix
Fix issue reported by Coverity Scan
2018-07-31 13:52:05 +03:00
Noah Metzger
d4490af06d Fix issue reported by Coverity Scan
CID: 1375395 (Dereference after null check)

Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2018-07-31 10:43:39 +02:00
zdenop
7d99cb4e28
Merge pull request #1811 from noahmetzger/winfix
Fix issue reported by Coverity Scan
2018-07-31 09:53:33 +02:00
Noah Metzger
83a4eb3b44 Fix issue reported by Coverity Scan
CID: 1391264 (Improper use of negative value)

Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2018-07-31 09:43:30 +02:00
Stefan Weil
9cf170cb7a Revert "Change default width for images output by text2image"
This reverts commit fdc243b363 because
it caused a regression reported in issue #1798.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-27 07:29:30 +02:00
Stefan Weil
b19e69086c training: Add new flag --workspace_dir to tesstraining_utils.sh
By default, that script creates two new temporary directories with random
names in /tmp.

The new command line flag --workspace_dir PATH uses the given path as
a base directory for all temporary files.

That allows better reproducable training results (no random directory
names in log files).

Signed-off-by: Stefan Weil <stweil@ub-backup.bib.uni-mannheim.de>
2018-07-26 17:14:19 +02:00
Noah Metzger
91c7504a35 Added a feature to enrich the hOCR output with glyph confidences
By using the parameter -c glyph_confidences=true the user is able to enrich
the hOCR output with additional information. Tesseract then lists additionally
the timesteps with all glyphs that were considered with their confidence
for every timestep of the LSTM.

The format of the hOCR output is slightly changed: There is now a linebreak
after every word for better readability by humans.

Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2018-07-25 18:18:58 +02:00
Stefan Weil
132c540c85 Increase limit for deserialization of large arrays
The last limit was still too small.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-21 11:10:09 +02:00
Stefan Weil
f577e292c2 Increase limit and add assertions for deserialization of large arrays
One of the checks was too restrictive, as lstmeval deserializes
char arrays with 14000000 elements, so raise the limit to 30000000.
That check was added in commit 992031e824.

Add also assertions which help finding such problems in debug mode.

Signed-off-by: Stefan Weil <stweil@ub-backup.bib.uni-mannheim.de>
2018-07-20 11:47:49 +02:00
Stefan Weil
ca25d88538 Add missing execute permission for script files
It is needed for running the training tutorial on Linux.

The correct mode was lost when moving the files in
commit 104fe7931c.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-19 20:25:41 +02:00
Stefan Weil
b7b8dba5db LSTMTrainer: Use new serialization API
Improve also portability by using int32_t instead of int
for a serialized member variable.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 19:28:05 +02:00
Stefan Weil
1dcda1aa8a LSTMRecognizer: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 19:28:05 +02:00
Stefan Weil
45a7ccf2d2 LSTM: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 19:28:05 +02:00
Stefan Weil
f4449ba41a Convolve: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 19:28:05 +02:00
Stefan Weil
dfc3e9691f SquishedDawg: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 19:28:05 +02:00
Stefan Weil
6cf508960a UnicharAndFonts, Shape: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:31:37 +02:00
Stefan Weil
07b363fec0 MasterTrainer: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:29:10 +02:00
Stefan Weil
88b3d940be TessdataManager: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:28:13 +02:00
Stefan Weil
da0217fa75 STRING: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:17:22 +02:00
Stefan Weil
5e05f2cb84 IndexMap: Use new serialization API and optimize code
By changing the type of sparse_size_ from int to int32_t,
a local copy can be removed.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:12:44 +02:00
Stefan Weil
edff1d1882 BitVector: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:07:03 +02:00
Stefan Weil
bb6c0123cc ICOORD: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:02:12 +02:00
Stefan Weil
66bc012d27 UNICHARSET: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 16:22:02 +02:00
Stefan Weil
eb90068b5f RecodedCharID: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 16:22:01 +02:00
Stefan Weil
0ca7cdd2c8 WordFeature, ImageData: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 16:22:01 +02:00
Stefan Weil
7133a6f43c GENERIC_2D_ARRAY: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 16:22:01 +02:00
Stefan Weil
ea660f83a3 fontinfo: Use new serialization API and optimize code
Combine several calls of Serialize in write_spacing_info and in write_set.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 16:22:01 +02:00
zdenop
daba37f4d4
Merge pull request #1784 from stweil/serialize
Simplify API for serialization and add first users
2018-07-18 15:54:05 +02:00
Stefan Weil
6ef267c432 Use TFile::Serialize, TFile::DeSerialize
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 11:19:37 +02:00
Stefan Weil
c383b1aaca TFile: Add helper functions for serialization of simple data types
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 11:19:37 +02:00
Stefan Weil
bdd2a7aedc Use tesseract::Serialize, tesseract::DeSerialize
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 11:19:37 +02:00
Stefan Weil
16832f9878 Add helper functions for serialization of simple data types
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 11:19:37 +02:00
Stefan Weil
216c2b31e7 Fix typo and add TODO comment
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 09:58:39 +02:00
Stefan Weil
2b6a356cba IntFeatureSpace: Remove unused DeSerialize method
The Serialize method is used indirectly by MasterTrainer::Serialize,
but there is no corresponding MasterTrainer::DeSerialize.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 09:56:43 +02:00
Stefan Weil
cfd72ff31e Fix --print-parameters (regression)
Commit 629ded223c had broken that
functionality.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-09 14:42:48 +02:00
Stefan Weil
88ebfea4b5 opencl: Fix crash and clean code a bit
OpenclDevice::getDeviceSelection crashed when outdated information
was read from file and device.score was not set.

Change also the struct definitions from C to C++ and
eliminate some type casts.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-08 18:38:11 +02:00
zdenop
e9cd6024d7
Merge pull request #1767 from stweil/unused
Remove unused macros and fix comments
2018-07-07 21:55:17 +02:00
Stefan Weil
0d4975933e Replace tprintf_internal by tprintf and clean tprintf code
Commit 4d514d5a60 introduced tprintf_internal
with an additional argument "level" which was removed again in commit
7dc5296fe9.

So we can now restore the original state without tprintf_internal.

Remove also the declaration of debug_window_on (it does not exist since
commit 030aae9896) and make the
configuration parameter debug_file local as it is only used by tprintf.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-07 21:47:10 +02:00
Stefan Weil
24bf8c334c Add include statements (needed for following commit)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-07 21:46:58 +02:00
Stefan Weil
8bd9567355 Fix some comments
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-07 21:19:01 +02:00
Stefan Weil
44415a706e Remove unused EXTERN macros
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-07 21:19:01 +02:00
Stefan Weil
7e80a850ad Remove unused macros
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-07 21:19:01 +02:00
Stefan Weil
609edd4600 Add missing include statement
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-07 21:19:00 +02:00
Stefan Weil
55f0ca5842 Add missing include statements and clean some include statements
The changes are based on an analysis done with include-what-you-use.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-07 16:24:53 +02:00
zdenop
71bb9c373b
Merge pull request #1761 from srdg/patch-1
Fixed "Invalid F spec" error while building FullyConnected layer.
2018-07-06 22:11:36 +02:00
Stefan Weil
0eb239ee8b Fix typo in comments
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 22:00:00 +02:00
Soumik Ranjan Dasgupta
001f536c16
Fixed "Invalid F spec" error while building FullyConnected layer.
`int depth = strtol(*str + 1, str, 10);`
`**str` holds the words in the VGSL specification, and `*str` holds a single word, lets say, `Fr64`. Now, the `strtol` function modifies `str` to point to the first character which a non-digit number, and assumes that ` *str+1 ` points to a number (of valid integer format) as a string (automatically skipping all the white spaces, and no other characters), where in reality, it seems to point to `r` in `Fr164`.This is a bad argument, which results in strtol returning 0.
` strtol (*str + 2, str, 10)` should be passed instead.
2018-07-06 23:54:22 +05:30
Stefan Weil
18edac4086 Fix CID 1164623 (Uninitialized scalar field)
Fix it by combining constructor and Init method.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 17:34:46 +02:00
Stefan Weil
66b71e6b58 Fix CID 1164621 (Uninitialized scalar field)
Format also some comments.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 17:12:12 +02:00
Stefan Weil
8582ee097c Fix CID 1164618 (Uninitialized scalar field)
Fix it by combining constructor and Init method.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 17:02:50 +02:00
Stefan Weil
37f82824f5 Fix CID 1164616 (Uninitialized scalar field)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 17:02:50 +02:00
Stefan Weil
430dc5c0b6 Fix CID 1164612 (Uninitialized scalar field)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 17:02:50 +02:00
Stefan Weil
19a99450bc Fix CID 1164611 (Uninitialized scalar field)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 17:02:50 +02:00
Stefan Weil
4bb41b8952 Fix CID 1164693 (Untrusted value as argument)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 16:11:29 +02:00
Stefan Weil
992031e824 Fix CID 1164702 (Untrusted value as argument)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 16:11:29 +02:00
Stefan Weil
c1da5fbac4 Fix CID 1164704 (Untrusted value as argument)
Limit the matrix to UINT16_MAX x UINT16_MAX.

Larger dimensions could also result in an arithmetic overflow
when multiplying the two dimensions.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 16:11:29 +02:00
Stefan Weil
8871f4d622 Fix CID 1164686 (Use of untrusted scalar value)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 16:11:29 +02:00
Stefan Weil
92e2ad0471 Fix CID 1164703 (Untrusted value as argument)
Wrong file data could give a large value for the number of vector elements
resulting in very large memory allocations.

Limit the allowed data range to UINT16_MAX (65535) elements
which hopefully should be sufficient for all use cases.

Changing the data type of the related member variables from int to
uint32_t allowed removing several type casts.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 16:11:10 +02:00
Egor Pugin
a078ce02bb
Merge pull request #1756 from stweil/cov
Fix two issues reported by Coverity Scan
2018-07-05 23:10:39 +03:00
Stefan Weil
bfe2a79502 Fix CID 1164566 (Dereference after null check)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-05 22:07:36 +02:00
Stefan Weil
036c72ca2f Fix CID 1164733 (Resource leak)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-05 22:07:36 +02:00
Stefan Weil
8d60a1849c Remove unused iterator
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-05 21:20:27 +02:00
Egor Pugin
3ea9cff149
Merge pull request #1752 from stweil/api
API fixes
2018-07-05 17:28:48 +03:00
Egor Pugin
09f4179e89
Merge pull request #1754 from stweil/fix
Fix compiler warnings [-Wmissing-prototypes]
2018-07-05 17:28:36 +03:00
Stefan Weil
d2febafdcd Fix compiler warnings [-Wmissing-prototypes]
Add missing include statements, add missing "static" qualifiers or
remove functions which are not used at all.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-05 16:03:02 +02:00
Stefan Weil
8ad7f049ea Fix empty statements
* Add break in default case to avoid potential problems with
  future case statements following the default case.

* Remove empty statement.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-05 15:17:54 +02:00