Commit Graph

3981 Commits

Author SHA1 Message Date
Stefan Weil
e14797563b Update documentation for supported languages
kur_ara.traineddata was renamed to kmr.traineddata.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-15 11:07:54 +01:00
Stefan Weil
85d7feebf7 Add missing documentation for --help-extra
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-15 09:36:10 +01:00
zdenop
0a36b38169
Merge pull request #2317 from eighttails/master
Added missing linker flags for MinGW.
2019-03-15 08:01:21 +01:00
Robert Schubert
14346e56b0 tesstrain: catch+handle SIGINT (to stop waiting on subjobs) 2019-03-15 00:03:16 +01:00
Robert Schubert
6cbad17e30 tesstrain: check all subjobs' retval 2019-03-14 14:38:51 +01:00
Robert Schubert
5316bcbb94 tesstrain: check failure of subjobs 2019-03-14 11:42:01 +01:00
Stefan Weil
4c2bbebecc Fix compiler warning (-Wunused-value)
Warning from clang++:

    ..\src\ccmain\ltrresultiterator.cpp(454,8):  warning: expression result unused [-Wunused-value]

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-13 20:56:03 +01:00
Stefan Weil
ed84ba0a44 Fix wrong comparison
symbol_steps is a vector, so testing for a nullptr was wrong.

clang++ reports:

    ..\src\ccmain\ltrresultiterator.cpp(440,19):  warning: comparison of address of 'this->word_res_->symbol_steps' equal to a null pointer is always false [-Wtautological-pointer-compare]
      if (&word_res_->symbol_steps == nullptr || !LSTM_mode_) return nullptr;
           ~~~~~~~~~~~^~~~~~~~~~~~    ~~~~~~~

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-13 20:38:38 +01:00
Tadahito Yao
bbbd262a8d Added missing linker flags for MinGW. 2019-03-13 22:10:36 +09:00
Stefan Weil
681e6301cd
Merge pull request #2316 from vidiecan/fix_accumulated_timesteps_check
`accumulated_timesteps` is not a pointer but a vector
2019-03-13 13:17:18 +01:00
jm server2
1206362d30 accumulated_timesteps is not a pointer but a vector and in case we use ChoiceIterator without lstm_choice_mode tesseract crashes (or similar) because the check is true and we reference not existing item 2019-03-13 12:55:14 +01:00
Stefan Weil
3baf0d8076 Fix boolean assignments
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-12 15:34:24 +01:00
zdenop
e965de77eb
Merge pull request #2314 from stweil/svpaint
Remove svpaint.cpp from libtesseract
2019-03-12 12:36:22 +01:00
Stefan Weil
8ad0489f0f Remove svpaint.cpp from libtesseract
svpaint is a standalone application (it includes a main function)
and should not be part of the Tesseract library.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-12 12:22:53 +01:00
zdenop
7546a01020
Merge pull request #2310 from noahmetzger/LSTMChoiceRIL
Lstm choice ril
2019-03-12 10:46:11 +01:00
zdenop
76fbade3ee
Merge pull request #2309 from stweil/fuzz
Fix several runtime errors (found by OSS-Fuzz)
2019-03-12 10:45:21 +01:00
Stefan Weil
35a999f91a Fix assertion caused by wrong unicharset
Credit to OSS-Fuzz: it found another case which triggered this assertion:

    contains_unichar_id(unichar_id):Error:Assert failed:in file ../../src/ccutil/unicharset.h, line 502

This is the OSS-Fuzz testcase:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=13662

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-12 09:31:21 +01:00
Stefan Weil
56a39bda77 Fix float division by zero
That runtime error is normally not visible because it does not abort
the program, but is detected when the code was compiled with sanitizers.

It can be triggered with this OSS-Fuzz testcase:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=13662

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-12 09:28:16 +01:00
Noah Metzger
5b3e2fe812 Integrated accumulated Symbol Choice in the Choice Iterator and made the api lstm_choice_mode independent
Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2019-03-12 09:15:10 +01:00
Stefan Weil
4c0b98bd12 Replace undefined shift operations by multiplications
Shift operations are undefined for negative numbers, but at least on
Intel they return the same value as a multiplication with 2 ^ shift value.

This fixes runtime errors reported by sanitizers and OSS-Fuzz:

    intmatcher.cpp:821:59: runtime error: left shift of negative value -14
    intmatcher.cpp:823:75: runtime error: left shift of negative value -512
    intmatcher.cpp:820:50: runtime error: left shift of negative value -80

See issue #2297 and
https://oss-fuzz.com/testcase-detail/4845195990925312 for details.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-12 06:56:54 +01:00
Stefan Weil
896698a4f5 Fix runtime error (left shift of negative value)
Runtime error:

    src/training/util.h:37:28: runtime error: left shift of negative value -17

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-12 06:56:54 +01:00
Egor Pugin
59cd716609
Merge pull request #2311 from stweil/global
Remove globals.h
2019-03-11 22:33:16 +03:00
Stefan Weil
5202208a8c Remove globals.h
It only included other files which are already included where needed.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-11 19:01:23 +01:00
Stefan Weil
e78b5f2af3 Update test submodule
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-11 13:00:17 +01:00
Noah Metzger
bc2b919805 Integrated Timesteps per symbol into ChoiceIterator
Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2019-03-11 10:50:56 +01:00
Noah Metzger
754e38d2b4 Added the option to get the timesteps separated by the suggested segmentation
Signed-off-by: Noah Metzger <noah.metzger@bib.uni-mannheim.de>
2019-03-11 10:50:56 +01:00
Egor Pugin
d2c3309df9
Update appveyor.yml 2019-03-11 02:12:13 +03:00
zdenop
e817607280 archive_version_details is available from libArchive version 3.2.0 2019-03-10 22:57:48 +01:00
Egor Pugin
c4dd537206 [cmake] Add visibility to all target_link_libraries calls. 2019-03-11 00:11:25 +03:00
Egor Pugin
b0f61dfd1c Propagate libarchive to tess users. 2019-03-11 00:06:50 +03:00
Egor Pugin
37b0c36e32 Add libarchive dependency to cppan and sw builds. 2019-03-11 00:03:45 +03:00
zdenop
5cfe4cc1f0
Merge pull request #2286 from Shreeshrii/lstmbox
Rename function to TessBaseAPIGetTsvText to be consistent to Create method
2019-03-10 21:41:52 +01:00
zdenop
02a1ffe87a Report libArchive support 2019-03-10 20:08:45 +01:00
zdenop
4ed44d70c5 cmake: enable libArchive support for non_cppan build 2019-03-10 20:08:19 +01:00
zdenop
e4bf971ad6
Merge pull request #2306 from stweil/fuzz
Fix two issues reported by OSS-Fuzz
2019-03-10 19:24:42 +01:00
Stefan Weil
b3aff7d633 Fix Index-out-of-bounds in IntegerMatcher::UpdateTablesForFeature
This fixes issue #2299, an issue which was already reported by
static code analyzers and now by OSS-Fuzz, see details at
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=13597.

The Tesseract code assigns an address which is out-of-bounds to a pointer
variable, but increments that variable later. So this is a false positive.

Change the code nevertheless to satisfy OSS-Fuzz.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-10 18:26:40 +01:00
Stefan Weil
91d0a71d51 Fix assertion caused by wrong unicharset (issue #2301)
Credit to OSS-Fuzz:
This fixes an issue which was reported by OSS-Fuzz, see details at
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=13592.

OSS-Fuzz triggered this assertion:

    contains_unichar_id(unichar_id):Error:Assert failed:in file ../../src/ccutil/unicharset.h, line 502

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-10 16:42:54 +01:00
zdenop
0e72733121
Merge pull request #2305 from stweil/fuzz
Fix Heap-buffer-overflow in GenericVector<int>::size (issue #2298)
2019-03-10 16:36:26 +01:00
Stefan Weil
71d4990c6d Fix Heap-buffer-overflow in GenericVector<int>::size (issue #2298)
Credit to OSS-Fuzz:
This fixes a security issue which was reported by OSS-Fuzz, see details at
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=13590.

Add also some assertions to catch similar bugs.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-10 16:12:30 +01:00
Robert Schubert
3912cb1c33 LSTM char_whitelist/blacklist (6ac2ff0): more robust
- unicharset can be null too
2019-03-09 10:40:40 +01:00
Stefan Weil
b7279f6d67 unittest: Remove tmp directory from repository and create it during build
This fixes out of tree builds.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-08 16:08:16 +01:00
Stefan Weil
bd95c9d2b8 unittest: Add missing libarchive
It is needed for the tests if Tesseract was built with libarchive.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-08 15:50:14 +01:00
Stefan Weil
b20f89006e unittest: Add another file from Abseil
It is needed for newer versions of Abseil.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2019-03-08 15:46:38 +01:00
Robert Schubert
b45999088c LSTM char_whitelist/blacklist (6ac2ff0): multi-code chars
- move decision from ComputeTopN to ContinueContext, where
  it belongs: block context continuations which emit final
  codes translating to disabled unichar_ids.
  (The normal logic for fallback from top2 > top2 > rest
   will apply.)
- pass UNICHARSET refs appropriately
2019-03-08 12:30:16 +01:00
Robert Schubert
8012d5e653 LSTM char_whitelist/blacklist (6ac2ff0): also sublangs 2019-03-07 18:32:50 +01:00
Robert Schubert
6ac2ff083e trying to add tessedit_char_whitelist etc. again:
- ignore matrix outputs in ComputeTopN if they
  belong to a disabled unichar_id
- pass UNICHARSET refs to check that
- in SetBlackAndWhitelist, also update the unicharset
  of the lstm_recognizer_ instance, if any
2019-03-07 01:37:23 +01:00
zdenop
f80085c0bf
Merge pull request #2289 from Armyke/master
Added an additional optional --tmp_dir parameter to specify the tempo…
2019-03-06 15:03:14 +01:00
zdenop
fe5c82fd24
Merge pull request #2291 from cjmayo/man_configfile
Document that configfile can be a file path
2019-03-06 10:19:27 +01:00
Chris Mayo
a9d3efb6e3 Document that configfile can be a file path
Useful for custom config or when pointing tessdata to alternate
traineddata.
2019-03-05 19:47:54 +00:00
zdenop
868a623f8d
Merge pull request #2290 from stweil/libarchive
Add initial support for traineddata files in standard archive formats
2019-03-05 17:42:13 +01:00