Commit Graph

166 Commits

Author SHA1 Message Date
Stefan Weil
132c540c85 Increase limit for deserialization of large arrays
The last limit was still too small.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-21 11:10:09 +02:00
Stefan Weil
f577e292c2 Increase limit and add assertions for deserialization of large arrays
One of the checks was too restrictive, as lstmeval deserializes
char arrays with 14000000 elements, so raise the limit to 30000000.
That check was added in commit 992031e824.

Add also assertions which help finding such problems in debug mode.

Signed-off-by: Stefan Weil <stweil@ub-backup.bib.uni-mannheim.de>
2018-07-20 11:47:49 +02:00
Stefan Weil
88b3d940be TessdataManager: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:28:13 +02:00
Stefan Weil
da0217fa75 STRING: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:17:22 +02:00
Stefan Weil
5e05f2cb84 IndexMap: Use new serialization API and optimize code
By changing the type of sparse_size_ from int to int32_t,
a local copy can be removed.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:12:44 +02:00
Stefan Weil
edff1d1882 BitVector: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 17:07:03 +02:00
Stefan Weil
66bc012d27 UNICHARSET: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 16:22:02 +02:00
Stefan Weil
eb90068b5f RecodedCharID: Use new serialization API
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 16:22:01 +02:00
Stefan Weil
c383b1aaca TFile: Add helper functions for serialization of simple data types
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 11:19:37 +02:00
Stefan Weil
16832f9878 Add helper functions for serialization of simple data types
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-18 11:19:37 +02:00
zdenop
e9cd6024d7
Merge pull request #1767 from stweil/unused
Remove unused macros and fix comments
2018-07-07 21:55:17 +02:00
Stefan Weil
0d4975933e Replace tprintf_internal by tprintf and clean tprintf code
Commit 4d514d5a60 introduced tprintf_internal
with an additional argument "level" which was removed again in commit
7dc5296fe9.

So we can now restore the original state without tprintf_internal.

Remove also the declaration of debug_window_on (it does not exist since
commit 030aae9896) and make the
configuration parameter debug_file local as it is only used by tprintf.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-07 21:47:10 +02:00
Stefan Weil
8bd9567355 Fix some comments
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-07 21:19:01 +02:00
Stefan Weil
7e80a850ad Remove unused macros
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-07 21:19:01 +02:00
Stefan Weil
0eb239ee8b Fix typo in comments
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 22:00:00 +02:00
Stefan Weil
4bb41b8952 Fix CID 1164693 (Untrusted value as argument)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 16:11:29 +02:00
Stefan Weil
992031e824 Fix CID 1164702 (Untrusted value as argument)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 16:11:29 +02:00
Stefan Weil
8871f4d622 Fix CID 1164686 (Use of untrusted scalar value)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 16:11:29 +02:00
Stefan Weil
92e2ad0471 Fix CID 1164703 (Untrusted value as argument)
Wrong file data could give a large value for the number of vector elements
resulting in very large memory allocations.

Limit the allowed data range to UINT16_MAX (65535) elements
which hopefully should be sufficient for all use cases.

Changing the data type of the related member variables from int to
uint32_t allowed removing several type casts.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-06 16:11:10 +02:00
Stefan Weil
d2febafdcd Fix compiler warnings [-Wmissing-prototypes]
Add missing include statements, add missing "static" qualifiers or
remove functions which are not used at all.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-05 16:03:02 +02:00
Stefan Weil
bdf09f40b1 Fix compiler warnings [-Wzero-as-null-pointer-constant]
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-04 20:40:56 +02:00
Stefan Weil
c8b5a29ce9 Remove unneeded type casts
This removes unneded type casts to (char*) and (const char*).

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-04 14:23:55 +02:00
Stefan Weil
6d170a15ec Replace tabs by blanks in source code
blobs.cpp had many tabs and was formatted with clang-format.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-03 16:29:14 +02:00
Stefan Weil
626a229cac Remove nwmain.h
The macro DECLARE_MAIN is not used by the current Tesseract code.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-03 15:54:41 +02:00
Stefan Weil
faae87beaa Replace FLOAT32 by float data type
On most systems float is the IEEE 754 single-precision binary
floating-point format (32 bits). Tesseract does not support other systems.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-02 13:29:39 +02:00
Stefan Weil
f6c3c8cf4d Replace MAX_FLOAT32 by standard FLT_MAX and remove unused MIN_FLOAT32
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-02 13:29:39 +02:00
Stefan Weil
919901eb19 Replace FLOAT64 by double data type
On most systems double is the IEEE 754 double-precision binary
floating-point format (64 bits). Tesseract does not support other systems.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-07-02 08:07:37 +02:00
Stefan Weil
9bb5a87760 Remove stderr.h and its include statements
MEMORY_OUT is no longer used.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-25 16:14:20 +02:00
Stefan Weil
db7f2009d9 Remove memry.cpp, memry.h
The proprietary memory allocators alloc_string, alloc_mem
are no longer used.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-25 16:13:59 +02:00
Stefan Weil
20e243d5c9 strngs: Replace alloc_mem, free_mem by standard functions
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-25 14:31:48 +02:00
Stefan Weil
91522dfba5 Remove memry.h from public API
It is no longer needed by genericvector.h.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-23 21:15:54 +02:00
Stefan Weil
1a151781ea Clean some include statements
The changes are based on an analysis done with include-what-you-use.

Replace also some standard header files by the corresponding
standard C++ header files.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-23 21:15:54 +02:00
Stefan Weil
484a1be98a Remove unneeded include statements for scanutils.h
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-22 19:16:08 +02:00
Stefan Weil
1371980f9f Replace string.h by standard C++ cstring
Remove the unneeded include statement in platform.h.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-21 20:40:26 +02:00
Stefan Weil
112aeb9826 Clean usage of assert.h
Remove unneeded include statements, remove conditional statements and
replace the remaining assert.h by their standard C++ variant cassert.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-21 19:31:05 +02:00
Stefan Weil
a9e2574eff Remove public API file ndminx.h
It is not needed for the Tesseract code, and the Tesseract API
should not provide MIN / MAX macros.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-21 08:33:30 +02:00
Stefan Weil
0cb128d56b Remove errcode.h from public API
It is no longer needed by genericvector.h.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-21 06:20:26 +02:00
Stefan Weil
44450094c3 Replace ASSERT_HOST in genericvector.h
genericvector.h used a mix of assert and ASSERT_HOST.

By using assert only, it does no longer depend on errcode.h
which defines the ASSERT_HOST macro.

Other files which still use ASSERT_HOST now need an explicit
include statement for errcode.h.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-20 22:32:17 +02:00
Stefan Weil
d6391ee811 Fix CID 1393540 (Explicit null dereferenced)
Coverity Scan does not like incrementing of a null pointer,
so increment an index value instead of a pointer.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-20 17:32:02 +02:00
Stefan Weil
e87e8967d7 Remove more header files from public API
Install only those headers which are needed by third party applications.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-20 11:54:38 +02:00
Stefan Weil
c1c87d73ee Require tesseract/ for API header files (fixes potential name conflicts)
The tesseract/ subdirectory is no longer automatically added to the
include path of the compiler. Therefore old code which used code like

    #include "capi.h"

must now change that to

    #include "tesseract/capi.h"

This avoids name conflicts with header files from other projects.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-17 22:01:19 +02:00
Stefan Weil
02277bed34 scanutils: Fix typos in comments
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-09 07:53:20 +02:00
Stefan Weil
280db06bbf scanutils: Fix illegal memory access
Format strings which contain "%*s" show this error in Valgrind:

==32503== Conditional jump or move depends on uninitialised value(s)
==32503==    at 0x2B8BB0: tvfscanf(_IO_FILE*, char const*, __va_list_tag*) (scanutils.cpp:486)
==32503==    by 0x2B825A: tfscanf(_IO_FILE*, char const*, ...) (scanutils.cpp:234)
==32503==    by 0x272B01: read_unlv_file(STRING, int, int, BLOCK_LIST*) (blread.cpp:54)
==32503==    by 0x1753CD: tesseract::Tesseract::SegmentPage(STRING const*, BLOCK_LIST*, tesseract::Tesseract*, OSResults*) (pagesegmain.cpp:115)
==32503==    by 0x1363CD: tesseract::TessBaseAPI::FindLines() (baseapi.cpp:2291)
==32503==    by 0x130CF1: tesseract::TessBaseAPI::Recognize(ETEXT_DESC*) (baseapi.cpp:802)
==32503==    by 0x1322D3: tesseract::TessBaseAPI::ProcessPage(Pix*, int, char const*, char const*, int, tesseract::TessResultRenderer*) (baseapi.cpp:1176)
==32503==    by 0x131A84: tesseract::TessBaseAPI::ProcessPagesMultipageTiff(unsigned char const*, unsigned long, char const*, char const*, int, tesseract::TessResultRenderer*, int) (baseapi.cpp:1013)
==32503==    by 0x132052: tesseract::TessBaseAPI::ProcessPagesInternal(char const*, char const*, int, tesseract::TessResultRenderer*) (baseapi.cpp:1129)
==32503==    by 0x131B1E: tesseract::TessBaseAPI::ProcessPages(char const*, char const*, int, tesseract::TessResultRenderer*) (baseapi.cpp:1032)
==32503==    by 0x12E00C: main (tesseractmain.cpp:537)
==32503==  Uninitialised value was created by a stack allocation
==32503==    at 0x272A60: read_unlv_file(STRING, int, int, BLOCK_LIST*) (blread.cpp:41)

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-08 15:28:30 +02:00
zdenop
ee2ab73224
Merge pull request #1637 from paulk124/master
Reserve extra byte in LoadDataFromFile() in case caller wants to appe…
2018-06-05 16:57:40 +02:00
Paul Kitchen
805fb7699d Reserve extra byte in LoadDataFromFile() in case caller wants to append '\0' 2018-06-05 08:19:41 -06:00
Stefan Weil
52fddc3ca9 TFile: Relax assertion and allow FRead, FWrite with count == 0
The assertions introduced by commit 8bea6bcc12
were too strict. The first one failed in osd_test, the second one failed
in `tesseract IMAGE BASE --psm 13 lstm.train`.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-04 22:42:19 +02:00
Stefan Weil
8bea6bcc12 TFile: Improve handling of potential integer overflow
Raise an assertion for unexpected arguments and use size_t instead of int
for the size argument which is typically sizeof(some_datatype).

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-06-04 13:53:36 +02:00
Jaroslaw Kubik
e254c9fa38 Return a value from default progress report function
The progress reporting function returns a boolean. The returned
value is never used by the tesseract and its meaing is not
documented, which renders the value meaningless. Still, lack of
return should not be premitted.
2018-05-30 10:44:13 +02:00
Jaroslaw Kubik
217e5e5881 Add a context-aware progress monitor pointer
The progress_callback field in the ETEXT_DESC monitor type does not
take any 'context' parameter, which may make implementing callback
functions difficult and may require use of global variables.
The new function receives the ETEXT_DESC pointer as an argument.
This makes it possible to share the cancel_this field as context
carrier if required.
The change is backwards-compatible: the old pointer remains as a
member of the class, and the default value for the new pointer is
a function calling the classic progress notifier. This way the code
unaware of the new member will continue to work as before.
2018-05-29 21:48:51 +02:00
Stefan Weil
509a6f0ce0 Fix some typos (most found by codespell)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-05-27 18:49:43 +02:00
Egor Pugin
5a56d0c291
Merge pull request #1588 from ZaMaZaN4iK/fix_own_bool
Use standard bool instead of BOOL8.
2018-05-23 16:53:12 +03:00
Alexander Zaitsev
785b5e8134 Use default keyword instead of empty ctors/dtors. 2018-05-21 13:35:46 +03:00
Alexander Zaitsev
6ff0b56597 More fixes BOOL8 -> bool 2018-05-21 00:40:58 +03:00
Alexander Zaitsev
a040bc2da5 Use standard bool instead of BOOL8. 2018-05-20 22:46:46 +03:00
Alexander Zaitsev
d54d7486b4 Use std::max/std::min instead of MAX/MIN macros. 2018-05-20 17:49:48 +03:00
Alexander Zaitsev
c34e145b1a Use numeric_limits instead of INT_MAX. 2018-05-20 14:49:35 +03:00
Alexander Zaitsev
7d08e117d8 Added more const. 2018-05-20 14:21:07 +03:00
Alexander Zaitsev
0697235bb2 Use using instead of typedef. Reason: https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#Rt-using 2018-05-20 01:31:03 +03:00
Alexander Zaitsev
0248c7ff9d Rename all C-style headers (e.g. <stdio.h>) to C++ style (<cstdio>). 2018-05-20 00:52:04 +03:00
Robert Clayton
684e875612
Update Makefile.am 2018-05-19 01:42:57 -05:00
Stefan Weil
c9b585cfc5 Don't disable compiler warnings for Visual Studio
It's still possible to set the warning level in the project settings,
but single source files should normally not disable compiler warnings.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-05-03 14:26:20 +02:00
Stefan Weil
9c1fe092f1 Add assertion to detect wrong endianness handling
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-05-03 07:18:55 +02:00
Stefan Weil
e2135de022 Fix CID 1385633 (Dereference before null check)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-04-26 17:18:15 +02:00
Stefan Weil
dabf3c299f Fix file endings
Text files should end with a LF, but not additional empty lines.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-04-25 19:35:33 +02:00
Stefan Weil
9ceb0c6430 Fix line endings
Replace DOS line endings (CRLF) by standard (LF only).

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2018-04-25 19:04:50 +02:00
Egor Pugin
e95ff1159e Move sources into src dir. Update build scripts. 2018-04-25 11:02:54 +03:00