Commit Graph

399 Commits

Author SHA1 Message Date
costan
6caf73ad9d Clean up Iterator.
This CL renames the private struct Iterator::Cleanup ->
Iterator::CleanupNode, to better reflect that it's a linked list node,
and extracts duplicated code from its user in IsEmpty() and Run()
methods.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199175058
2018-06-04 17:24:44 -07:00
cmumford
6a6bdafcf1 Corrected typo in docs: "cache" to "block_cache".
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197452015
2018-06-04 16:14:34 -07:00
costan
1868398150 Clean up SnapshotImpl.
* Omit SnapshotImpl::list_ when assert() isn't on
* Make SnapshotImpl::number_ const and set it in the constructor
* Make SnapshotImpl::number_ private and access it via a getter
* Rename SnapshotImpl::number_ to SnapshotImpl::sequence_number_
* Rename SnapshotList::list_ to SnapshotList::head_
* Wrap casting from Snapshot* to SnapshotImpl* in ToSnapshotImpl()

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194852828
2018-04-30 16:01:39 -07:00
cmumford
e7840de9f3 Fix documentation for log file growth.
This fixes #546 reported on GitHub.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194549692
2018-04-30 15:50:26 -07:00
cmumford
bc23e00f95 Update default log file size in doc.
The default size was changed in #f779e7a5 but the documentation was
never updated.

This fixes #566 reported on GitHub.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194547959
2018-04-30 15:49:58 -07:00
costan
4de9594f6f Add move constructor to Status.
This will result in smaller code generation when Status instances are
passed around.

Benchmarks don't indicate a significant change either way.
CPU:        48 * Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz
CPUCache:   30720 KB
Keys:       16 bytes each
Values:     100 bytes each (50 bytes after compression)
Entries:    1000000
RawSize:    110.6 MB (estimated)
FileSize:   62.9 MB (estimated)

Baseline:
fillseq      :       3.589 micros/op;   30.8 MB/s
fillsync     :    4165.299 micros/op;    0.0 MB/s (1000 ops)
fillrandom   :       5.864 micros/op;   18.9 MB/s
overwrite    :       7.830 micros/op;   14.1 MB/s
readrandom   :       5.534 micros/op; (1000000 of 1000000 found)
readrandom   :       4.292 micros/op; (1000000 of 1000000 found)
readseq      :       0.312 micros/op;  354.1 MB/s
readreverse  :       0.501 micros/op;  220.8 MB/s
compact      :  886211.000 micros/op;
readrandom   :       3.518 micros/op; (1000000 of 1000000 found)
readseq      :       0.251 micros/op;  441.2 MB/s
readreverse  :       0.456 micros/op;  242.4 MB/s
fill100K     :    1329.723 micros/op;   71.7 MB/s (1000 ops)
crc32c       :       1.976 micros/op; 1976.7 MB/s (4K per op)
snappycomp   :       4.705 micros/op;  830.2 MB/s (output: 55.1%)
snappyuncomp :       0.958 micros/op; 4079.1 MB/s
acquireload  :       0.727 micros/op; (each op is 1000 loads)

New:
fillseq      :       3.129 micros/op;   35.4 MB/s
fillsync     :    2748.099 micros/op;    0.0 MB/s (1000 ops)
fillrandom   :       5.394 micros/op;   20.5 MB/s
overwrite    :       7.253 micros/op;   15.3 MB/s
readrandom   :       5.655 micros/op; (1000000 of 1000000 found)
readrandom   :       4.425 micros/op; (1000000 of 1000000 found)
readseq      :       0.298 micros/op;  371.3 MB/s
readreverse  :       0.508 micros/op;  217.9 MB/s
compact      :  885842.000 micros/op;
readrandom   :       3.545 micros/op; (1000000 of 1000000 found)
readseq      :       0.252 micros/op;  438.2 MB/s
readreverse  :       0.425 micros/op;  260.2 MB/s
fill100K     :    1418.347 micros/op;   67.2 MB/s (1000 ops)
crc32c       :       1.987 micros/op; 1966.0 MB/s (4K per op)
snappycomp   :       4.767 micros/op;  819.4 MB/s (output: 55.1%)
snappyuncomp :       0.916 micros/op; 4264.9 MB/s
acquireload  :       0.665 micros/op; (each op is 1000 loads)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194002392
2018-04-23 16:22:30 -07:00
costan
d177a0263c Replace port_posix with port_stdcxx.
The porting layer implements threading primitives: atomic pointers,
condition variables, mutexes, thread-safe initialization. These are all
specified in C++11, so the reference open source port implementation can
become platform-independent.

The porting layer will remain in place to allow the use of other
implementations with more features, such as the built-in deadlock
detection in abseil's Mutex.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193245934
2018-04-17 13:26:47 -07:00
MarcoFalke
14cce848e7 Fix sign mismatch warnings in GCC.
This was contributed in https://github.com/google/leveldb/pull/492

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193080913
2018-04-16 18:13:09 -07:00
costan
8046a51b21 Add forgotten <limits> header to util/logging.cc.
Commit a0008deb67 introduced
std::numeric_limits usage in logging.cc, but didn't #include <limits>

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192840190
2018-04-13 16:21:07 -07:00
costan
a0008deb67 Reimplement ConsumeDecimalNumber.
The old implementation caused odd crashes on ARM, which were fixed by
changing a local variable type. The main suspect is the use of a static
local variable. This CL replaces the static local variable with
constexpr, which still ensures the compiler sees the expressions as
constants.

The CL also replaces Slice operations in the functions' inner loop with
iterator-style pointer operations, which can help the compiler generate
less code.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192832175
2018-04-13 15:37:20 -07:00
costan
1f7dd5d5f6 Add tests for ConsumeDecimalNumber.
ConsumeDecimalNumber has fairly non-trivial logic, and a previous
version has crashed inexplicably on Android. Having some test coverage
will make it easier to tweak / simplify the function later on.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192821751
2018-04-13 15:36:55 -07:00
costan
1cc8b10b82 Document the building process.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192490601
2018-04-11 12:57:42 -07:00
costan
09217fd067 Replace NULL with nullptr in C++ files.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192365747
2018-04-10 16:26:43 -07:00
costan
6a3b915166 Remove PLATFORM_IS_LITTLE_ENDIAN from port/posix.h.
This is an accidental leftover from the CMake migration. The macro has
been replaced with LEVELDB_IS_BIG_ENDIAN.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192364918
2018-04-10 16:26:27 -07:00
costan
260655b4c2 Define LEVELDB_HAS_PORT_CONFIG_H for old compilers.
The CMake-based build relies on __has_include, which is standardized in
C++17. Unfortunately, __has_include is available without requiring
--std=c++17 on all the compilers on CI, so this problem was not caught.

Fixes https://github.com/google/leveldb/issues/572

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192208842
2018-04-09 16:56:50 -07:00
costan
6fa4566670 Rename CMake project / targets from Leveldb to leveldb.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190262375
2018-03-23 13:53:50 -07:00
costan
0db30413a4 leveldb: Add more thread safety annotations.
After this CL, all classes with Mutex members should be covered by annotations. Exceptions are atomic members, which shouldn't need locking, and DBImpl members that cause errors when annotated, which will be tackled separately.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190260865
2018-03-23 12:56:14 -07:00
costan
04f39105c5 Take <atomic> for granted in port/atomic_pointer.h.
C++11 requires <atomic>. This lets us remove the header detection
(LEVELDB_ATOMIC_PRESENT) and simplify port/atomic_pointer.h.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189919098
2018-03-21 09:40:40 -07:00
costan
74f032ff6f leveldb: Require C++11.
This CL switches the public headers to C++11 default and deleted constructors, and adds override to the relevant leveldb::EnvWrapper methods. This should be a good test for C++11 compiler support.

Once this CL settles, the rest of the codebase can be safely modernized to C++11.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189873212
2018-03-21 01:17:59 -07:00
costan
8e75db8623 Remove build configuration for make.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189824264
2018-03-20 16:50:13 -07:00
Hyzeta
df9a841a4f Add export.h to CMakeLists.txt
The header is missing from the file lists, resulting in incomplete installs.

This was originally contributed in https://github.com/google/leveldb/pull/568.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189822907
2018-03-20 16:48:58 -07:00
costan
50fbc87e8c Replace SIZE_MAX with std::numeric_limits.
helpers/memenv/memenv.cc used SIZE_MAX without including <stdint.h>.
Since we're fixing this problem, replace SIZE_MAX with
std::numeric_limits<size_t>::max(), which is clearer.

Fixes https://github.com/google/leveldb/issues/562

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189821707
2018-03-20 16:48:42 -07:00
costan
739c25100e Add CMake build support.
Fixes https://github.com/google/leveldb/issues/466

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189425354
2018-03-16 19:17:27 -07:00
costan
0fa5a4f7b1 Extend thread safety annotations.
This CL makes it easier to reason about thread safety by:

1) Adding Clang thread safety annotations according to comments.
2) Expanding a couple of variable names, without adding extra lines of code.
3) Adding const in a couple of places.
4) Replacing an always-non-null const pointer with a reference.
5) Fixing style warnings in the modified files.

This CL does not annotate the DBImpl members that claim to be protected
by the instance mutex, but are accessed without the mutex being held.
Those members (and their unprotected accesses) will be addressed in
future CLs.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189354657
2018-03-16 10:32:40 -07:00
costan
8143c12f3f Fix includes in util/testharness.h.
This CL removes unused headers included by util/testharness.h, adds
precise includes where the build breaks, and fixes style errors in the
edited files.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189331061
2018-03-16 10:31:48 -07:00
costan
aece2068d7 Remove extern from function declarations.
External linkage is the default for function declarations in C++.

This also fixes ClangTidy errors generated by removing the "extern"
keyword as described above.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188730416
2018-03-12 09:24:48 -07:00
costan
ddab751002 Add tests for {Old}InfoLogFileName().
This change was contributed by GitHub user @LopatkinEvgeniy in
https://github.com/google/leveldb/pull/559.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188728845
2018-03-12 09:24:25 -07:00
costan
7fd7c00721 Remove unused function ExtractValueType.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188728505
2018-03-12 09:24:07 -07:00
costan
594cc987af Bypass OSMemoryBarrier() warning on Mac.
This is a stopgap for removing warnings on Mac builds, so -Werror can be
turned on. C++11 will be required in the nearby future, which guarantees
<atomic> support. Once that happens, the simplified version of this will
match https://github.com/google/leveldb/pull/503

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188553251
2018-03-09 16:37:44 -08:00
costan
49f35d3fc9 leveldb: Update Travis CI configuration for open source build.
This aligns the Travis CI configuration with google/crc32c and
google/snappy, to simplify maintenance.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188547648
2018-03-09 15:49:36 -08:00
costan
623d014a54 Expose Env::GetTempDirectory() for use in C test.
This removes the use of the non-portable headers <sys/types.h> and <unistd.h> in c_test.c.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188503102
2018-03-09 10:38:04 -08:00
costan
8c8024ea33 Switch HAVE_ library detection macros to 0/1.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188488298
2018-03-09 09:34:42 -08:00
costan
41172a2401 Enable thread safety annotations in open source version.
The thread safety annotations used by leveldb got opensourced in Abseil
[1]. This CL replaces leveldb's stubs with the relevant definitions from
[1], and adds annotations to the Mutex classes in the POSIX port.

[1] https://github.com/abseil/abseil-cpp/blob/master/absl/base/thread_annotations.h
2018-02-13 22:40:41 -08:00
cmumford
47cb9e2a21 Add leveldb_options_set_max_file_size to the C API.
When the max file size option was added in CL 134391640 the C API
was not modified to support this.

This change was contributed by GitHub user @olt and fixes issue #439.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173466388
2017-11-03 15:04:26 -07:00
cmumford
b5d4a22e64 Fixed style guide link in CONTRIBUTING.md
The C++ style guide URL was wrong.

This fixes issue #394. Reported by GitHub user @Loki-Astari.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173188573
2017-11-03 15:04:08 -07:00
cmumford
3da4d8b989 Deleted unused assignments in Reader.
Deleted two unused assignments:

1. offset_in_block in Reader::SkipToInitialBlock().
2. in_fragmented_record in Reader::ReadRecord().

Reasons for the change:
1. offset_in_block is not read again after the if condition.
2. The kFullRecordType switch branch returns, so
   in_fragmented_record isn't read again.
3. The kFirstType switch branch sets in_fragmented_record to
   true after the if, so the write in the if is ignored.

Change contributed by @C0deAi on GitHub.

This fixes https://github.com/google/leveldb/issues/517

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172763897
2017-11-03 15:03:44 -07:00
cmumford
0509414f85 leveldb::DestroyDB will now delete empty directories.
Env's that filtered out dot files ("." and "..") would return an
empty vector of children causing DestroyDB to do nothing. This fixes
https://github.com/google/leveldb/issues/215

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172501335
2017-11-03 15:03:20 -07:00
costan
23162ca1c6 Fix typo (forgotten reference operator) in test.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171708408
2017-10-10 11:47:13 -07:00
costan
5c39524f36 Replace SSE-optimized CRC32C in POSIX port with external library.
Maintaining a hardware-accelerated CRC32C implementation tailored for
all modern platforms deserves a repository of its own. We extracted the
implementation here into https://github.com/google/crc32c and improved
it in that repository. This CL removes the SSE-optimized implementation
from this codebase, and adds the ability to use the google/crc32c
library, if it is present on the system.

The benchmarks below show the performance impact of the change. In
summary, open source builds that use the google/crc32c library can
expect a 3x improvement in CRC32C throughput, whereas builds that do not
use the library will see a 50% drop in CRC32C throughput. This
translates in much smaller changes in overall leveldb performance.

Baseline, MacBookPro13,3 with Core i7 6920HQ:
LevelDB:    version 1.20
Keys:       16 bytes each
Values:     100 bytes each (50 bytes after compression)
Entries:    1000000
RawSize:    110.6 MB (estimated)
FileSize:   62.9 MB (estimated)
------------------------------------------------
fillseq      :       3.064 micros/op;   36.1 MB/s
fillsync     :      57.861 micros/op;    1.9 MB/s (1000 ops)
fillrandom   :       3.887 micros/op;   28.5 MB/s
overwrite    :       4.140 micros/op;   26.7 MB/s
readrandom   :       7.433 micros/op; (1000000 of 1000000 found)
readrandom   :       6.825 micros/op; (1000000 of 1000000 found)
readseq      :       0.244 micros/op;  453.4 MB/s
readreverse  :       0.387 micros/op;  285.8 MB/s
compact      :  449707.000 micros/op;
readrandom   :       4.196 micros/op; (1000000 of 1000000 found)
readseq      :       0.228 micros/op;  485.8 MB/s
readreverse  :       0.320 micros/op;  345.2 MB/s
fill100K     :     562.556 micros/op;  169.6 MB/s (1000 ops)
crc32c       :       0.768 micros/op; 5085.0 MB/s (4K per op)
snappycomp   :       4.220 micros/op;  925.7 MB/s (output: 55.1%)
snappyuncomp :       0.635 micros/op; 6155.7 MB/s
acquireload  :      13.054 micros/op; (each op is 1000 loads)

New with crc32c, MacBookPro13,3 with Core i7 6920HQ:
LevelDB:    version 1.20
Keys:       16 bytes each
Values:     100 bytes each (50 bytes after compression)
Entries:    1000000
RawSize:    110.6 MB (estimated)
FileSize:   62.9 MB (estimated)
------------------------------------------------
fillseq      :       2.820 micros/op;   39.2 MB/s
fillsync     :      51.988 micros/op;    2.1 MB/s (1000 ops)
fillrandom   :       3.747 micros/op;   29.5 MB/s
overwrite    :       4.047 micros/op;   27.3 MB/s
readrandom   :       7.287 micros/op; (1000000 of 1000000 found)
readrandom   :       6.927 micros/op; (1000000 of 1000000 found)
readseq      :       0.253 micros/op;  437.5 MB/s
readreverse  :       0.411 micros/op;  269.2 MB/s
compact      :  440405.000 micros/op;
readrandom   :       4.159 micros/op; (1000000 of 1000000 found)
readseq      :       0.230 micros/op;  481.1 MB/s
readreverse  :       0.320 micros/op;  345.9 MB/s
fill100K     :     558.222 micros/op;  170.9 MB/s (1000 ops)
crc32c       :       0.214 micros/op; 18263.5 MB/s (4K per op)
snappycomp   :       4.471 micros/op;  873.7 MB/s (output: 55.1%)
snappyuncomp :       0.833 micros/op; 4688.5 MB/s
acquireload  :      13.289 micros/op; (each op is 1000 loads)

New without crc32c, MacBookPro13,3 with Core i7 6920HQ
LevelDB:    version 1.20
Keys:       16 bytes each
Values:     100 bytes each (50 bytes after compression)
Entries:    1000000
RawSize:    110.6 MB (estimated)
FileSize:   62.9 MB (estimated)
------------------------------------------------
fillseq      :       3.094 micros/op;   35.8 MB/s
fillsync     :      52.160 micros/op;    2.1 MB/s (1000 ops)
fillrandom   :       4.090 micros/op;   27.0 MB/s
overwrite    :       4.006 micros/op;   27.6 MB/s
readrandom   :       6.584 micros/op; (1000000 of 1000000 found)
readrandom   :       6.676 micros/op; (1000000 of 1000000 found)
readseq      :       0.280 micros/op;  395.2 MB/s
readreverse  :       0.391 micros/op;  283.2 MB/s
compact      :  433911.000 micros/op;
readrandom   :       4.261 micros/op; (1000000 of 1000000 found)
readseq      :       0.251 micros/op;  440.5 MB/s
readreverse  :       0.356 micros/op;  310.9 MB/s
fill100K     :     584.023 micros/op;  163.3 MB/s (1000 ops)
crc32c       :       1.384 micros/op; 2822.3 MB/s (4K per op)
snappycomp   :       4.763 micros/op;  820.1 MB/s (output: 55.1%)
snappyuncomp :       0.766 micros/op; 5098.6 MB/s
acquireload  :      12.931 micros/op; (each op is 1000 loads)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171667771
2017-10-10 11:46:40 -07:00
costan
ca216e493f leveldb: Rename SNAPPY to HAVE_SNAPPY.
This follows the general naming convention for preprocessor macros used
to detect feature (library / header file / symbol) presence.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171184641
2017-10-05 12:19:09 -07:00
costan
25767d066c leveldb: Remove *_unlocked feature detection from POSIX port.
CL 170738066 removed all instances of fread_unlocked, fwrite_unlocked
and fflush_unlocked calls from the codebase, so the feature detection
can be removed as well.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171154269
2017-10-05 12:18:49 -07:00
cmumford
4a7e7f50dc Add LEVELDB_EXPORT macro to export public symbols.
gcc defaults to exporting all symbols, but other linkers do not. Adding
the LEVELDB_EXPORT macro allows a project to set LEVELDB_SHARED_LIBRARY
when building/linking with leveldb as a shared library.

This is to allow leveldb to be created as a shared library on all
platforms support by Chrome and enables a fix for
https://bugs.chromium.org/p/chromium/issues/detail?id=764810.

This also has the benefit of reducing the shared library size from
418863 to 380367 bytes (64-bit Linux).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171037148
2017-10-04 11:53:12 -07:00
costan
542590d2a8 leveldb: Include <algorithm> in util/env_test.cc.
CL 170738066 introduced std::min and std::max to env_test.cc. These
require the <algorithm> header.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171024062
2017-10-04 10:38:33 -07:00
costan
8ae7998aab Fix FD leak in POSIX Env.
Deleting a PosixWritableFile without calling Close() leaks the file
descriptor. While the API description in include/leveldb/env.h does not
specify whether the caller is responsible for Close()ing the file before
deleting it, all other Env file implementations do release underlying
resources when destroyed, even if Close() is not called.

The leak shows up when running db_tests on Mac Travis, or on a vanilla
MacOS install.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170906843
2017-10-03 13:45:04 -07:00
costan
d9a9e02edf leveldb: Add tests for CL 170769101.
This also removes std::unique_ptr introduced in CL 170738066, because
it's C++11-only, and the open source version still supports older
versions at the moment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170876919
2017-10-03 11:32:02 -07:00
costan
4447f9cace Remove handling for unused LRUHandle representation special case.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170876103
2017-10-03 11:32:02 -07:00
sanjay
2372ac574f Fix file writing bug in CL 170738066.
If the file already existed, we should have truncated it. This was not
detected by leveldb tests since leveldb code avoids reusing same files,
but there was code elsewhere that was directly using leveldb files and
relying on this behavior.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170769101
2017-10-03 11:32:02 -07:00
cmumford
1c75e88055 Fix use of uninitialized value in LRUHandle.
If leveldb::Options::block_cache is set to a cache of zero capacity
then it is possible for LRUHandle::next to be used without having been
set.

Conditional jump or move depends on uninitialised value(s):
  leveldb::(anonymous namespace)::LRUHandle::key() const (cache.cc:58)
  leveldb::(anonymous namespace)::LRUCache::Unref(leveldb::(anonymous namespace)::LRUHandle*) (cache.cc:234)
  leveldb::(anonymous namespace)::LRUCache::Release(leveldb::Cache::Handle*) (cache.cc:266)
  leveldb::(anonymous namespace)::ShardedLRUCache::Release(leveldb::Cache::Handle*) (cache.cc:375)
  leveldb::CacheTest::Insert(int, int, int) (cache_test.cc:59)

This bug forced a commit reversion in Chromium. For more information see
https://bugs.chromium.org/p/chromium/issues/detail?id=761398#c4

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170749054
2017-10-03 11:30:48 -07:00
sanjay
7e12c00ecf Fix issue 474: a race between the f*_unlocked() STDIO calls in
env_posix.cc and concurrent application calls to fflush(NULL).

The fix is to avoid using stdio in env_posix.cc but add our own
buffering where we need it.

Added a test to reproduce the bug.

Added a test for Env reads/writes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170738066
2017-10-03 11:27:09 -07:00
costan
bcd9a8ea4a Use portable CRC32C from google/crc32c.
Benchmark results below. More results at
354d61ef97.

New, MacBookPro13,3 with Core i7 6920HQ:
LevelDB:    version 1.20
Keys:       16 bytes each
Values:     100 bytes each (50 bytes after compression)
Entries:    1000000
RawSize:    110.6 MB (estimated)
FileSize:   62.9 MB (estimated)
WARNING: Snappy compression is not enabled
------------------------------------------------
fillseq      :       2.952 micros/op;   37.5 MB/s
fillsync     :      43.932 micros/op;    2.5 MB/s (1000 ops)
fillrandom   :       3.856 micros/op;   28.7 MB/s
overwrite    :       4.053 micros/op;   27.3 MB/s
readrandom   :       4.234 micros/op; (1000000 of 1000000 found)
readrandom   :       3.923 micros/op; (1000000 of 1000000 found)
readseq      :       0.201 micros/op;  550.8 MB/s
readreverse  :       0.356 micros/op;  310.6 MB/s
compact      :  436800.000 micros/op;
readrandom   :       2.375 micros/op; (1000000 of 1000000 found)
readseq      :       0.151 micros/op;  734.3 MB/s
readreverse  :       0.298 micros/op;  370.7 MB/s
fill100K     :     554.075 micros/op;  172.1 MB/s (1000 ops)
crc32c       :       1.393 micros/op; 2805.0 MB/s (4K per op)
snappycomp   :    3902.000 micros/op; (snappy failure)
snappyuncomp :    3821.000 micros/op; (snappy failure)
acquireload  :      13.088 micros/op; (each op is 1000 loads)

Baseline, MacBookPro13,3 with Core i7 6920HQ:
LevelDB:    version 1.20
Keys:       16 bytes each
Values:     100 bytes each (50 bytes after compression)
Entries:    1000000
RawSize:    110.6 MB (estimated)
FileSize:   62.9 MB (estimated)
WARNING: Snappy compression is not enabled
------------------------------------------------
fillseq      :       3.000 micros/op;   36.9 MB/s
fillsync     :      46.721 micros/op;    2.4 MB/s (1000 ops)
fillrandom   :       3.922 micros/op;   28.2 MB/s
overwrite    :       4.080 micros/op;   27.1 MB/s
readrandom   :       4.409 micros/op; (1000000 of 1000000 found)
readrandom   :       3.895 micros/op; (1000000 of 1000000 found)
readseq      :       0.190 micros/op;  582.4 MB/s
readreverse  :       0.413 micros/op;  267.6 MB/s
compact      :  441076.000 micros/op;
readrandom   :       2.308 micros/op; (1000000 of 1000000 found)
readseq      :       0.170 micros/op;  651.2 MB/s
readreverse  :       0.302 micros/op;  366.2 MB/s
fill100K     :     614.289 micros/op;  155.3 MB/s (1000 ops)
crc32c       :       3.547 micros/op; 1101.2 MB/s (4K per op)
snappycomp   :    3393.000 micros/op; (snappy failure)
snappyuncomp :    3171.000 micros/op; (snappy failure)
acquireload  :      12.761 micros/op; (each op is 1000 loads)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170100372
2017-09-26 18:50:41 -07:00