Commit Graph

408 Commits

Author SHA1 Message Date
Chris Mumford
2a47801868 Use partial path to benchmark/benchmark.h.
Using the partial path offers more flexibility to projects which
may checkout google/benchmark to a different location.

PiperOrigin-RevId: 357819911
2021-02-16 16:51:09 -08:00
Victor Costan
4a919ea4f7 IWYU fixes in db/c.cc.
Fixes https://github.com/google/leveldb/issues/872

PiperOrigin-RevId: 353657701
2021-01-25 17:18:22 +00:00
Victor Costan
1998c0ef15 Fix build errors.
PiperOrigin-RevId: 351442409
2021-01-12 21:58:19 +00:00
leveldb Team
8f1861462b Sync MANIFEST before closing in db_impl when creating a new DB.
Add logging with debugging information when failing to load a version set.

PiperOrigin-RevId: 351432332
2021-01-12 21:58:08 +00:00
leveldb Team
8cce47e450 Optimize leveldb block seeks to utilize the current iterator location.
This is beneficial when iterators are reused and seeks are not random
but increasing. It is additionally beneficial with larger block sizes and keys with common prefixes.

Add a benchmark "seekordered" to db_bench that reuses iterators across
increasing seeks.  Add support to the benchmark to count comparisons made and to support common key prefix length. Change benchmark random seeds to be reproducible for entire benchmark suite executions but unique for threads in different benchmarks runs.  This changes a benchmark suite of readrandom,seekrandom from having a 100% found ratio as previously it had the same seed used for fillrandom.

./db_bench --benchmarks=fillrandom,compact,seekordered --block_size=262144 --comparisons=1 --key_prefix=100

without this change (though with benchmark changes):
seekrandom   :      55.309 micros/op; (631820 of 1000000 found)
Comparisons: 27001049
seekordered  :       1.732 micros/op; (631882 of 1000000 found)
Comparisons: 26998402

with this change:
seekrandom   :      55.866 micros/op; (631820 of 1000000 found)
Comparisons: 26952143
seekordered  :       1.686 micros/op; (631882 of 1000000 found)
Comparisons: 25549369

For ordered seeking, this is a reduction of 5% comparisons and a 3% speedup. For random seeking (with single use iterators) the comparisons and speed are less than 1% and likely noise.

PiperOrigin-RevId: 351149832
2021-01-11 15:41:38 +00:00
Victor Costan
fdc8f72895 Merge pull request #862 from rex4539:https
PiperOrigin-RevId: 349711809
2021-01-01 09:32:07 +00:00
Dimitris Apostolou
532be85306
Fix insecure links 2020-12-19 20:07:38 +02:00
Victor Costan
6721eda0b4 Update Travis CI config.
PiperOrigin-RevId: 347391876
2020-12-14 08:37:35 -08:00
Victor Costan
295ce1336f
Merge pull request #855 from cmumford/submodule-fix
Fixup for adding the third_party/benchmark submodule.
2020-11-30 16:37:05 -08:00
Chris Mumford
c3b52f7db6 Fixup for adding the third_party/benchmark submodule. 2020-11-30 16:26:16 -08:00
Chris Mumford
28df52115d Merge pull request #853 from cmumford:benchmark
PiperOrigin-RevId: 344909677
2020-11-30 16:16:39 -08:00
Chris Mumford
2dcbd4a2c5 Merge pull request #854 from cmumford:printf-fix
PiperOrigin-RevId: 344871226
2020-11-30 16:16:25 -08:00
Chris Mumford
b754fdca72 Fixed fprintf of 64-bit value. 2020-11-30 10:48:17 -08:00
Chris Mumford
37d36c92f8 Added google/benchmark submodule. 2020-11-30 10:47:15 -08:00
Sanjay Ghemawat
2802398c94 Fix bug in filter policy documentation example.
PiperOrigin-RevId: 344817715
2020-11-30 09:11:08 -08:00
leveldb Team
99ab4730d6 Use external benchmark API header
PiperOrigin-RevId: 339310928
2020-11-30 09:10:59 -08:00
leveldb Team
ed781070b4 Internal test cleanup
PiperOrigin-RevId: 339287832
2020-11-30 09:10:46 -08:00
leveldb Team
b7d3023269 Internal cleanup migrating StatusOr.
PiperOrigin-RevId: 329720018
2020-10-07 21:15:26 +00:00
Chris Mumford
1454924aac Merge pull request #822 from jl0x61:bugFix
PiperOrigin-RevId: 321372819
2020-07-15 09:20:04 -07:00
jl0x61
1754c12c54 update index.md
remove return value of GetApproximateSizes in index.md
2020-07-14 19:32:03 +08:00
Victor Costan
c46e79c760 Merge pull request #819 from wzk784533:master
PiperOrigin-RevId: 321000544
2020-07-13 19:14:04 +00:00
wzk784533
28602d3625 avoid unnecessary memory copy 2020-07-11 13:44:11 +08:00
Victor Costan
5bd5f0f67a Merge pull request #798 from lntotk:master
PiperOrigin-RevId: 309738404
2020-05-04 22:47:40 +00:00
Victor Costan
23b6337f69 Fix Travis CI build.
PiperOrigin-RevId: 309138195
2020-04-30 01:21:01 +00:00
Victor Costan
5c6dd75897 Fix accidental double std:: qualifiers.
PiperOrigin-RevId: 309136120
2020-04-30 01:20:50 +00:00
Victor Costan
a6b3a2012e Add some std:: qualifiers to types and functions.
PiperOrigin-RevId: 309110431
2020-04-29 22:33:14 +00:00
Victor Costan
3f934e3705 Switch from C headers to C++ headers.
This CL makes the following substitutions.

* assert.h -> cassert
* math.h -> cmath
* stdarg.h -> cstdarg
* stddef.h -> cstddef
* stdint.h -> cstdint
* stdio.h -> cstdio
* stdlib.h -> cstdlib
* string.h -> cstring

PiperOrigin-RevId: 309080151
2020-04-29 20:51:13 +00:00
Victor Costan
23d67e7c1f Fix C++11 build.
PiperOrigin-RevId: 308839805
2020-04-28 18:05:22 +00:00
leveldb Team
98a3b8cf65 change const to constexpr
PiperOrigin-RevId: 307113877
2020-04-28 00:17:51 +00:00
lntotk
10bc0f2595 remove unnessary status judge 2020-04-24 02:00:12 +00:00
Victor Costan
201f52201f Remove leveldb::port::kLittleEndian.
Clang 10 includes the optimizations described in
https://bugs.llvm.org/show_bug.cgi?id=41761. This means that the
platform-independent implementations of {Decode,Encode}Fixed{32,64}()
compile to one instruction on the most recent Clang and GCC.

PiperOrigin-RevId: 306330166
2020-04-14 01:10:05 +00:00
Victor Costan
ba369ddbaf Use LLVM 10 on Travis CI.
PiperOrigin-RevId: 306236199
2020-04-14 01:09:54 +00:00
Victor Costan
5903e7a112 Remove Windows workarounds in some tests.
leveldb::Env::DeleteFile was replaced with leveldb::Env::RemoveFile in
all tests. This allows us to remove workarounds for windows.h #defining
DeleteFile.
PiperOrigin-RevId: 289121105
2020-01-14 18:31:37 -08:00
Victor Costan
a0191e5563 Add Env::Remove{File,Dir} which obsolete Env::Delete{File,Dir}.
The "DeleteFile" method name causes pain for Windows developers, because
<windows.h> #defines a DeleteFile macro to DeleteFileW or DeleteFileA.
Current code uses workarounds, like #undefining DeleteFile everywhere an
Env is declared, implemented, or used.

This CL removes the need for workarounds by renaming Env::DeleteFile to
Env::RemoveFile. For consistency, Env::DeleteDir is also renamed to
Env::RemoveDir. A few internal methods are also renamed for consistency.
Software that supports Windows is expected to migrate any Env
implementations and usage to Remove{File,Dir}, and never use the name
Env::Delete{File,Dir} in its code.

The renaming is done in a backwards-compatible way, at the risk of
making it slightly more difficult to build a new correct Env
implementation. The backwards compatibility is achieved using the
following hacks:

1) Env::Remove{File,Dir} methods are added, with a default
    implementation that calls into Env::Delete{File,Dir}. This makes old
    Env implementations compatible with code that calls into the updated
    API.
2) The Env::Delete{File,Dir} methods are no longer pure virtuals.
    Instead, they gain a default implementation that calls into
    Env::Remove{File,Dir}. This makes updated Env implementations
    compatible with code that calls into the old API.

The cost of this approach is that it's possible to write an Env without
overriding either Rename{File,Dir} or Delete{File,Dir}, without getting
a compiler warning. However, attempting to run the test suite will
immediately fail with an infinite call stack ending in
{Remove,Delete}{File,Dir}, making developers aware of the problem.

PiperOrigin-RevId: 288710907
2020-01-09 09:18:14 -08:00
leveldb Team
d152b23f3b Defend against inclusion of windows.h in tests that invoke
Env::DeleteFile.

PiperOrigin-RevId: 283607548
2020-01-09 09:17:59 -08:00
Victor Costan
58a89bbcb2 Add WITHOUT ROWID to SQLite benchmark.
The SQLite-specific schema feature is documented at
https://www.sqlite.org/withoutrowid.html and
https://www.sqlite.org/rowidtable.html.

By default, SQLite stores each table in a B-tree keyed by an integer,
called the ROWID. Any index, including the PRIMARY KEY index, is a
separate B-tree mapping index keys to ROWIDs. Tables without ROWIDs are
stored in a B-tree keyed by the primary key. Additional indexes (the
PRIMARY KEY index is implicitly built into the table) are stored as
B-trees mapping index keys to row primary keys.

This CL introduces a boolean --use-rowids flag to db_bench_sqlite. When
the flag is false (default), the schema of the test table includes
WITHOUT ROWID. The test table uses a primary key, so adding WITHOUT
ROWID to the schema reduces the number of B-trees used by the benchmark
from 2 to 1. This brings SQLite's disk usage closer to LevelDB.

When WITHOUT ROWID is used, SQLite fares better (than today) on
benchmarks with small (16-byte) keys, and worse on benchmarks with large
(100kb) keys.

Baseline results:
fillseq      :      21.310 micros/op;    5.2 MB/s
fillseqsync  :     146.377 micros/op;    0.8 MB/s (10000 ops)
fillseqbatch :       2.065 micros/op;   53.6 MB/s
fillrandom   :      34.767 micros/op;    3.2 MB/s
fillrandsync :     159.943 micros/op;    0.7 MB/s (10000 ops)
fillrandbatch :      15.055 micros/op;    7.3 MB/s
overwrite    :      43.660 micros/op;    2.5 MB/s
overwritebatch :      27.691 micros/op;    4.0 MB/s
readrandom   :      12.725 micros/op;
readseq      :       2.602 micros/op;   36.7 MB/s
fillrand100K :     606.333 micros/op;  157.3 MB/s (1000 ops)
fillseq100K  :     657.457 micros/op;  145.1 MB/s (1000 ops)
readseq      :      46.523 micros/op; 2049.9 MB/s
readrand100K :      54.943 micros/op;

Results after this CL:
fillseq      :      16.231 micros/op;    6.8 MB/s
fillseqsync  :     147.460 micros/op;    0.8 MB/s (10000 ops)
fillseqbatch :       2.294 micros/op;   48.2 MB/s
fillrandom   :      27.871 micros/op;    4.0 MB/s
fillrandsync :     141.979 micros/op;    0.8 MB/s (10000 ops)
fillrandbatch :      16.087 micros/op;    6.9 MB/s
overwrite    :      26.829 micros/op;    4.1 MB/s
overwritebatch :      19.014 micros/op;    5.8 MB/s
readrandom   :      11.657 micros/op;
readseq      :       0.155 micros/op;  615.0 MB/s
fillrand100K :     816.812 micros/op;  116.8 MB/s (1000 ops)
fillseq100K  :     754.689 micros/op;  126.4 MB/s (1000 ops)
readseq      :      47.112 micros/op; 2024.3 MB/s
readrand100K :     287.679 micros/op;

Results after this CL, with --use-rowids=1
fillseq      :      20.655 micros/op;    5.4 MB/s
fillseqsync  :     146.408 micros/op;    0.8 MB/s (10000 ops)
fillseqbatch :       2.045 micros/op;   54.1 MB/s
fillrandom   :      34.080 micros/op;    3.2 MB/s
fillrandsync :     154.582 micros/op;    0.7 MB/s (10000 ops)
fillrandbatch :      14.404 micros/op;    7.7 MB/s
overwrite    :      42.928 micros/op;    2.6 MB/s
overwritebatch :      27.829 micros/op;    4.0 MB/s
readrandom   :      12.835 micros/op;
readseq      :       2.483 micros/op;   38.4 MB/s
fillrand100K :     603.265 micros/op;  158.1 MB/s (1000 ops)
fillseq100K  :     662.473 micros/op;  144.0 MB/s (1000 ops)
readseq      :      45.478 micros/op; 2097.0 MB/s
readrand100K :      54.439 micros/op;
PiperOrigin-RevId: 283407101
2019-12-02 13:51:20 -08:00
Victor Costan
c0d43142ff
Merge pull request #756 from pwnall/third_party_2
Fixup for adding the third_party/googletest submodule.
2019-12-02 13:50:43 -08:00
Victor Costan
e36b831851 Fixup for adding the third_party/googletest submodule. 2019-12-02 12:18:34 -08:00
leveldb Team
583a42b596 Internal change.
PiperOrigin-RevId: 282373286
2019-12-02 11:44:39 -08:00
Victor Costan
db8352187b Fixup for adding the third_party/googletest submodule. (#754) 2019-11-25 07:22:35 -08:00
Victor Costan
1c58902bdc Switch testing harness to googletest.
PiperOrigin-RevId: 281815695
2019-11-21 13:11:40 -08:00
Victor Costan
2c9c80bd53 Move CI to Visual Studio 2019.
PiperOrigin-RevId: 279785825
2019-11-11 12:07:40 -08:00
Victor Costan
ed72a3496e Allow different C/C++ standards when this is used as a subproject.
Inspired by https://github.com/google/snappy/pull/85

PiperOrigin-RevId: 279649967
2019-11-10 18:22:41 -08:00
Victor Costan
41c8d83914 Align CMake configuration with related projects.
PiperOrigin-RevId: 279238007
2019-11-07 22:44:08 -08:00
Victor Costan
0c40829872 Remove redundant PROJECT_SOURCE_DIR usage from CMake config.
Inspired by https://github.com/google/crc32c/pull/32

PiperOrigin-RevId: 278718726
2019-11-05 16:32:26 -08:00
Victor Costan
5abdf4c019 Fix installed target definition.
Using CMAKE_INSTALL_INCLUDEDIR before including GNUINstallDirs results
in a broken installation when CMAKE_INSTALL_PREFIX is a non-standard
directory.

Inspired from https://github.com/google/crc32c/pull/39

PiperOrigin-RevId: 278427974
2019-11-05 00:32:51 -08:00
Victor Costan
cf4d9ab23d Test CMake installation on Travis.
PiperOrigin-RevId: 278300591
2019-11-03 21:42:23 -08:00
Chris Mumford
95d0ba1cb0 Renamed local variable in DBImpl::Write.
The local variable `updates` in DBImpl::Write was hiding the
`updates` parameter. Renamed to avoid this conflict.

PiperOrigin-RevId: 277089971
2019-10-28 13:24:08 -07:00
Chris Mumford
657ba51429 Added return in Version::Get::State::Match to quiet warning.
Added unreached return at the end of Version::Get::State::Match
to stop this _incorrect_ warning:

    version_set.cc:376:5: warning: control reaches end of
    non-void function [-Wreturn-type]

This warning was being emitted when building with clang 6.0.1-10
and also emitted by lgtm.com when statically analyzing leveldb even
though all SaverState enumeration values were handled.

PiperOrigin-RevId: 272455474
2019-10-28 13:23:53 -07:00
Liao Tonglang
f2dae4e74a
fix typo in comment of LRUHandle
LRUHandle has no member "next_", fix it to "next" instead.
2019-10-22 18:46:43 +08:00