chrislusf
b39c384d6d
Revert "Merge pull request #284 from thinxer/binary"
...
This reverts commit 3523ad52395d100994b1
2016-04-10 00:24:22 -07:00
Jianfei Wang
a192373c25
storage: do not copy sync.Mutex by value
2016-04-10 13:47:03 +08:00
Jianfei Wang
18254d9aae
replace util/bytes.go with binary.BigEndian
2016-04-09 14:38:01 +08:00
chrislusf
5d100994b1
Revert "Merge pull request #281 from thinxer/binary"
...
This reverts commit a87fe8ffce6876bfa685
2016-04-08 15:52:03 -07:00
Jianfei Wang
7bc10d46e3
replace util/bytes.go with binary.BigEndian
2016-04-06 17:08:52 +08:00
chrislusf
86cd40fba8
Add "weed backup" command.
...
This is a pre-cursor for asynchronous replication.
2015-05-26 00:58:41 -07:00
Stuart P. Bentley
f0c2a2dcb3
Change all chrislusf/weed-fs links to point to chrislu/seaweedfs
2015-04-16 19:18:06 +00:00
chrislusf
1b6ab2f6af
Add boltdb for volume needle map
...
boltdb is fairly slow to write, about 6 minutes for recreating index
for 1553934 files. Boltdb loads 1,553,934 x 16 = 24,862,944bytes from
disk, and generate the boltdb as large as 134,217,728 bytes in 6
minutes.
To compare, for leveldb, it recreates index in leveldb as large as
27,188,148 bytes in 8 seconds.
For in memory version, it loads the index in
To test the memory consumption, the leveldb or boltdb index are
created. And the server is restarted. Using the benchmark tool to read
lots of files. There are 7 volumes in benchmark collection, each with
about 1553K files.
For leveldb, the memory starts at 142,884KB, and stays at 179,340KB.
For boltdb, the memory starts at 73,756KB, and stays at 144,564KB.
For in-memory, the memory starts at 368,152KB, and stays at 448,032KB.
2015-03-29 11:04:32 -07:00
chrislusf
020ba6c9a8
add leveldb support for needle map
...
This supposedly should reduce memory consumption. However, for tests
with millions of, this shows consuming more memories. Need to see
whether this will work out. If not, later boltdb will be tested.
2015-03-27 16:34:58 -07:00
chrislusf
9d8a6d2562
adjust log printing
2015-03-09 01:10:04 -07:00
Chris Lu
af416189f1
Cleanup error printing.
2015-01-13 17:04:41 -08:00
Chris Lu
179d36ba0e
formatting code by: goimports -w=true .
2014-10-26 11:34:55 -07:00
wyy
4126280d55
use github.com/chrislusf instead of github.com/aszxqw
2014-09-25 16:57:22 +08:00
wyy
1cd19447e3
use github.com/aszxqw instead of code.google.com/p
2014-09-25 00:47:09 +08:00
Chris Lu
faf497feb8
Important Fix: Avoid overwriting the index file!!!
2014-05-29 19:19:03 -07:00
Chris Lu
3b5035c468
1. v0.54
...
2. go vet found many printing format errors
2014-04-17 00:16:44 -07:00
Chris Lu
51939efeac
1. volume server now sends master server its max file key, so that
...
master server does not need to store the sequence on disk any more
2. fix raft server's failure to init cluster during bootstrapping
2014-04-16 23:43:27 -07:00
Chris Lu
3dbebfd1e1
Thread-safe fixes:
...
1. avoid sharing []byte
2. switch to use ReadAt()
2014-03-19 04:41:16 -07:00
Chris Lu
cd10c277b2
can now delete a collection! Is this a dangerous feature? Only enabling
...
deleting "benchmark" collections for now.
2014-03-10 11:43:54 -07:00
Chris Lu
82b74c7940
issue 43 "go fmt" chagnes from "Ryan S. Brown" <sb@ryansb.com>
...
some basic changes to parse upload url
2013-09-01 23:58:21 -07:00
Chris Lu
ed154053c8
switching to temporarily use glog library
2013-08-08 23:57:22 -07:00
Chris Lu
50269b74ce
add dataCenter option when assign file keys
...
add dataCenter option when starting volume servers
some work related to freeze a volume. Not tested yet.
2013-06-19 18:10:38 -07:00
Chris Lu
fb635146a1
refactoring needle mapper interface to separate index file storage logic
...
out
2013-04-18 00:23:14 -07:00
Chris Lu
db8e27be6e
add lots of error checking by GThomas
2013-02-26 22:54:22 -08:00
Chris Lu
a4cef2fbd4
only cares about error other than EOF
2013-02-11 23:54:21 -08:00
Chris Lu
91492b4947
avoid EOF error on windows. This is a "try to load" function anyway.
2013-02-11 23:38:48 -08:00
Chris Lu
aecea22666
Issue 15: Stress test corrupts volume
2013-02-10 14:00:06 -08:00
Chris Lu
5071f528f6
testing compilation with remove package
2013-02-10 03:49:51 -08:00
