Jianfei Wang
a192373c25
storage: do not copy sync.Mutex by value
2016-04-10 13:47:03 +08:00
Jianfei Wang
18254d9aae
replace util/bytes.go with binary.BigEndian
2016-04-09 14:38:01 +08:00
Jianfei Wang
f0cc264757
storage: add test for ParseKeyHash
2016-04-09 11:33:06 +08:00
chrislusf
5d100994b1
Revert "Merge pull request #281 from thinxer/binary"
...
This reverts commit a87fe8ffce6876bfa685
2016-04-08 15:52:03 -07:00
Jianfei Wang
7bc10d46e3
replace util/bytes.go with binary.BigEndian
2016-04-06 17:08:52 +08:00
Alexandr Razumov
ca2a6cc36b
use klauspost/crc32 instead of hash/crc32
2016-04-01 22:18:58 +03:00
chrislusf
e921cb1a9d
format changes
2015-12-14 22:38:58 -08:00
tnextday
662915e691
Delete all chunks when delete a ChunkManifest
...
LoadChunkManifest can uncompress buffer
move compress.go from storage to operation because of import cycle
MakeFile add cross complete command
2015-12-02 21:27:29 +08:00
tnextday
2c0a7fe75e
complete submit chunked file
2015-12-02 15:00:46 +08:00
tnextday
de5e07ce3e
rename FlagChunkedFile to FlagChunkManifest
2015-12-01 20:53:44 +08:00
tnextday
6b0894d806
update ChunkedFile to seekable reader, so we can use io.* to read data
2015-12-01 20:23:50 +08:00
tnextday
1817864a45
add chunk file helper to support large file
2015-11-29 23:49:41 +08:00
Timur Nurutdinov
4b0904e9c4
fix extract collection name from .dat file
2015-08-18 19:58:45 +03:00
chrislusf
320e946d50
fix ttl change detection
...
https://github.com/chrislusf/seaweedfs/issues/166
2015-07-17 19:30:25 -07:00
chrislusf
430f371a97
fix wrong logic
2015-07-11 12:20:39 -07:00
chrislusf
9b8f8f4bb9
add logging
2015-07-11 12:20:09 -07:00
chrislusf
2c595d2d16
skip isFileUnchanged checking since ttl always change
...
skip checking since ttl always change. Fixing
https://github.com/chrislusf/seaweedfs/issues/166
2015-07-10 09:43:49 -07:00
chrislusf
a4d3f2da3d
Avoid nil case when fixing data
2015-07-08 23:21:56 -07:00
chrislusf
418878edce
In case lastNode==0, need to set it to -1
...
Fix https://github.com/chrislusf/seaweedfs/issues/156
2015-06-23 22:49:09 -07:00
chrislusf
86cd40fba8
Add "weed backup" command.
...
This is a pre-cursor for asynchronous replication.
2015-05-26 00:58:41 -07:00
chrislusf
36a31771f1
Add Key String() function
2015-05-25 23:50:19 -07:00
chrislusf
8f88d382a5
Rename variables
2015-05-23 10:16:01 -07:00
chrislusf
85bfab612c
Add comments
2015-05-23 10:13:26 -07:00
chrislusf
c7cf4fa46c
Fix looking for master nodes.
2015-05-22 08:12:51 -07:00
chrislusf
3188382ea7
More debug information for connecting to master
2015-05-14 21:46:59 -07:00
chrislusf
b8314fb054
Textual changes.
2015-05-08 23:44:11 -07:00
chrislusf
66317b8438
Increase compact revision during each compaction
2015-05-08 23:43:59 -07:00
chrislusf
dac3b592ed
Add compact revision in volume super block
2015-05-08 23:34:14 -07:00
yanyiwu
dc47db5512
unittest: TestSortVolumeInfos
2015-04-21 01:22:13 +08:00
yanyiwu
b7a18580b8
Sort VolumeInfos by VolumeId in Store.Status();
...
Ordered VolumeInfos is more Human-readable,
especially when there is a lot of volumes.
2015-04-21 01:21:21 +08:00
chrislusf
5c81fa1075
Prepend prefix "0" for odd number of hex characters.
2015-04-19 18:54:05 -07:00
Stuart P. Bentley
f0c2a2dcb3
Change all chrislusf/weed-fs links to point to chrislu/seaweedfs
2015-04-16 19:18:06 +00:00
chrislusf
49d639ecab
Add error checking for file reads.
2015-04-14 23:05:33 -07:00
chrislusf
b96911c517
remove verbose logging.
2015-03-29 11:44:16 -07:00
chrislusf
1b6ab2f6af
Add boltdb for volume needle map
...
boltdb is fairly slow to write, about 6 minutes for recreating index
for 1553934 files. Boltdb loads 1,553,934 x 16 = 24,862,944bytes from
disk, and generate the boltdb as large as 134,217,728 bytes in 6
minutes.
To compare, for leveldb, it recreates index in leveldb as large as
27,188,148 bytes in 8 seconds.
For in memory version, it loads the index in
To test the memory consumption, the leveldb or boltdb index are
created. And the server is restarted. Using the benchmark tool to read
lots of files. There are 7 volumes in benchmark collection, each with
about 1553K files.
For leveldb, the memory starts at 142,884KB, and stays at 179,340KB.
For boltdb, the memory starts at 73,756KB, and stays at 144,564KB.
For in-memory, the memory starts at 368,152KB, and stays at 448,032KB.
2015-03-29 11:04:32 -07:00
chrislusf
0d23f49f74
removing unused files
2015-03-27 16:35:42 -07:00
chrislusf
020ba6c9a8
add leveldb support for needle map
...
This supposedly should reduce memory consumption. However, for tests
with millions of, this shows consuming more memories. Need to see
whether this will work out. If not, later boltdb will be tested.
2015-03-27 16:34:58 -07:00
chrislusf
78bc7b393f
tweaking ui
2015-03-22 12:50:04 -07:00
chrislusf
b07d81fb08
follow golint suggestions
2015-03-10 00:20:34 -07:00
chrislusf
f511b507a5
Add read only public port on volume server
...
Add read only public port on volume server
2015-03-09 01:10:04 -07:00
chrislusf
d48d76cb4f
adding special handling to recover data if possible
...
For bug #87 and #93 , add special handling to recover data if possible.
2015-03-09 01:10:04 -07:00
chrislusf
853701cb6b
Avoid wrong way to delete on replication failure
...
Avoid wrong way to delete on replication failure. This deletion has bug
to write. The better fix is not to use the deletion on failure at all.
2015-03-09 01:10:04 -07:00
chrislusf
9d8a6d2562
adjust log printing
2015-03-09 01:10:04 -07:00
chrislusf
4ee766e966
avoid auto unzip tar.gz files
...
fix https://github.com/chrislusf/weed-fs/issues/90
2015-02-25 23:59:44 -08:00
Chris Lu
f7998f8652
merge conflicts
2015-02-07 15:35:28 -08:00
Chris Lu
7b4a53b2c1
Add optional admin port to volume server, to seperate admin operations from normal file operations.
2015-01-18 17:03:38 -08:00
yourchanges
a86042e413
fix the #59 and #56
2015-01-16 17:30:23 +08:00
Chris Lu
41bd5179f3
Resolve Conflicts
2015-01-14 09:56:13 -08:00
Chris Lu
af416189f1
Cleanup error printing.
2015-01-13 17:04:41 -08:00
Lei Xue
029e3a3822
fix some typos
2015-01-13 18:46:56 +08:00
