16 Commits

Author	SHA1	Message	Date
Chris Lu	766512c475	close current needle map when switching	2016-05-09 10:04:21 -07:00
chrislusf	9b8f8f4bb9	add logging	2015-07-11 12:20:09 -07:00
chrislusf	8f88d382a5	Rename variables	2015-05-23 10:16:01 -07:00
chrislusf	66317b8438	Increase compact revision during each compaction	2015-05-08 23:43:59 -07:00
Stuart P. Bentley	f0c2a2dcb3	Change all chrislusf/weed-fs links to point to chrislu/seaweedfs	2015-04-16 19:18:06 +00:00
chrislusf	1b6ab2f6af	Add boltdb for volume needle map ... boltdb is fairly slow to write, about 6 minutes for recreating index for 1553934 files. Boltdb loads 1,553,934 x 16 = 24,862,944bytes from disk, and generate the boltdb as large as 134,217,728 bytes in 6 minutes. To compare, for leveldb, it recreates index in leveldb as large as 27,188,148 bytes in 8 seconds. For in memory version, it loads the index in To test the memory consumption, the leveldb or boltdb index are created. And the server is restarted. Using the benchmark tool to read lots of files. There are 7 volumes in benchmark collection, each with about 1553K files. For leveldb, the memory starts at 142,884KB, and stays at 179,340KB. For boltdb, the memory starts at 73,756KB, and stays at 144,564KB. For in-memory, the memory starts at 368,152KB, and stays at 448,032KB.	2015-03-29 11:04:32 -07:00
chrislusf	020ba6c9a8	add leveldb support for needle map ... This supposedly should reduce memory consumption. However, for tests with millions of, this shows consuming more memories. Need to see whether this will work out. If not, later boltdb will be tested.	2015-03-27 16:34:58 -07:00
Chris Lu	179d36ba0e	formatting code by: goimports -w=true .	2014-10-26 11:34:55 -07:00
wyy	4126280d55	use github.com/chrislusf instead of github.com/aszxqw	2014-09-25 16:57:22 +08:00
wyy	1cd19447e3	use github.com/aszxqw instead of code.google.com/p	2014-09-25 00:47:09 +08:00
Chris Lu	b9aee2defb	add TTL support ... The volume TTL and file TTL are not necessarily the same. as long as file TTL is smaller than volume TTL, it'll be fine. volume TTL is used when assigning file id, e.g. http://.../dir/assign?ttl=3h file TTL is used when uploading	2014-09-20 12:38:59 -07:00
Chris Lu	4b7b439be9	Reduce memory usage for "weed fix"	2014-05-31 17:10:51 -07:00
Chris Lu	3e5b4da361	fix compilation error	2014-05-26 17:36:41 -07:00
Chris Lu	625f880b15	Adjust "weed server" parameters.	2014-05-26 17:34:54 -07:00
Chris Lu	fe3f06435e	Refactor out volume vacuum.	2014-05-19 20:54:39 -07:00
Chris Lu	e7aaa24da8	Refactor out volume vacuum.	2014-05-19 19:24:35 -07:00