24 Commits

Author	SHA1	Message	Date
chrislusf	1b6ab2f6af	Add boltdb for volume needle map ... boltdb is fairly slow to write, about 6 minutes for recreating index for 1553934 files. Boltdb loads 1,553,934 x 16 = 24,862,944bytes from disk, and generate the boltdb as large as 134,217,728 bytes in 6 minutes. To compare, for leveldb, it recreates index in leveldb as large as 27,188,148 bytes in 8 seconds. For in memory version, it loads the index in To test the memory consumption, the leveldb or boltdb index are created. And the server is restarted. Using the benchmark tool to read lots of files. There are 7 volumes in benchmark collection, each with about 1553K files. For leveldb, the memory starts at 142,884KB, and stays at 179,340KB. For boltdb, the memory starts at 73,756KB, and stays at 144,564KB. For in-memory, the memory starts at 368,152KB, and stays at 448,032KB.	2015-03-29 11:04:32 -07:00
chrislusf	b07d81fb08	follow golint suggestions	2015-03-10 00:20:34 -07:00
chrislusf	9d8a6d2562	adjust log printing	2015-03-09 01:10:04 -07:00
Chris Lu	179d36ba0e	formatting code by: goimports -w=true .	2014-10-26 11:34:55 -07:00
Chris Lu	a5d6e70299	fix commenting error.	2014-10-26 11:25:02 -07:00
wyy	4126280d55	use github.com/chrislusf instead of github.com/aszxqw	2014-09-25 16:57:22 +08:00
wyy	1cd19447e3	use github.com/aszxqw instead of code.google.com/p	2014-09-25 00:47:09 +08:00
Chris Lu	4c58cef24a	a bit refactoring to prepare for volume format change and backward ... compatibility.	2014-08-25 11:37:00 -07:00
Chris Lu	4b7b439be9	Reduce memory usage for "weed fix"	2014-05-31 17:10:51 -07:00
Chris Lu	3b5035c468	1. v0.54 ... 2. go vet found many printing format errors	2014-04-17 00:16:44 -07:00
Chris Lu	a0955aa4dd	refactor functions	2014-03-23 21:57:10 -07:00
Chris Lu	96b67a7c35	adjusting logs	2014-03-19 04:45:25 -07:00
Chris Lu	27c74a7e66	Major: ... change replication_type to ReplicaPlacement, hopefully cleaner code works for 9 possible ReplicaPlacement xyz x : number of copies on other data centers y : number of copies on other racks z : number of copies on current rack x y z each can be 0,1,2 Minor: weed server "-mdir" default to "-dir" if empty	2014-03-02 22:16:54 -08:00
Chris Lu	bd6e1a8a70	Issue 60: weed export -collection doesn't work	2014-01-21 20:51:07 -08:00
Chris Lu	3b68711139	support for collections!	2013-11-12 02:21:22 -08:00
Chris Lu	cca1df83da	debug message	2013-10-31 12:56:05 -07:00
Chris Lu	69ac6b6bf6	Issue 45 in weed-fs: [Compact issue] Offset overflow ... New issue 45 by hieu.hcmus@gmail.com: [Compact issue] Offset overflow http://code.google.com/p/weed-fs/issues/detail?id=45 You are using uint32(Maximum 4Gb) to store needle offset(Maximum 32Gb) when compacting. Currently It is ok if the volume size is < 4gb Change variable "offset" in ScanVolumeFile function to uint64 to fix the issue.	2013-09-19 11:06:14 -07:00
Chris Lu	82b74c7940	issue 43 "go fmt" chagnes from "Ryan S. Brown" <sb@ryansb.com> ... some basic changes to parse upload url	2013-09-01 23:58:21 -07:00
Chris Lu	ed154053c8	switching to temporarily use glog library	2013-08-08 23:57:22 -07:00
Chris Lu	915b16f97a	refactoring, same logic, but the store replication logic is moved to a ... stand-alone file, for later easier improvements	2013-04-16 00:10:21 -07:00
Chris Lu	a3eddde525	fix documentation error	2013-03-18 21:29:25 -07:00
Chris Lu	db8e27be6e	add lots of error checking by GThomas	2013-02-26 22:54:22 -08:00
Chris Lu	38f4425a65	checking error	2013-02-10 13:41:25 -08:00
Chris Lu	d4e5a22e53	rename from cmd to weed for easier "go build"	2013-02-10 04:05:28 -08:00