mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-22 08:07:55 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,770 Commits 32 Branches 296 Tags 152 MiB
69bcdf470b
Commit Graph

119 Commits

Author SHA1 Message Date
Chris Lu
c8b2dac6c1 volume: avoid sharing volume dat file handle 
possibly help on https://github.com/chrislusf/seaweedfs/issues/1184
 2020-01-21 21:18:01 -08:00
Chris Lu
acf7ca7b93 volume: fix compaction 2020-01-08 09:45:03 -08:00
Chris Lu
efd2f50ede compaction changed to .idx based deletion 2019-12-24 14:55:50 -08:00
Chris Lu
abffe857a1 change btree map to in memory level db 2019-12-24 10:18:56 -08:00
Chris Lu
72a561ab7c refactoring 2019-12-24 08:13:01 -08:00
Chris Lu
09ca936c78 shell: add ec.decode command 2019-12-23 12:48:20 -08:00
Chris Lu
f60154f330 master load backend storage config from master.toml 2019-11-28 18:33:18 -08:00
Chris Lu
85f8649320 refactor memory mapped file into backend storage 2019-11-09 00:10:59 -08:00
Chris Lu
db30a46050 skip memory mapped file when compacting 2019-10-29 23:34:38 -07:00
Chris Lu
5b950c735e Merge branch 'master' into refactoring_dat_backend 2019-10-29 23:18:41 -07:00
Chris Lu
57e441d67b fix compaction logic 2019-10-29 23:18:01 -07:00
Chris Lu
4b5ba4927f adjust NewDiskFile() paramaters 2019-10-29 22:37:36 -07:00
Chris Lu
19b6a16003 changed from os.file to backend.DataStorageBackend 2019-10-29 00:35:16 -07:00
Chris Lu
fc412e428b refactor ScanVolumeFileFrom() 2019-10-22 00:50:30 -07:00
Chris Lu
faec9076a4 adjust parameter names 2019-10-21 22:57:01 -07:00
j.laycock
6fc6322c90 Change joeslay paths to chrislusf paths 2019-09-12 14:18:21 +01:00
j.laycock
5885ab67b3 rename volume property MemoryMap to MemoryMapMaxSizeMB 2019-09-05 11:56:48 +01:00
Tom Maxwell
4a878c0006 Changed the InMemory bool to a uint32 so that it can be used to alter how much space to reserve 2019-09-04 15:27:14 +01:00
j.laycock
1f01eb78e8 Rename mem_map to mMap, remove some in_memory variables being passed around, added MemoryMapped member to volume struct 2019-09-03 17:00:59 +01:00
Tom Maxwell
d637d86d22 Changes to try and pass the URL parameters through - in memory flag not working still 2019-09-03 15:41:28 +01:00
j.laycock
9a459d984b Do not vacuum memory mapped files 2019-09-03 15:05:43 +01:00
j.laycock
595a1beff0 Swap imports to use joeslay 2019-09-02 11:28:40 +01:00
Chris Lu
d829df4f59 volume: protect against nil needle map 
fix @mastak reported nil problem in https://github.com/chrislusf/seaweedfs/issues/1037
 2019-08-14 01:08:01 -07:00
Chris Lu
e40634e6b4 volume: fail the volume deletion if compaction is in progress 
fix https://github.com/chrislusf/seaweedfs/issues/1035
 2019-08-12 00:53:50 -07:00
Chris Lu
f3b99cbfe5 fix wrong alignment fix 2019-07-21 13:50:42 -07:00
Chris Lu
644c5c27b4 refactoring 2019-07-21 13:50:24 -07:00
Chris Lu
33b3366043 volume: avoid compacting wait group 2019-07-21 13:49:59 -07:00
Chris Lu
a7b1b23c58 fix wrong volume count 
fix https://github.com/chrislusf/seaweedfs/issues/1013
 2019-07-17 00:03:15 -07:00
Chris Lu
115558e5f5 adjust counters 2019-06-17 21:02:50 -07:00
Chris Lu
289fd7eb39 count number of volumes and ec shards 2019-06-16 02:24:15 -07:00
Chris Lu
17ac1290c0 volume: load ec shards during heartbeats to master 2019-05-21 22:41:20 -07:00
Chris Lu
cf58fc0e63 refactor: extract out the write throttler 2019-05-06 13:56:08 -07:00
Chris Lu
b335f81a4f volume: add option to limit compaction speed 2019-05-03 17:22:39 -07:00
Chris Lu
3b3651dea3 volume: atomic copying file, adds version and stopOffset 2019-04-19 12:29:49 -07:00
Chris Lu
ac2727853f fix needle map entry size 2019-04-19 00:39:34 -07:00
Chris Lu
e5506152c0 refactoring 2019-04-18 21:43:36 -07:00
Chris Lu
000ee725fc refactor Offset into a struct of bytes 2019-04-08 19:40:56 -07:00
Chris Lu
df95ce0b6c weed backup: efficient delta backup 
fix https://github.com/chrislusf/seaweedfs/issues/399
 2019-03-25 23:01:53 -07:00
Chris Lu
70815e9124 WIP 2019-03-25 09:16:12 -07:00
Chris Lu
da871896c3 weed filer: set grpc port to port + 10000 2019-03-19 05:47:41 -07:00
Chris Lu
916b809c08 add error checking 2019-03-19 05:34:43 -07:00
bingoohuang
ab6be025d7 go fmt and fix some typo 2019-01-17 09:17:19 +08:00
bingoohuang
6ddfaf33cb extract VolumeFileScanner for ScanVolumeFile 2019-01-16 17:48:59 +08:00
Chris Lu
9383c91eb1 wait to read again if the volume is compacting 2019-01-05 19:52:17 -08:00
Chris Lu
1478d7ea21 reduce file seek when writing 2018-12-31 15:08:32 -08:00
Chris Lu
f050b22d6c close file in ScanVolumeFile() 
fix https://github.com/chrislusf/seaweedfs/issues/761
 2018-11-04 00:28:24 -07:00
Chris Lu
496a7398d1 adjust logging 2018-10-18 20:47:30 -07:00
Chris Lu
333709657c adjust log level 2018-10-15 00:40:12 -07:00
Chris Lu
a362261972 avoid possible racing issue if leveldb or boltdb is used. 
remove the to-be-generated files during commit

fix https://github.com/chrislusf/seaweedfs/issues/738
 2018-09-26 18:45:51 -07:00
Chris Lu
ea8ecf35f4 set needle AppendAtNs value during writes 2018-07-24 02:44:33 -07:00
Chris Lu
4f317c7e3d working version3 2018-07-24 01:36:04 -07:00
Chris Lu
7e2031b18f go fmt 2018-07-21 17:39:10 -07:00
Chris Lu
d4d7ced922 refactoring: add type for needle id, offset 
later the type size can possibly be adjusted
 2018-07-08 02:28:04 -07:00
Chris Lu
defcd4edc5 adjust log level 2018-07-07 00:48:58 -07:00
Chris Lu
9bcaa65574 avoid divided by 0 
fix https://github.com/chrislusf/seaweedfs/issues/650
 2018-06-25 01:20:15 -07:00
Chris Lu
9ba335a7c4 prepare for flexible super block 2018-06-24 11:37:08 -07:00
Chris Lu
7f7e4e9885 fix error for deleted files during compaction 
deletion during commit may cause trouble when make up the difference during commitCompact()
 2018-06-23 18:24:59 -07:00
Chris Lu
58344980e4 preallocate disk space during compaction also, add cleanup for failed compaction 2017-08-29 23:59:53 -07:00
Chris Lu
5047bdb4a2 skip bytes cache 2017-06-03 11:44:24 -07:00
Chris Lu
82c3ccc8dd add btree for volume index 2017-05-26 22:51:25 -07:00
brstgt
4fb5bb09b6 Remove obsolete property Volume.dataFileSize 2017-01-20 16:31:11 +01:00
Chris Lu
ed44f12f6d support Fallocate on linux 2017-01-08 11:01:46 -08:00
Chris Lu
13e7069eb9 keep track of total data file size 2017-01-06 10:22:20 -08:00
霍晓栋
ce1f7ab662 makediff func with UT case 2016-10-07 16:22:24 +08:00
霍晓栋
ed848425c7 supplemental data between compacting and commit compacting 2016-09-29 13:57:23 +08:00
Chris Lu
dffad65f2f fix compilation 2016-09-26 22:30:44 -07:00
Chris Lu
7e29218327 add a template for makeupDiff 2016-09-26 22:26:41 -07:00
Chris Lu
1bc041b46d add a new way to manually compact corrupted volume 
fix https://github.com/chrislusf/seaweedfs/issues/371
 2016-09-22 20:31:20 -07:00
Chris Lu
5ce6bbf076 directory structure change to work with glide 
glide has its own requirements. My previous workaround caused me some
code checkin errors. Need to fix this.
 2016-06-02 18:09:14 -07:00