mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-28 11:57:54 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,171 Commits 32 Branches 296 Tags 153 MiB
12df236def
Commit Graph

1742 Commits

Author SHA1 Message Date
joeslay
48b093b3d2
Merge pull request #7 from chrislusf/master 
merge seaweed master
 2019-09-19 12:00:05 +01:00
j.laycock
476140fd6b minor change to setProcessWorkingSetSize function 2019-09-19 11:59:00 +01:00
Chris Lu
972e881d48 add more logs 
add more logs
 2019-09-16 09:48:30 -07:00
Chris Lu
20ae559437 add logs 
add logs for https://github.com/chrislusf/seaweedfs/issues/1064
 2019-09-16 08:19:10 -07:00
Chris Lu
ad3efbb197 tweaking data types 2019-09-14 01:21:51 -07:00
Chris Lu
b0e4771135
Merge pull request #1060 from divinerapier/master 
fix: non-thread-safe rand will panic
 2019-09-14 01:06:14 -07:00
divinerapier
bb31462b52
fix: thread unsafe 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-09-13 20:06:02 +08:00
j.laycock
6fc6322c90 Change joeslay paths to chrislusf paths 2019-09-12 14:18:21 +01:00
joeslay
d8c34b032f
Merge pull request #6 from chrislusf/master 
merge seaweed master
 2019-09-12 11:44:09 +01:00
Chris Lu
ae53f63680 filer: recursive deletion optionally ignoring any errors 
fix https://github.com/chrislusf/seaweedfs/issues/1062
 2019-09-11 20:26:20 -07:00
Chris Lu
5e9c65469e volume: skip readonly checking when Destroy() 
fix https://github.com/chrislusf/seaweedfs/issues/1063
 2019-09-11 09:45:09 -07:00
j.laycock
c1a928887a Add some comments/documentation 2019-09-11 10:42:37 +01:00
joeslay
9ea9a5e97a
Merge pull request #5 from chrislusf/master 
merge seaweed master
 2019-09-11 10:25:27 +01:00
j.laycock
15a4c91c28 Remove Finalizer 2019-09-10 17:05:54 +01:00
j.laycock
9c9dff7386 Virtual lock memory pages and Set Process Working set size to hint to windows harder not to write pages to disk as much, add finalize function, minor renaming 2019-09-10 16:18:51 +01:00
Chris Lu
02710f9624 refactoring: remove unused functions 2019-09-09 10:03:41 -07:00
Chris Lu
0f861d23a2 avoid nil needle map 
fix https://github.com/chrislusf/seaweedfs/issues/1061
 2019-09-09 09:48:08 -07:00
joeslay
d757ed2fe3
Merge pull request #4 from chrislusf/master 
merge seaweed master
 2019-09-06 10:49:27 +01:00
Chris Lu
803345e6af remove glide 2019-09-06 00:15:59 -07:00
j.laycock
63acc8c972 rename mem_buffer to mBuffer 2019-09-05 13:46:29 +01:00
j.laycock
44ae041e80 rename more memorymapped variables to memoryMapMaxSizeMB 2019-09-05 13:42:23 +01:00
j.laycock
5885ab67b3 rename volume property MemoryMap to MemoryMapMaxSizeMB 2019-09-05 11:56:48 +01:00
j.laycock
5a6dfb05fc Only use truncate flag when creating non memory map files, rename and use memory map size for reserving max file size 2019-09-05 11:46:12 +01:00
j.laycock
f6146d6ad2 use 0 instead of false for max memory flag 2019-09-05 11:43:18 +01:00
j.laycock
4257582db5 Allocate in 16MB chunks, make creation of memory maps always aligned to 16MB chunks 2019-09-05 11:15:01 +01:00
j.laycock
84f2dc3b53 Remove test code 2019-09-05 11:14:02 +01:00
Chris Lu
20d90dea5a filer: avoid hard-coded upload timeout 2019-09-05 01:26:59 -07:00
Tom Maxwell
4a878c0006 Changed the InMemory bool to a uint32 so that it can be used to alter how much space to reserve 2019-09-04 15:27:14 +01:00
joeslay
6ee65356e3
Merge pull request #3 from chrislusf/master 
merge seaweed master
 2019-09-04 14:58:04 +01:00
j.laycock
cc756ddd79 Rename End_of_file variable, implement platform indepenent memory_map.go 2019-09-04 14:24:17 +01:00
j.laycock
9aa0859697 Remove inmemory from vaccum stuff 2019-09-03 18:20:45 +01:00
j.laycock
de5a7e60a2 remove temp hack 2019-09-03 18:20:09 +01:00
j.laycock
cbd0a98fa1 Add InMemory to Volume Grow structure 2019-09-03 18:19:02 +01:00
Chris Lu
ee8bcb0c12 remove println 2019-09-03 09:22:56 -07:00
j.laycock
1f01eb78e8 Rename mem_map to mMap, remove some in_memory variables being passed around, added MemoryMapped member to volume struct 2019-09-03 17:00:59 +01:00
Tom Maxwell
d637d86d22 Changes to try and pass the URL parameters through - in memory flag not working still 2019-09-03 15:41:28 +01:00
j.laycock
9a459d984b Do not vacuum memory mapped files 2019-09-03 15:05:43 +01:00
Chris Lu
89a0cb2e11 1.43 2019-09-03 01:31:58 -07:00
j.laycock
0123c7a898 Change to max 2GB for now 2019-09-02 17:46:22 +01:00
joeslay
e91139348e
Merge pull request #2 from chrislusf/master 
merge seaweed master
 2019-09-02 11:39:40 +01:00
j.laycock
595a1beff0 Swap imports to use joeslay 2019-09-02 11:28:40 +01:00
Chris Lu
60c9215a00 HCFS: avoid verbose file not found exception stack trace 2019-09-01 21:40:26 -07:00
j.laycock
1e62a2b233 Fix memory not being unmapped 2019-08-30 16:55:33 +01:00
j.laycock
840ccdc35d Refactor to pass memory maps by reference instead of value, fix memory maps not being created properly or written to properly 2019-08-30 12:30:00 +01:00
j.laycock
dc50701e7d Overload createFile function, expose File in memory_map struct, delete from memory_map map on delete 2019-08-30 12:30:00 +01:00
j.laycock
523f3a12b3 use Os.File/uintptr instead of windows.Handle in memory map struct 2019-08-30 12:30:00 +01:00
j.laycock
0e5d3b1a70 Move memory_map_windows to new folder, intercept most of the read and write calls to volume dat files 2019-08-30 12:30:00 +01:00
Chris Lu
58168a8c52 volume: truncate dat files when creating 2019-08-26 13:01:57 -07:00
joeslay
36bde84000
Merge pull request #1 from chrislusf/master 
merge changes from chrislusf/seaweed master
 2019-08-23 17:03:37 +01:00
j.laycock
1e89e719b3 Add windows memory map functions to storage package 2019-08-23 16:19:00 +01:00
Chris Lu
35d471a818
Revert "use std encoding lib" 2019-08-23 07:59:09 -07:00
Chris Lu
fa160e00f3
Merge pull request #1051 from xushuxun/master 
volume: read dat files' last modified time correctly
 2019-08-20 22:25:37 -07:00
xushuxun
9af9f36b89 volume: read dat files' last modified time no matter dat files are catWrite or readonly 2019-08-21 11:26:01 +08:00
divinerapier
e798e4e9c5
use std encoding lib 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-08-19 18:07:54 +08:00
xushuxun
5904d78bd4 weed backup: add ttl and replication parameter 2019-08-16 11:05:22 +08:00
Chris Lu
f9c1cd7370 avoid dead lock from dataFileAccessLock 
fix https://github.com/chrislusf/seaweedfs/issues/1043
 2019-08-15 09:24:54 -07:00
xushuxun
46ae35cd7e wdclient: clear client cache when failed to connect to master leader 2019-08-15 15:13:16 +08:00
Chris Lu
d829df4f59 volume: protect against nil needle map 
fix @mastak reported nil problem in https://github.com/chrislusf/seaweedfs/issues/1037
 2019-08-14 01:08:01 -07:00
Chris Lu
e40634e6b4 volume: fail the volume deletion if compaction is in progress 
fix https://github.com/chrislusf/seaweedfs/issues/1035
 2019-08-12 00:53:50 -07:00
Chris Lu
6f75df8660 server: fix starting default master list 2019-08-10 03:10:03 -07:00
Chris Lu
0dc43ecdca skip writing content if not modified 
related to https://github.com/chrislusf/seaweedfs/issues/1036
 2019-08-09 14:38:54 -07:00
Chris Lu
b9f6236f8a filer: fix typo 
fix https://github.com/chrislusf/seaweedfs/issues/1031
 2019-08-06 00:42:19 -07:00
Chris Lu
c79274f7eb notes on supported meta data store 2019-08-05 16:45:21 -07:00
Feng Ye
fc0498798b Add etcd filestore 2019-08-01 10:16:45 +08:00
Chris Lu
ca2b81a944 fix test error 2019-07-31 02:09:13 -07:00
Chris Lu
19360ab10a master: redirect clients to the new leader 2019-07-31 01:54:42 -07:00
Chris Lu
8afd8d35b3 master: followers can also lookup and redirect 
improve scalability
 2019-07-28 03:58:13 -07:00
Chris Lu
2c6cf72e73 refactoring 2019-07-28 01:55:05 -07:00
Chris Lu
fb90d63589 shell: Added minor changes to fs.meta.save 
* added an option to set the output file name
* added an option to see verbose output, default to non-verbose mode

related to: https://github.com/chrislusf/seaweedfs/issues/1019
 2019-07-26 09:35:22 -07:00
divinerapier
9e3b77c3ad avoid double warpping 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-07-24 16:41:45 +08:00
Chris Lu
344ce90315 remove weedfuse 2019-07-24 00:42:12 -07:00
Chris Lu
5956dfd08d mount: add umask option 
related to https://github.com/chrislusf/seaweedfs/issues/978
 2019-07-24 00:03:05 -07:00
Chris Lu
3f851feb59 1.42 2019-07-21 21:52:48 -07:00
Chris Lu
862c2cb8e6 go fmt 2019-07-21 21:51:38 -07:00
Chris Lu
79762385bd master: ensure only one exclusive vacuum process 
fix https://github.com/chrislusf/seaweedfs/issues/1011
 2019-07-21 21:49:10 -07:00
Chris Lu
f3b99cbfe5 fix wrong alignment fix 2019-07-21 13:50:42 -07:00
Chris Lu
644c5c27b4 refactoring 2019-07-21 13:50:24 -07:00
Chris Lu
33b3366043 volume: avoid compacting wait group 2019-07-21 13:49:59 -07:00
Chris Lu
4b15c8f0c4 volume: lock writables changes 2019-07-21 13:49:09 -07:00
Chris Lu
fd16adbde1 stricter checking before writes 2019-07-21 11:21:30 -07:00
Chris Lu
5a135fef87 purge old code 2019-07-21 10:45:36 -07:00
Chris Lu
c54d9221b9 write requests also checks cookie if overwrites 
protect against edge cases, avoid https://github.com/chrislusf/seaweedfs/issues/1014
 2019-07-17 23:57:34 -07:00
Chris Lu
898d943b25 refactoring 2019-07-17 23:43:48 -07:00
Chris Lu
0264a7f50a set the max file key when ever possible 2019-07-17 23:23:01 -07:00
Chris Lu
c33f423955 stop early if compaction fails 
fix https://github.com/chrislusf/seaweedfs/issues/1015
 2019-07-17 23:22:01 -07:00
Chris Lu
aff911c00d skip all azuresb 2019-07-17 01:24:20 -07:00
Chris Lu
0ea98dc6a2 disable azure due to api changes 2019-07-17 00:39:36 -07:00
Chris Lu
a7b1b23c58 fix wrong volume count 
fix https://github.com/chrislusf/seaweedfs/issues/1013
 2019-07-17 00:03:15 -07:00
Chris Lu
9e47c4749a shell: add fs.mv 
fix https://github.com/chrislusf/seaweedfs/issues/954
 2019-07-11 09:29:04 -07:00
Chris Lu
1d1b355f9e filer: set file name when uploading to a directory 
fix https://github.com/chrislusf/seaweedfs/issues/1005
 2019-07-11 04:20:03 -07:00
Chris Lu
61b7a650f4 filer.copy default to 32MB chunks 2019-07-10 21:48:27 -07:00
Chris Lu
476a7c269c detect a filer path is a file or a directory 
fix https://github.com/chrislusf/seaweedfs/issues/1004
 2019-07-10 12:11:18 -07:00
Chris Lu
1e2127e261 s3: adding Location for multipart upload 
fix https://github.com/chrislusf/seaweedfs/issues/1002
 2019-07-08 12:37:45 -07:00
Chris Lu
62843991f2 s3: adjust object key 2019-07-08 12:37:20 -07:00
Chris Lu
cf2804eebd 1.41 2019-07-07 20:40:20 -07:00
Chris Lu
7872fc38ea filer leveldb2: correct directory listing 
fix https://github.com/chrislusf/seaweedfs/issues/1001
 2019-07-07 20:37:48 -07:00
Chris Lu
fe2882ce7d fix test 2019-07-02 21:42:06 -07:00
Chris Lu
36250154cc remove println 2019-07-02 21:33:23 -07:00
Chris Lu
4d579f7f57 adjust 2019-07-02 21:28:51 -07:00
Chris Lu
23d36dfbaf 1.40 
enable azure replication
 2019-07-02 21:27:05 -07:00
Chris Lu
dcc331cf5a parallelize leveldb 2019-07-02 21:25:53 -07:00
Chris Lu
88fed85bba temp skipping azure 
can not release due to https://github.com/Azure/azure-pipeline-go/issues/23
 2019-06-30 23:17:45 -07:00
Chris Lu
ce5aab66e9 add versions 2019-06-30 22:43:10 -07:00
Chris Lu
36022f9416 1.39 2019-06-30 00:50:17 -07:00
Chris Lu
15ecf4e3bf filer: default to leveldb2 2019-06-30 00:44:57 -07:00
Chris Lu
ee933cd859 filer: add metrics for go memory, gc stats 2019-06-29 03:02:28 -07:00
Chris Lu
92c7f7e069 fix compilation error 2019-06-28 11:19:08 -07:00
Chris Lu
9f3f2f7c79 protect locations slice 
fix https://github.com/chrislusf/seaweedfs/issues/995
 2019-06-28 09:47:29 -07:00
Chris Lu
327336ecf3 filer: avoid concurrent modification to result slice 
fix https://github.com/chrislusf/seaweedfs/issues/972
 2019-06-28 01:12:41 -07:00
Chris Lu
a3d1296ed9 go fmt 2019-06-27 12:18:59 -07:00
Chris Lu
ec75b2d761 volume: fix bug with 8000GB version if using in memory index 
fix https://github.com/chrislusf/seaweedfs/issues/994
 2019-06-27 12:18:45 -07:00
Chris Lu
6883f9e322 mark volume readonly before ec encoding 2019-06-26 23:02:22 -07:00
Chris Lu
c2960c7a86 defaulting file ttl to volume ttl 
fix https://github.com/chrislusf/seaweedfs/issues/992
 2019-06-26 11:18:23 -07:00
Lei Liu
1d9b75b536
weed.go: remove unused parameter 
Signed-off-by: Lei Liu <liul.stone@gmail.com>
 2019-06-26 10:46:32 +08:00
Chris Lu
359563e95a refactoring 2019-06-25 09:49:27 -07:00
Chris Lu
fdff927b37 1.38 2019-06-24 00:36:46 -07:00
Chris Lu
a8b4fe6eae adjust help text 2019-06-24 00:26:03 -07:00
Chris Lu
c777102da9 go fmt 2019-06-23 15:30:16 -07:00
Chris Lu
d5560f2705 simplify metrics settings 2019-06-23 15:29:49 -07:00
Chris Lu
6f8b335007 fix duplicated flags 2019-06-23 03:11:21 -07:00
Chris Lu
f16375621f big refactoring 2019-06-23 03:08:27 -07:00
Chris Lu
6c01fb6d2d go fmt 2019-06-23 01:57:51 -07:00
Chris Lu
8da5d5b094 filer.copy: use filer settings, avoid unnecessary command line options 
fix https://github.com/chrislusf/seaweedfs/issues/968
 2019-06-23 01:57:35 -07:00
Chris Lu
d1cd8f8c5b add metrics, refactoring 2019-06-22 22:53:52 -07:00
Chris Lu
a14bd31f5b go fmt 2019-06-22 20:05:25 -07:00
Chris Lu
3fa1f150d9 refactoring 2019-06-22 20:04:56 -07:00
Chris Lu
1babec00e7 check deleted chunks faster 2019-06-22 13:22:22 -07:00
Chris Lu
cd45ab072a fix compilation error 2019-06-22 12:30:08 -07:00
Chris Lu
c369e5a13b add metrics for filer store 2019-06-22 12:23:25 -07:00
Chris Lu
6bc3dee5b3 refactoring 2019-06-22 10:56:54 -07:00
Chris Lu
95ad56d99d 1.37 2019-06-21 20:56:34 -07:00
Chris Lu
a111f26fe6 avoid nil 
fix https://github.com/chrislusf/seaweedfs/issues/988
 2019-06-21 20:56:27 -07:00
Chris Lu
308102f023 fix cassandra creation problem 
fix https://github.com/chrislusf/seaweedfs/issues/986
 2019-06-21 17:10:38 -07:00
Chris Lu
fe4eb6bffa 1.36 2019-06-21 13:11:40 -07:00
Chris Lu
059ef879a8 fix issue 986 
fix issue 986
 2019-06-21 13:06:04 -07:00
Chris Lu
ff97acae62 more logs 2019-06-21 12:14:40 -07:00
Chris Lu
d0ce5b020b logging 2019-06-21 12:05:00 -07:00
Chris Lu
78994024ff adjust logging 2019-06-21 11:46:12 -07:00
Chris Lu
f88a8bda7b ec deletion works 2019-06-21 01:14:10 -07:00
Chris Lu
613a2e8060 delete garbage only when successful 2019-06-20 23:46:00 -07:00
Chris Lu
11be0b5e91 more logging 2019-06-20 23:45:30 -07:00
Chris Lu
4b0f084b5d delete chunks only when file writing is successful 2019-06-20 21:58:35 -07:00
Chris Lu
eebc44cce3 refactoring 2019-06-20 21:57:17 -07:00
Chris Lu
3106065b58 fix startup problem with metrics 2019-06-20 09:56:49 -07:00
Chris Lu
c74dc2b306 go fmt 2019-06-20 00:55:52 -07:00
Chris Lu
e63317fb08 ec deletion code complete, not tested yet 2019-06-20 00:55:30 -07:00
Chris Lu
4cea8aefd0 add grpc VolumeEcBlobDelete 2019-06-20 00:17:11 -07:00
Chris Lu
856da7aae2 ec volume support deletes 2019-06-19 22:57:14 -07:00
Chris Lu
115558e5f5 adjust counters 2019-06-17 21:02:50 -07:00
Chris Lu
d8ed73926d volume servers get metrics address and interval from the master 2019-06-17 14:51:47 -07:00
Chris Lu
68d1bef236 instance format 2019-06-16 21:57:32 -07:00
Chris Lu
935639b908 metrics: disk size for <collection, normal/EC> 2019-06-16 21:56:41 -07:00
Chris Lu
0fdb1e705d collect volume disk usage metrics 2019-06-16 02:44:20 -07:00
Chris Lu
289fd7eb39 count number of volumes and ec shards 2019-06-16 02:24:15 -07:00
Chris Lu
ff6a6dd11e refactoring 2019-06-15 21:46:55 -07:00
Chris Lu
8b43679ae3 refactoring 2019-06-15 12:21:44 -07:00
Chris Lu
5336008dcd refactoring 2019-06-14 13:06:01 -07:00
Chris Lu
d67189921f go fmt 2019-06-14 00:55:12 -07:00
Chris Lu
5f6c9825f8 volume server: adds basic metrics 2019-06-14 00:54:56 -07:00
Chris Lu
a11525fe4e filer: adds basic metrics pushing to Prometheus gateway 2019-06-13 02:01:54 -07:00
Chris Lu
be3f1f84a2 1.35 2019-06-10 21:34:06 -07:00
Chris Lu
57092d69ac go fmt 2019-06-10 21:33:32 -07:00
Chris Lu
f9d8bd51ad ec shard balancing 2019-06-10 21:32:56 -07:00
Chris Lu
9d9162ca35 ec.balance: collect dc rack info 2019-06-07 00:25:05 -07:00
Chris Lu
e66bddd84c customizable sleep intervals 2019-06-06 00:39:08 -07:00
Chris Lu
50aa769554 jwt for read access control 2019-06-06 00:29:02 -07:00
Chris Lu
d344e0a035 fix ec related bugs 2019-06-05 23:20:26 -07:00
Chris Lu
450f4733ce report empty volume or ec shards 2019-06-05 13:32:33 -07:00
Chris Lu
7f6fb9ffd0 every 17 minutes instead of seconds 
TODO: make this configurable
 2019-06-05 13:32:08 -07:00
Chris Lu
6b08db65b0 fix shard count reporting 2019-06-05 01:58:37 -07:00
Chris Lu
784141c5e6 adjust script parameter 2019-06-05 01:48:03 -07:00
Chris Lu
ede876cfdb periodic scripts exeuction from leader master 2019-06-05 01:30:24 -07:00
Chris Lu
b9e138713c ec.encode: add -fullPercent option 2019-06-05 00:13:13 -07:00
Chris Lu
0e49add2bc adjust help message 2019-06-05 00:11:37 -07:00
Chris Lu
eaa76f11b7 free volume slots factor in ec shard counts 2019-06-04 23:41:56 -07:00
Chris Lu
ca8a2bb534 go fmt 2019-06-04 22:04:10 -07:00
Chris Lu
2215e81be7 ui add ec shard statuses 2019-06-04 21:52:37 -07:00
Chris Lu
0e52862586 avoid the test that depends on ordering in a different folder 2019-06-04 02:22:32 -07:00
Chris Lu
2f7710a75d debug sporadic test error on travis 2019-06-04 02:09:06 -07:00
Chris Lu
06133ae98f correct count for free ec slots, avoid removing generated shards 2019-06-04 01:49:51 -07:00
Chris Lu
6cea23d091 purge duplicated ec shards 2019-06-04 01:32:36 -07:00
Chris Lu
b05456fe07 able to purge extra ec shard copies 2019-06-03 20:25:02 -07:00
Chris Lu
11cffb3168 fix ec.rebuild bugs 2019-06-03 11:50:54 -07:00
Chris Lu
d85b41b904 fix ec.encode not finding the local ec shards 2019-06-03 10:38:21 -07:00
Chris Lu
7e80b2b882 fix multiple bugs 2019-06-03 02:26:31 -07:00
Chris Lu
55be09996d fix volume balance bug 2019-06-03 00:13:31 -07:00
Chris Lu
9ce6b949bf adjust help message 2019-06-01 10:37:08 -07:00
Chris Lu
d6e2c877fa ec.balance: use specific collection names 2019-06-01 02:00:18 -07:00
Chris Lu
133b772fb5 destroy ec volume if it is empty 2019-06-01 01:51:28 -07:00
Chris Lu
ba18314aab ec shard delete also check ec volumes, in addition to volumes 2019-06-01 01:41:22 -07:00
Chris Lu
f919d0235c ec encode volumes quiet for a period of time 2019-05-31 23:41:17 -07:00
Chris Lu
a72cef3c42 encode by collection 2019-05-31 15:48:40 -07:00
Chris Lu
de1a35acdd conditionally fresh the shard locations 2019-05-31 02:22:12 -07:00
Chris Lu
47f1901843 ask for the ec volume version 2019-05-31 00:58:51 -07:00
Chris Lu
689930f092 forget shards that are broken 2019-05-31 00:19:13 -07:00
Chris Lu
aaab2c25f5 resolve concurrent modification problem 
n and err can be modified concurrently
 2019-05-30 13:59:07 -07:00
Chris Lu
f53024d79d delete ec collection 2019-05-30 10:40:22 -07:00
Chris Lu
40ca2f2903 add collection.delete 2019-05-30 09:47:54 -07:00
Chris Lu
1d111d6ce8 collection list normal and ec volumes 2019-05-30 09:27:23 -07:00
Chris Lu
e941d0c2f4 list ec collections also 2019-05-30 09:17:58 -07:00
Chris Lu
866197eee3 print out the ec balancing plan 2019-05-30 01:38:59 -07:00
Chris Lu
5c6c757619 clean up test files 2019-05-29 00:34:30 -07:00
Chris Lu
5dd67f9acf reading by recover from other shards 2019-05-28 23:48:39 -07:00
Chris Lu
3f9ecee40f working with reading remote intervals 2019-05-28 21:29:07 -07:00
Chris Lu
302d9fbc6d adjust error message for file not found 2019-05-28 14:27:21 -07:00
Chris Lu
4f76342cbc WIP 
no errors, but not returning file content
* the interval needs to use actual file zie
* need to read the actual version instead of the current version
 2019-05-28 00:51:01 -07:00
Chris Lu
2858a954b3 read ec volume shards locations from master 2019-05-28 00:13:13 -07:00
Chris Lu
713596e781 caching ec shard locations 2019-05-27 22:54:58 -07:00
Chris Lu
3a8c1055a2 refactoring ecx to ecVolume 2019-05-27 22:00:36 -07:00
Chris Lu
217cde0a3b refactoring 2019-05-27 21:40:51 -07:00
Chris Lu
03b9291e5d volum server passes grpc option and master location to store 2019-05-27 21:22:23 -07:00
Chris Lu
ac1ba3b667 remove comments 2019-05-27 21:21:52 -07:00
Chris Lu
b4b407e403 add grpc ec shard read 2019-05-27 11:59:03 -07:00
Chris Lu
a4f3d82c57 convert needle id to ec intervals to read from 2019-05-27 01:29:46 -07:00
Chris Lu
a463759edf adjust error message 2019-05-26 01:14:42 -07:00
Chris Lu
92f9c7b1aa adjust ec shard status on disconnect 2019-05-26 01:05:08 -07:00
Chris Lu
ac14cd5ad0 wdclient need to have ec shard info on initial dump 2019-05-26 01:01:04 -07:00
Chris Lu
b58e25e588 broadcast vid->location map to master clients for ec chards 2019-05-26 00:49:15 -07:00
Chris Lu
af67d99ca4 incrementally update master ec shards state 2019-05-26 00:21:17 -07:00
Chris Lu
db94a41f9e mount/unmount ec shards 2019-05-25 23:23:19 -07:00
Chris Lu
41e8ae61f8 generate, balance, delete copied shards, delete old volume 2019-05-25 14:02:06 -07:00
Chris Lu
f0e6574d5e allocate ec shards to volume servers 2019-05-25 02:02:44 -07:00
Chris Lu
6f4b09b6a4 pb shard info uses ShardBits instead one message for one shard 2019-05-24 13:28:44 -07:00
Chris Lu
228850d588 shard id starts from zero 2019-05-24 11:52:23 -07:00
Chris Lu
24b8ad9a14 push the ec shard info to master 2019-05-23 23:47:49 -07:00
Chris Lu
8838176d82 register ec shards to topology 2019-05-23 23:34:29 -07:00
Chris Lu
8a96445f40 register ec shards to each data node 2019-05-23 22:51:18 -07:00
Chris Lu
4659d80035 prepare to register ec shard info in master 2019-05-23 00:42:28 -07:00
Chris Lu
e913ee380a add comments 2019-05-23 00:04:24 -07:00
Chris Lu
7180520889 ec shard info can be queried via VolumeList() 2019-05-22 22:44:28 -07:00
Chris Lu
17ac1290c0 volume: load ec shards during heartbeats to master 2019-05-21 22:41:20 -07:00
Chris Lu
54b835e1ae filer: redis cluster add option for password 
fix https://github.com/chrislusf/seaweedfs/issues/971
 2019-05-20 09:00:30 -07:00
Chris Lu
fbbc74abb4 adds VolumeEcGenerateSlices, VolumeEcCopy 2019-05-20 00:53:17 -07:00
Chris Lu
ae499fd5aa Merge branch 'master' into erasure_coding 2019-05-19 21:38:18 -07:00
Chris Lu
698dea779c refactoring 2019-05-19 21:37:49 -07:00
Chris Lu
693d6c0f1b clean up tests 2019-05-19 21:16:01 -07:00
Chris Lu
0094ca3f8a mark the webdav as unstable 2019-05-19 21:08:26 -07:00
Chris Lu
6386a3174b able to validate by randomly selected ec files 2019-05-19 14:24:33 -07:00
Chris Lu
7c2c60c376 add locating data inside the ec files 2019-05-19 03:01:58 -07:00
Chris Lu
87f63b9c08 generate ec01~ec14, generate ecx file with sorted needle values 2019-05-18 22:46:24 -07:00
Chris Lu
12dc6608f0 Merge branch 'master' into erasure_coding 2019-05-18 11:16:07 -07:00
Chris Lu
f2c4c888f6 1.34 2019-05-17 22:53:30 -07:00
Chris Lu
8123a488cd filer: optimize leveldb parameters 2019-05-17 18:14:25 -07:00
Chris Lu
76cf57a0e1 volume: optimzation for leveldb 2019-05-17 17:34:03 -07:00
Chris Lu
0fe286a6ca filer: optimization for leveldb, add leveldb2 
large filer db may see high CPU and disk usage due to background compaction
 2019-05-17 17:33:49 -07:00
Chris Lu
7e0c3bb732 filer: adjust recommended mysql meta data type to LONGBLOB 
fix https://github.com/chrislusf/seaweedfs/issues/966
 2019-05-17 09:11:08 -07:00
Chris Lu
bf9d490538 fix for tests 2019-05-17 02:28:20 -07:00
Chris Lu
82b0759493 filer: migrating filer store from persisting shorter structured file id instead of a string 2019-05-17 02:03:23 -07:00
Chris Lu
8156958ee9 move function to make travis happy 2019-05-15 10:02:44 -07:00
Chris Lu
0a36f628c6 testing RS coding 2019-05-15 01:02:00 -07:00
Chris Lu
ab22407763 filer ip only use bind ip 2019-05-14 23:43:40 -07:00
Chris Lu
017d0957c3 avoid switching master node if same ip and host name 
fix https://github.com/chrislusf/seaweedfs/issues/955
 2019-05-11 18:42:46 -07:00
Chris Lu
4dfcd2169e weedfuse: adapt to fstab command line pattern 2019-05-11 01:17:55 -07:00
Chris Lu
9a4fb14ea0 weedfuse: daemonize 2019-05-10 17:08:15 -07:00
Chris Lu
1ca1ec906a add weedfuse 2019-05-10 15:03:31 -07:00
Chris Lu
69b7dd398d adjust help message 2019-05-07 14:02:01 -07:00
Chris Lu
d01807ba05 shell: volume.balance add option for all_collections, each_collection 2019-05-06 14:12:19 -07:00
Chris Lu
8ffdd5dcdb writel throttling moving volumes 2019-05-06 13:58:42 -07:00
Chris Lu
cf58fc0e63 refactor: extract out the write throttler 2019-05-06 13:56:08 -07:00
Chris Lu
4e42e7b5e7 shell: volume.balance add dataCenter filter 2019-05-06 13:30:14 -07:00
Chris Lu
4c392ca762 use ip.bind to listen for grpc calls 
help to avoid https://github.com/chrislusf/seaweedfs/issues/952
 2019-05-06 08:10:18 -07:00
Chris Lu
8656102127 shell: volume.balance runs for each collection, and for ALL at the end 2019-05-05 22:28:14 -07:00
Chris Lu
f08bbb72de refactoring 2019-05-05 22:21:28 -07:00
Chris Lu
0df6346611 add error checking 2019-05-05 21:58:46 -07:00
Chris Lu
3ff92037cc shell: add volume.balance command 2019-05-05 21:17:23 -07:00
Chris Lu
3f2a30bd05 shell: volume.list output sorted voume list 2019-05-05 20:23:50 -07:00
Chris Lu
46a675ecc2 align memory to avoid nil with atomic operations 2019-05-04 21:33:05 -07:00
Chris Lu
177f3add81 adjust memory layout to avoid possible nil pointer 2019-05-04 08:47:11 -07:00
Chris Lu
25941e0500 master: add jwt expires_after_seconds 2019-05-04 08:42:25 -07:00
Chris Lu
bd8af92b54 remove unused code 2019-05-04 08:20:22 -07:00
Chris Lu
4b27f892e6 remove 128 bit needle id build option 
fix https://github.com/chrislusf/seaweedfs/issues/949
 2019-05-03 22:39:00 -07:00
Chris Lu
3fbe22b203 better error handling for nil raftServer 
addressing https://github.com/chrislusf/seaweedfs/issues/717
 2019-05-03 22:33:08 -07:00
Chris Lu
5681208f2d refactor: remove logger 2019-05-03 22:21:06 -07:00
Chris Lu
8c982db25d 1.33 2019-05-03 17:27:50 -07:00
Chris Lu
b335f81a4f volume: add option to limit compaction speed 2019-05-03 17:22:39 -07:00
Chris Lu
f0f981e7c8 webdav: fix directory renaming 2019-05-03 14:12:51 -07:00
Chris Lu
bfbecd7253 webdav: fix directory creation 2019-05-03 13:13:11 -07:00
Chris Lu
9d3c945b00
Merge pull request #950 from chrislusf/webdav 
add WebDav
 2019-05-03 01:25:10 -07:00
Chris Lu
7064b2ea48 webdav: can write now 2019-05-03 00:55:52 -07:00
Chris Lu
b30c14b631 webdav: can read now 2019-05-03 00:24:35 -07:00
Chris Lu
47f14775d7 scaffolding, but seems hard to satisfy mac, windows, and different os versions 
https://www.reddit.com/r/golang/comments/3wrxo3/webdav_server_that_provide_filesystem_using/
 2019-05-02 14:22:10 -07:00
Wine93
32f93fb09e style: go fmt for all 2019-04-30 03:22:19 +00:00
Chris Lu
82e06a780b mount: allow nonempty fuse mount to work with Samba 
possibly fix https://github.com/chrislusf/seaweedfs/issues/936
 2019-04-29 10:42:09 -07:00
Chris Lu
108d0fb08d adjust log level 2019-04-26 09:32:37 -07:00
Chris Lu
575926cdec minor 2019-04-26 09:32:37 -07:00
Chris Lu
630cf1e33e fix 2019-04-24 12:31:52 -07:00
Chris Lu
ea4592161c fix 2019-04-24 12:31:14 -07:00
Chris Lu
c5a931b22c logging 2019-04-24 12:30:36 -07:00
Chris Lu
9beea63960 memory alignment 
fix https://github.com/chrislusf/seaweedfs/issues/939
 2019-04-24 09:23:19 -07:00
Chris Lu
9774ede3cf adjust usage description 2019-04-24 00:25:20 -07:00
Chris Lu
b04d7e3ac0 weed server: also optionally start S3 gateway 2019-04-24 00:18:01 -07:00
Chris Lu
dabc9c9a89 filer: listen on specified ip address 2019-04-24 00:01:39 -07:00
Chris Lu
21fe5bde7c filer: remove customizable grpc port, fixed to filer port + 10000 2019-04-23 23:58:52 -07:00
Chris Lu
067b935763 report unknown commands 2019-04-23 22:29:36 -07:00
Chris Lu
53e2e73de2 fix metrics 
fix https://github.com/chrislusf/seaweedfs/issues/935
 2019-04-22 12:32:10 -07:00
Chris Lu
0c96cfdb2e minor 2019-04-21 15:50:51 -07:00
Chris Lu
00b6f653fa shell: add fs.cat 2019-04-21 15:43:43 -07:00
Chris Lu
59d532a8c3 1.31 2019-04-21 14:09:50 -07:00
Chris Lu
5c411f3e5f minor 2019-04-21 13:33:32 -07:00
Chris Lu
440111a349 volume: support http status 304 for the same file id 2019-04-21 13:33:23 -07:00
Chris Lu
ebe94be1a1 maintain layout correctness when changing volumes 2019-04-21 13:32:36 -07:00
Chris Lu
0302b9496c volume: also check cookie for possible duplicates 2019-04-21 13:31:45 -07:00
Chris Lu
132921ad41 adjust log or error 2019-04-21 10:14:17 -07:00
Chris Lu
316bd27f75 add remove volumes with version info 2019-04-20 23:53:37 -07:00
Chris Lu
64a9a0e104 shell: fix volume.copy 2019-04-20 20:48:07 -07:00
Chris Lu
5f3d0e33a1 shell: add commands volume.copy volume.delete volume.mount volume.unmount 2019-04-20 20:31:35 -07:00
Chris Lu
d06d3c57d4 add notes 2019-04-20 20:12:07 -07:00
Chris Lu
2ffe98443b refactoring 2019-04-20 12:05:28 -07:00
Chris Lu
6fc1f53018 shell: add command volume.move 2019-04-20 11:35:23 -07:00
stlpmo-jn
2200ea9cb9 add function ParseFileIdFromString 2019-04-20 18:39:06 +08:00
Chris Lu
3b3651dea3 volume: atomic copying file, adds version and stopOffset 2019-04-19 12:29:49 -07:00
Chris Lu
730a032137 filer mysql: adjust scaffolding instructions 2019-04-19 09:55:46 -07:00
Chris Lu
0be2d51c96 read volume lastAppendAtNs when loading a volume 2019-04-19 01:56:38 -07:00
Chris Lu
ac2727853f fix needle map entry size 2019-04-19 00:39:34 -07:00
Chris Lu
e5506152c0 refactoring 2019-04-18 21:43:36 -07:00
Chris Lu
33c92b819a refactoring 2019-04-18 21:17:43 -07:00
Chris Lu
a2d34d4802 filer.copy: add adjustable compression level 2019-04-18 20:21:28 -07:00
Chris Lu
fa176fe80f volume tailing chunks large files 2019-04-18 19:22:13 -07:00
Chris Lu
3dce1016cb add volume tailer 2019-04-18 11:05:02 -07:00
Chris Lu
b142f9f1d5 go fmt 2019-04-18 00:19:18 -07:00
Chris Lu
b09e8dbb37 add VolumeStreamFollow, but not used yet 2019-04-18 00:18:29 -07:00
Chris Lu
13ad5c1966 refactoring 2019-04-17 22:04:49 -07:00
Chris Lu
3b9446606a adjust log level 2019-04-16 23:31:44 -07:00
Chris Lu
07091c44cd lock btree writes 
fix https://github.com/chrislusf/seaweedfs/issues/930
 2019-04-16 19:48:07 -07:00
Chris Lu
4393b99332 add notes 2019-04-16 13:57:45 -07:00
Chris Lu
338e6d60a5 refactor: prepare for snapshotting 2019-04-16 09:55:37 -07:00
Chris Lu
af49aea0c6 weed shell: fs.meta.save adjusts meta data file name 2019-04-16 09:40:27 -07:00
Chris Lu
8ea1ee6dfa weed shell: add fs.meta.notify, removing filer.export 2019-04-16 01:58:28 -07:00
Chris Lu
014906ec79 weed shell: add command fs.meta.load to restore meta data 2019-04-16 01:37:11 -07:00
Chris Lu
79c2cca9c1 better error message 2019-04-16 01:15:30 -07:00
Chris Lu
d35023c713 weed shell: add command fs.meta.save 2019-04-16 01:06:32 -07:00
Chris Lu
b3b42bc947 replicate need to include new entry path 2019-04-16 00:44:31 -07:00
Chris Lu
967e108b9a increase max idle conn per hosts 2019-04-15 22:00:50 -07:00
Chris Lu
f567777d64 volume: fix needle checking for deleted needles 2019-04-15 21:58:43 -07:00
Chris Lu
bd76ad0ff1 Revert "Revert "volume: close request body if any parsing error"" 
This reverts commit 359b7c1650.
 2019-04-15 21:43:28 -07:00
Chris Lu
359b7c1650 Revert "volume: close request body if any parsing error" 
This reverts commit 5996656141.
 2019-04-15 21:35:12 -07:00
Chris Lu
5996656141 volume: close request body if any parsing error 2019-04-15 21:11:45 -07:00
Chris Lu
6e116b3d67 volume: reset leader if error happens 2019-04-15 09:09:46 -07:00
Chris Lu
e85048bcdc http exhaust and close response body 2019-04-14 23:28:24 -07:00
Chris Lu
3e8a3a8fec fix race detector found problems 2019-04-14 23:00:37 -07:00
Chris Lu
2e03f3adef filer: rename -port.public to -port.readonly 2019-04-11 20:42:55 -07:00
Chris Lu
59a9e15733 1.30 2019-04-11 19:40:36 -07:00
Chris Lu
e11e127d91 close grpc connections during errors 2019-04-11 09:27:08 -07:00
Wine93
361912224d typo: remove blank 2019-04-11 09:18:53 +00:00
Chris Lu
0e33272f56 small refactoring 2019-04-10 23:39:53 -07:00
Chris Lu
cb2061f86a
Merge pull request #925 from stlpmo-jn/checkUnhealthReplication 
the implement of repair the error replications of the volume
 2019-04-10 23:37:12 -07:00
Chris Lu
c0d6cc0067 weed mount: mount to the same folder name 2019-04-10 23:25:41 -07:00
stlpmo-jn
c3fa50d3b3 remove the health chekcer, because it's the same as command_volume_fix_replication 2019-04-11 13:40:31 +08:00
Chris Lu
2f76681d62 refactor: adjust grpc API name 2019-04-10 21:41:17 -07:00
stlpmo-jn
f2031884f0 fix bug : CI build failed 2019-04-11 09:53:31 +08:00
Chris Lu
4df10e50b3 update glide.yaml 2019-04-10 14:27:28 -07:00
stlpmo-jn
c1a0403da2 repair the error replications of the volume 2019-04-10 19:41:55 +08:00
Chris Lu
3555628ad5 weed volume: fix bug, remove boltdb, btree 2019-04-09 10:08:59 -07:00
Chris Lu
8a5ce16e96 weed volume: remove boltdb, btree options, add options for leveldb medium, large 2019-04-09 09:42:06 -07:00
Chris Lu
9924fa3b1a weed volume: disable block cache on leveldb 
saving 8MB for each volume, but at the cost of slower lookup
 2019-04-09 00:24:32 -07:00
Chris Lu
35aba35c07 Revert "weed volume: disable block cache on leveldb" 
This reverts commit a55e708d36.
 2019-04-09 00:22:40 -07:00
Chris Lu
a55e708d36 weed volume: disable block cache on leveldb 
saving 8MB for each volume, but at the cost of slower (offset,size) lookup
 2019-04-09 00:18:32 -07:00
Chris Lu
6d4c44a85c 1.29 2019-04-08 22:41:59 -07:00
Chris Lu
88ba08cb9e add large disk support 2019-04-08 22:32:42 -07:00
Chris Lu
784c5bb73a add build option support 5-byte offset 2019-04-08 22:01:29 -07:00
Chris Lu
7a4b234ea1 divide offset into higher and lower sections 2019-04-08 21:44:06 -07:00
Chris Lu
000ee725fc refactor Offset into a struct of bytes 2019-04-08 19:40:56 -07:00
Chris Lu
a41ba79119
Merge pull request #914 from jba/gocdk-receive 
replication: add GoCDK PubSub support
 2019-04-08 10:31:02 -07:00
Chris Lu
d14b614407 weed filer.copy: use existing file owner and gropu id 2019-04-07 11:31:50 -07:00
Chris Lu
a32abda1a3 adjust compression to optimize for speed 2019-04-07 11:01:28 -07:00
Chris Lu
c731d9be6d weed filer.copy: add option to profile the execution 2019-04-07 09:13:24 -07:00
Chris Lu
174bf1e8b2 more efficient client side gzip compression 2019-04-06 14:14:28 -07:00
Chris Lu
868913aa04 printout statistics 2019-04-06 11:12:35 -07:00
Chris Lu
de7626bd22 weed shell: volume.list add summary statistics 2019-04-06 09:25:29 -07:00
Chris Lu
94302935b2 weed volume: only store compressed data if compression is effective 2019-04-06 00:10:52 -07:00
Chris Lu
b2d92a2992 weed shell: fs.tree fix directory tree listing 2019-04-05 23:42:36 -07:00
Chris Lu
24c020104a weed filer.copy: parallelize the copying to increase throughput 
improvement https://github.com/chrislusf/seaweedfs/issues/915
 2019-04-05 23:35:30 -07:00
Chris Lu
c789b496d8 use cached grpc client 2019-04-05 20:31:58 -07:00
Chris Lu
5808caa2f5 use cached grpc client 2019-04-05 20:26:52 -07:00
Chris Lu
cd6d35aa52 weed shell: fs.tree improvements 2019-04-05 11:40:54 -07:00
Jonathan Amsterdam
72920efc20 added entry to scaffold 2019-04-05 13:43:38 -04:00
Jonathan Amsterdam
a6a5d80401 fix typo 2019-04-05 08:13:32 -04:00
Chris Lu
300b39b5af weed filer.copy: use cached filer grpc connection 
another attemp for https://github.com/chrislusf/seaweedfs/issues/915
 2019-04-05 01:09:06 -07:00
Chris Lu
921231268b weed filer.copy: skip files that can not be opened 
such as socket
 2019-04-05 00:04:00 -07:00
Chris Lu
4f714cef22 default weed filer.copy and weed upload chunk size to 32MB 2019-04-04 23:14:32 -07:00
Chris Lu
c94823f52c set default http idle connection per host 
possible fix https://github.com/chrislusf/seaweedfs/issues/915
 2019-04-04 20:28:40 -07:00
Chris Lu
0bd7ced7c2 fix test 2019-04-04 20:11:59 -07:00
Chris Lu
9cc73f4a9a fix test 2019-04-04 19:34:15 -07:00
Chris Lu
af37b374cb weed shell: fs.cd change current directory 2019-04-04 19:27:51 -07:00
Chris Lu
766396d249 weed master: atomic volume counting 
possible fix for https://github.com/chrislusf/seaweedfs/issues/913
 2019-04-04 19:27:00 -07:00
Jonathan Amsterdam
c0c9a8bad5 replication: add GoCDK PubSub support 2019-04-04 17:22:45 -04:00
Chris Lu
715a38da1e weed shell: add fs.cd, fs.pwd to change to a directory and print current directory 2019-04-03 00:20:00 -07:00
Chris Lu
20dcb44077 fix tests 2019-04-01 23:59:31 -07:00
Chris Lu
beaa2bd71a weed filer: properly delete cached directory entry 
fix https://github.com/chrislusf/seaweedfs/issues/910
 2019-04-01 17:03:04 -07:00
Chris Lu
78b9db34d5 weed s3: generate time format acceptable by aws cli s3 
fix https://github.com/chrislusf/seaweedfs/issues/911
 2019-04-01 16:13:29 -07:00
Chris Lu
2a52e70a41 weed filer: increase default mysql/postgres dir or name length to 65535 2019-04-01 12:37:54 -07:00
Chris Lu
b4c276263f weed filer: mysql/postgres use ReadCommitted transaction isolation level for atomic rename 2019-04-01 11:03:04 -07:00
Chris Lu
c5a3ff1c35 weed s3: multipart upload goes to the right bucket 
fix https://github.com/chrislusf/seaweedfs/issues/908
 2019-03-31 11:10:19 -07:00
Chris Lu
189c890715 weed replicate: replicate atomic rename to other systems 2019-03-31 08:10:47 -07:00
Chris Lu
8c823abe1f 1.28 2019-03-30 23:22:41 -07:00
Chris Lu
78ac2bef3c go fmt 2019-03-30 23:09:16 -07:00
Chris Lu
97406333a5 support atomic renaming for mysql/postgres filer store 2019-03-30 23:08:29 -07:00
Chris Lu
920b4e56aa refactoring 2019-03-29 22:53:35 -07:00
Chris Lu
f60d383947 weed filer.copy: support multiple masters 2019-03-29 22:50:14 -07:00
Chris Lu
487ca51491 weed filer.export: fix pagination error 2019-03-28 00:46:15 -07:00
Chris Lu
07775998a9 some comments 2019-03-28 00:38:30 -07:00
Chris Lu
53b81fcfcd weed shell: add fs.tree 2019-03-28 00:05:04 -07:00
Chris Lu
c7e7b6229f detect mime type 2019-03-27 14:25:18 -07:00
Chris Lu
65757ae2fd weed filer: set content-type consistent with filer store 2019-03-27 11:41:11 -07:00
Chris Lu
37ce4c5269 weed shell: add fs.ls 2019-03-26 12:43:51 -07:00
Chris Lu
7a14cdc90c refactoring, go fmt 2019-03-25 23:18:40 -07:00
Chris Lu
19728fe3f6 remove deprecated code 2019-03-25 23:12:14 -07:00
Chris Lu
df95ce0b6c weed backup: efficient delta backup 
fix https://github.com/chrislusf/seaweedfs/issues/399
 2019-03-25 23:01:53 -07:00
Chris Lu
c913583e2d Merge branch 'master' into add_volume_follow 2019-03-25 21:19:34 -07:00
Chris Lu
91b9a68493 udpate glide 2019-03-25 09:39:54 -07:00
Chris Lu
a32797518b weed master: redirect to leader for default admin UI 
fix https://github.com/chrislusf/seaweedfs/issues/898
 2019-03-25 09:20:05 -07:00
Chris Lu
70815e9124 WIP 2019-03-25 09:16:12 -07:00
Chris Lu
eaa42c3865 1.27 2019-03-23 14:14:40 -07:00
Chris Lu
cd8a3b99bb textual changes 2019-03-23 12:57:35 -07:00
Chris Lu
bd1c0735e0 weed shell: adjust help text format 2019-03-23 11:54:26 -07:00
Chris Lu
6b70b36105 weed shell: add "volume.fix.replication" 2019-03-23 11:34:09 -07:00
Chris Lu
95e0520182 weed volume: add grpc operation to relicate a volume to local 2019-03-23 11:33:34 -07:00
Chris Lu
a3490b600c weed filer, weed master: add option to disable http 2019-03-21 16:00:46 -07:00
Chris Lu
531add52c2 weed master: skip proxied file read in http read only mode 2019-03-21 09:49:04 -07:00
Jonathan Amsterdam
e966033098 Merge branch 'master' of https://github.com/chrislusf/seaweedfs into gocdk 2019-03-20 16:02:38 -04:00
Chris Lu
977b30e992 weed filer: http HEAD response header add ETag 
fix https://github.com/chrislusf/seaweedfs/issues/892
 2019-03-20 10:13:53 -07:00
Jonathan Amsterdam
8db82e2b75 notification: add Go CDK pubsub support 
Add the gocdk_pub_sub package, which supports the Go Cloud Development
Kit pubsub API.

Link in all current providers.

Update the notification scaffold.
 2019-03-20 07:57:58 -04:00
chenwanli
45a52b17fd benchmark: add replication flag 2019-03-20 16:38:11 +08:00
Chris Lu
88ab932f7d refactoring function parameter 2019-03-19 23:01:23 -07:00
Chris Lu
5ae4b963a4 avoid using global rand 2019-03-19 22:20:14 -07:00
Chris Lu
f3d316a846 weed shell: adding file system disk usage fs.du 
fix https://github.com/chrislusf/seaweedfs/issues/889

echo "fs.du http://localhost:8888/some/path" | weed shell
 2019-03-19 21:58:00 -07:00
Chris Lu
cea5c53bf7 fix length checking 
fix https://github.com/chrislusf/seaweedfs/issues/890
 2019-03-19 20:56:27 -07:00
Chris Lu
da871896c3 weed filer: set grpc port to port + 10000 2019-03-19 05:47:41 -07:00
Chris Lu
916b809c08 add error checking 2019-03-19 05:34:43 -07:00
Chris Lu
eb4a54d9fe refactoring 2019-03-19 05:19:37 -07:00
Chris Lu
ece9d13312 volume info collect compact revision number 2019-03-18 09:32:21 -07:00
Chris Lu
104922a3db text wrapping 2019-03-18 00:35:15 -07:00
Chris Lu
44647a46c0 needle scanner read appendAtNs 2019-03-17 23:28:43 -07:00
Chris Lu
8b26d15740 refactoring: simplify function parameter 2019-03-17 22:32:01 -07:00
Chris Lu
2fcc88116e go fmt 2019-03-17 20:27:36 -07:00
Chris Lu
aca653c08b weed shell: list volumes 2019-03-17 20:27:08 -07:00
Chris Lu
22fbbf023b adjust collection list output 2019-03-17 17:31:05 -07:00
Chris Lu
e48267e287 adjust output 2019-03-17 17:28:29 -07:00
Chris Lu
657dd2e6c9 add shell command to list all collections 2019-03-16 13:43:16 -07:00
Chris Lu
b92122b885 fix test 2019-03-16 09:50:57 -07:00
Chris Lu
36b632ebcb fix tests 2019-03-16 09:50:21 -07:00
Chris Lu
42cb05c088 add context.Context 2019-03-15 17:23:02 -07:00
Chris Lu
55bab1b456 add context.Context 2019-03-15 17:20:24 -07:00
Chris Lu
cece860bfd add context to all filer APIs 2019-03-15 15:55:34 -07:00
Chris Lu
55c85f3a66 adding context 2019-03-15 15:26:09 -07:00
Chris Lu
d30600e6d6 fix filer starting under "weed server" when ip is not localhost 
fix https://github.com/chrislusf/seaweedfs/issues/886
 2019-03-15 09:57:33 -07:00
Chris Lu
a4e8aa98a0 weed server: mkdir first, and set the master ip address 
fix https://github.com/chrislusf/seaweedfs/issues/885
 2019-03-15 00:32:42 -07:00
Chris Lu
95ef4513c8 print out memory size 2019-03-13 23:14:40 -07:00
Chris Lu
346541a101 print out per entry memory usage 2019-03-13 23:07:24 -07:00
Chris Lu
99ce10daeb Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2019-03-11 00:58:17 -07:00
Chris Lu
2ce4dae795 move to 1.26 2019-03-11 00:58:12 -07:00
chenwanli
c668e55e07 Revert "fix https://github.com/chrislusf/seaweedfs/issues/780" 
This reverts commit 31038acccf.
 2019-03-11 11:48:11 +08:00
Chris Lu
6fe071175d mount: add option to disable allow others 
fix https://github.com/chrislusf/seaweedfs/issues/877
 2019-03-10 19:45:40 -07:00
Chris Lu
ad08a52ab6 synchronously upload files 
fix https://github.com/chrislusf/seaweedfs/issues/807
 2019-03-09 13:24:22 -08:00
Chris Lu
4773497d2c add back "/cluster/status" 
fix https://github.com/chrislusf/seaweedfs/issues/870
 2019-03-06 12:10:45 -08:00
Chris Lu
ab32556a51 weed server: fix error coming from moving raft to grpc 
fix https://github.com/chrislusf/seaweedfs/issues/872
 2019-03-06 10:21:49 -08:00
Chris Lu
0898e6c38e Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2019-03-04 13:00:10 -08:00
Chris Lu
d312c55bbe file path supports windows, avoiding back slashes 
fix https://github.com/chrislusf/seaweedfs/issues/868
 2019-03-04 13:00:08 -08:00
Chris Lu
f9dcf56465 adjust log level for loading configurations 2019-03-04 09:02:58 -08:00
chenwanli
f233bbe9c0 Fix https://github.com/chrislusf/seaweedfs/issues/825#issuecomment-469097375 2019-03-04 12:43:43 +08:00
Chris Lu
219b651bc3 jwt check the base file id 
fix https://github.com/chrislusf/seaweedfs/issues/867
 2019-03-03 10:17:44 -08:00
Chris Lu
2812c14520 master: add option to disable http operations 2019-03-02 05:10:05 -08:00
chenwanli
900aad4935 Fix https://github.com/chrislusf/seaweedfs/issues/864 2019-02-28 11:37:44 +08:00
Chris Lu
cb07d15254 add namespace for s3 2019-02-27 00:21:37 -08:00
Chris Lu
70ac2f6ea8 add namespace for ListAllMyBucketsResult 2019-02-26 23:23:04 -08:00
chenwanli
fd27ed7755 Fix https://github.com/chrislusf/seaweedfs/issues/861 2019-02-26 17:12:39 +08:00
Chris Lu
344caf3cd7 1.25 2019-02-25 00:44:48 -08:00
Chris Lu
7bbe24dd28 volume server directly support https 2019-02-25 00:43:36 -08:00
Chris Lu
c892b898a1 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2019-02-25 00:34:25 -08:00
Chris Lu
0d84e32f56 update library versions 2019-02-25 00:34:21 -08:00
chenwanli
0bc3cdd26a Fix https://github.com/chrislusf/seaweedfs/issues/825 2019-02-25 10:47:41 +08:00
Chris Lu
0a106c1757 updated glide 2019-02-20 09:09:47 -08:00
Chris Lu
e108688990 avoid grpc 5 seconds timeout 
some operations may take longer than 5 seconds.

only keep the timeout for raft operations
 2019-02-20 01:01:01 -08:00
Chris Lu
0fa1be8c4b adjust formatting and comments 2019-02-19 21:10:10 -08:00
Chris Lu
72c2e6bb8f remove unused file 2019-02-19 14:16:19 -08:00
Chris Lu
07af52cb6f raft change from http to grpc 
master grpc port is fixed to http port + 10000
 2019-02-18 22:38:14 -08:00
Chris Lu
448645203a remove unused variables 2019-02-18 20:05:55 -08:00
Chris Lu
d37c3ab7a5 adjust error message 2019-02-18 18:03:27 -08:00
Chris Lu
2442d56671 adjust package name 2019-02-18 18:03:16 -08:00
Chris Lu
9a4dda3011
Merge pull request #855 from chrislusf/add_jwt 
Add jwt
 2019-02-18 15:05:32 -08:00
Chris Lu
a1c7dc3806 avoid changing attributes for mount directory 2019-02-18 12:14:28 -08:00
Chris Lu
77b9af531d adding grpc mutual tls 2019-02-18 12:11:52 -08:00
Chris Lu
55761ae806 skip nil options 2019-02-16 12:49:58 -08:00
Chris Lu
98a03b38e5 avoid util package depends on security package 2019-02-16 12:23:35 -08:00
Chris Lu
31aa60ce05 Merge branch 'master' into add_jwt 2019-02-16 08:55:38 -08:00
Chris Lu
ad257ae179 simplify isLeader() logic 2019-02-16 08:55:23 -08:00
Chris Lu
712a0e19c8 Merge branch 'master' into add_jwt 2019-02-15 10:01:16 -08:00
Chris Lu
2ec6a679c2 avoid "fchmod failed" during cp 2019-02-15 10:00:27 -08:00
Chris Lu
157c0f7c01 add jwt for writes 2019-02-15 09:59:22 -08:00
Chris Lu
7103c1ab7e go fmt 2019-02-15 00:09:48 -08:00
Chris Lu
74fb237727 benchmark can work in secure mode 2019-02-15 00:09:19 -08:00
Chris Lu
a3b0e39b06 weed mount deletion always goes to weed filer 2019-02-14 07:07:38 -08:00
Chris Lu
215cd27b37 add authorizing fileId write access 
need to secure upload/update/delete for benchmark/filer/mount
need to add secure grpc
 2019-02-14 00:08:20 -08:00
Chris Lu
4ff4a147b2 cleanup security.Secret 2019-02-09 21:56:32 -08:00
Chris Lu
501bd72b1c wip: add security.toml file 2019-02-09 21:07:12 -08:00
Chris Lu
729d71fd4c Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2019-02-09 12:52:12 -08:00
Chris Lu
8afc632484 raft: use the first master to bootstrap the election 2019-02-09 12:52:09 -08:00
Chris Lu
744abc2690 log raft state changes 2019-02-09 12:51:14 -08:00
Sergey
aa5ccff6d2
fixing of typos 2019-02-06 18:59:15 +05:00
Chris Lu
1334507595 Revert "randomize based on self address" 
This reverts commit 6230eb28a6.
 2019-01-28 12:12:51 -08:00