mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-02 16:39:16 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,464 Commits 32 Branches 296 Tags 149 MiB
6f8b426f4f
Commit Graph

2775 Commits

Author SHA1 Message Date
Chris Lu
f8fea19669 1.99 2020-09-12 13:46:33 -07:00
Chris Lu
b0c7de186d filer: fix postgres prefixed directory listing problem 
fix https://github.com/chrislusf/seaweedfs/issues/1465
 2020-09-12 13:37:03 -07:00
Chris Lu
1a7afe7e6a adjust default value 
related to https://github.com/chrislusf/seaweedfs/issues/1453
 2020-09-12 12:46:42 -07:00
Chris Lu
f2723c1bc8 do not idx file format 
revert c9ab8d05fa
 2020-09-12 12:42:36 -07:00
Chris Lu
ba984a4e29 1.98 2020-09-12 04:13:02 -07:00
Chris Lu
446e476a11 go fmt 2020-09-12 04:08:03 -07:00
Chris Lu
ea26a98753 volume: validate volume correctness if last entry is a deletion 2020-09-12 04:07:04 -07:00
Chris Lu
d15682b4a1 shell: volume.balance plan by ratio of fullness 2020-09-12 04:06:26 -07:00
Chris Lu
c0ee78d2fa adjust make file 2020-09-12 04:05:42 -07:00
Chris Lu
cd9b89ba55 reduce default wait time to 10s before shutting down 2020-09-12 04:05:33 -07:00
Chris Lu
e2c741f76f adjust replica placement after move 2020-09-12 01:01:19 -07:00
Chris Lu
2a0925590c filer: etcd store avoid read with nil option 
fix https://github.com/chrislusf/seaweedfs/issues/1463
 2020-09-11 15:29:45 -07:00
Chris Lu
3984c3962f add comment 2020-09-11 15:07:19 -07:00
Chris Lu
baa6bdf4d4 s3: listMultipartUploads fix output format 2020-09-11 15:04:01 -07:00
Chris Lu
3eda8d6dfc s3: ListParts output xml format 
fix https://github.com/chrislusf/seaweedfs/issues/1461
 2020-09-11 14:53:50 -07:00
Chris Lu
ab201c2798 1.97 2020-09-11 13:47:50 -07:00
Chris Lu
9d4bdfcfdf fix volume integrity checking 2020-09-11 11:34:10 -07:00
Chris Lu
1b8094ef75 weed export: export deleted files 2020-09-11 02:05:14 -07:00
Chris Lu
12a8f5294d test for multi dc replication 2020-09-11 01:21:17 -07:00
Chris Lu
e60b2117c3 shell: volume balance follows replica placement 2020-09-11 00:29:25 -07:00
Chris Lu
89a62e8007 refactoring 2020-09-10 23:05:00 -07:00
Chris Lu
19537c9d21 1.96 2020-09-10 19:52:07 -07:00
Chris Lu
eaf9fdde99 avoid sharing context over separate goroutine 2020-09-10 19:51:03 -07:00
Chris Lu
cc5fe6f5ee handle special characters in html link 2020-09-10 19:46:02 -07:00
Chris Lu
401ccf1509 fix test 2020-09-10 15:13:23 -07:00
Chris Lu
c9ab8d05fa fixes for reading deleted fid 2020-09-10 14:42:52 -07:00
ruitao.liu
5b0676049a change elastic initialize process similar as others. 2020-09-10 23:35:20 +08:00
Chris Lu
660d7c0edd 1.95 2020-09-10 01:32:05 -07:00
Chris Lu
0d9b858cfb
Merge pull request #1455 from taozix/master 
add more basic elastic options.
 2020-09-10 01:31:17 -07:00
ruitao.liu
9be4e97625 change logs print format. 2020-09-10 16:30:15 +08:00
ruitao.liu
719dc43af1 modify elastic urls from string to array. 2020-09-10 16:24:09 +08:00
ruitao.liu
72f9d7f047 use util to generate md5. 2020-09-10 16:11:18 +08:00
ruitao.liu
6a5b38c0d4 fix elastic kv ops. 2020-09-10 15:59:16 +08:00
ruitao.liu
3f7fbfddca add more basic elastic options. 2020-09-10 14:22:07 +08:00
Chris Lu
b183ae54b5 master: changing 301 to 308 on redirect to volume servers 
fix https://github.com/chrislusf/seaweedfs/issues/1454
 2020-09-09 22:36:10 -07:00
Chris Lu
47b3f932e4 watch: adjust output format 2020-09-09 22:34:48 -07:00
Chris Lu
daf0a449f7 properly cancel context for streaming grpc 2020-09-09 12:07:15 -07:00
Chris Lu
7f69acd1f2 sync pprof 2020-09-09 11:33:52 -07:00
Chris Lu
387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu
4fc0bd1a81 return http response directly 2020-09-09 03:53:09 -07:00
Chris Lu
5e13bc878c adjust log level 2020-09-09 03:49:26 -07:00
Chris Lu
d1b816212f return nil if no need to do anything 2020-09-07 16:03:05 -07:00
Chris Lu
64a621bcc8 shell: volume.fix.replication also purge over replicated volumes 2020-09-07 16:00:10 -07:00
Chris Lu
c18ea21f7a 1.94 2020-09-07 13:13:53 -07:00
Chris Lu
18b98fdb72 fix test 2020-09-07 12:57:38 -07:00
Chris Lu
44b3f2efc8 filer: Elastic Search return correct kv error 
fix https://github.com/chrislusf/seaweedfs/issues/1452
 2020-09-07 12:44:51 -07:00
Chris Lu
d80538a187 refactoring 2020-09-07 12:35:02 -07:00
Chris Lu
1a09bc43d1 refactor 2020-09-07 11:31:33 -07:00
Chris Lu
2b643f477d typo 2020-09-06 12:47:55 -07:00
Chris Lu
05034aade5 printout over replicated locations 2020-09-06 12:44:02 -07:00
Chris Lu
432fe3d926 1.93 2020-09-06 12:12:09 -07:00
Chris Lu
d8af6f62df adjust logging 2020-09-06 12:10:40 -07:00
Chris Lu
07f32feed0 print sync progress 2020-09-06 00:50:51 -07:00
Chris Lu
0af6252b2d fix compilation 2020-09-06 00:50:38 -07:00
Chris Lu
ac494ff5e8 Update meta_aggregator.go 2020-09-06 00:29:16 -07:00
Chris Lu
c9f8f25ba5 read peer filer from start 2020-09-06 00:12:41 -07:00
Chris Lu
30dc365cbd add self filer 2020-09-06 00:11:46 -07:00
Chris Lu
8acd714600 apply meta changes only if store is different 2020-09-05 23:25:03 -07:00
Chris Lu
d741ed66db check whether shares the same filer store 2020-09-05 23:02:52 -07:00
Chris Lu
bba90ff3c8 read filer signature 2020-09-05 22:52:15 -07:00
Chris Lu
71b0e256c7 filer: setOrLoadFilerStoreSignature 2020-09-05 14:08:59 -07:00
ruitao.liu
450cf07505 skip the index that for kv usage. 2020-09-04 21:49:03 +08:00
ruitao.liu
1384ff9a2f 1.split kv in one file. 
2.disable query for kv in es index.
 2020-09-04 17:34:26 +08:00
ruitao.liu
83080b5e03 ES backended filer support kv ops. 2020-09-04 15:40:13 +08:00
Chris Lu
46f65a84a6 filer: elastic7 adjust default value 2020-09-03 20:12:38 -07:00
Chris Lu
f815fce81b
Merge pull request #1446 from taozix/master 
new filer store option to es7.
 2020-09-03 20:10:23 -07:00
Chris Lu
44b4ebf0ff filer: Redis cleanly delete directory 
fix https://github.com/chrislusf/seaweedfs/issues/1448
 2020-09-03 19:17:39 -07:00
Chris Lu
b8f32bcab9 filer: compress stored metadata 2020-09-03 11:00:20 -07:00
Chris Lu
f76a2b2c8a printout meta data size 2020-09-03 09:51:21 -07:00
ruitao.liu
798280e980 change filer2 to filer. 2020-09-03 17:05:26 +08:00
ruitao.liu
a93d27d1e8 new filer option to es v7. 2020-09-03 16:34:58 +08:00
Chris Lu
1d56ea24ef fix 2020-09-03 00:08:37 -07:00
Chris Lu
7e1aad0b54 mount: map uid/gid between local and filer 2020-09-03 00:07:22 -07:00
Chris Lu
68e878adb5 fix formatting 2020-09-02 21:42:12 -07:00
Chris Lu
0d99a5da3c c*: support kv operations 2020-09-02 18:39:24 -07:00
Chris Lu
645a4af3db mongodb: support kv operations 2020-09-02 17:19:14 -07:00
Chris Lu
ecaa30c408 better error message 2020-09-02 17:17:44 -07:00
Chris Lu
7c770b727c Update abstract_sql_store_kv.go 2020-09-02 10:13:48 -07:00
Chris Lu
05a6e2dc7f sql kv upsert 2020-09-02 10:12:44 -07:00
Chris Lu
ed62f52452 fix sql insert with duplicated primary key 2020-09-02 10:09:49 -07:00
Chris Lu
9ea290aa12 fix error message 2020-09-01 23:21:41 -07:00
Chris Lu
06a1188267 mysql/postgres: support kv operations 2020-09-01 22:48:36 -07:00
Chris Lu
aa40295f06 leveldb: report not found entry 2020-09-01 22:48:23 -07:00
Chris Lu
87d7312bf6 mysql/postgres: properly report entry not found 2020-09-01 22:47:57 -07:00
Chris Lu
72b0a5f1d1 mysql or postgres: log find error 2020-09-01 22:25:17 -07:00
Chris Lu
2815bbe6c0 go fmt 2020-09-01 21:59:26 -07:00
Chris Lu
37234bf3f8 filer store adds kv support 
can compile now, need to implement those unimplemented
 2020-09-01 21:59:00 -07:00
James Hartig
8e54e34576 volume: Don't unmount before deleting volume in copy 
If we unmount first and then delete, the delete fails because the volume
was unmounted. Delete ends up doing the same thing as the unmount anyways.
 2020-09-01 22:00:07 -04:00
Chris Lu
2b14ae5819 fix tests 2020-09-01 01:33:43 -07:00
Chris Lu
d91ec535b3 fix tests 2020-09-01 01:29:13 -07:00
Chris Lu
eb7929a971 rename filer2 to filer 2020-09-01 00:21:19 -07:00
Chris Lu
38e06d783d volume: check disk space before compaction 
fix https://github.com/chrislusf/seaweedfs/issues/1440
 2020-08-31 18:10:53 -07:00
Chris Lu
dbb10e0f0b testing with more than 30days 2020-08-31 17:59:29 -07:00
Chris Lu
dd176cdb8b avoid tiny step pagination 2020-08-31 11:28:03 -07:00
Chris Lu
97733c3dff typo 2020-08-31 10:49:17 -07:00
Chris Lu
c8398bdb09 adjust metrics 2020-08-31 10:41:05 -07:00
Chris Lu
8a1c8e41b3 simplify if else logic 2020-08-31 10:39:24 -07:00
Chris Lu
f0c89cfacd go fmt 2020-08-31 10:23:31 -07:00
Chris Lu
edb9d65e05
Merge pull request #1431 from kmlebedev/wip-prefix-search 
Wip prefix search
 2020-08-31 10:22:14 -07:00
Konstantin Lebedev
9a195bebfd accurate limit 2020-08-31 22:13:13 +05:00
Konstantin Lebedev
18b98f9747 not convert err to string 2020-08-31 21:55:18 +05:00
Konstantin Lebedev
22f32e75c4 rename 2020-08-31 21:52:05 +05:00
Konstantin Lebedev
a8de332beb fix 2020-08-31 21:45:44 +05:00
Konstantin Lebedev
7f013d77d6 add var UnsupportedListDirectoryPrefixedErr 2020-08-31 21:43:32 +05:00
Konstantin Lebedev
60a86cfe04 add const 2020-08-31 21:35:16 +05:00
Konstantin Lebedev
82ea121d09 rm func 2020-08-31 18:50:06 +05:00
Konstantin Lebedev
2ea638f865 filer store wrapper can implement the logic to filter by prefi 2020-08-31 17:13:56 +05:00
Chris Lu
408e339c53 also delete the manifest chunk itself 2020-08-31 00:16:03 -07:00
Chris Lu
dd52037c27 1.92 2020-08-30 22:56:35 -07:00
Chris Lu
33d8c6c617 change log to warning 2020-08-30 21:01:44 -07:00
Chris Lu
e62d7f221f a bit more logs 2020-08-30 21:00:24 -07:00
Chris Lu
22fe4ae573 mount: fix for UrBackup 2020-08-30 21:00:09 -07:00
Chris Lu
ae84a9ee2e add logs 2020-08-30 20:59:04 -07:00
Chris Lu
a41588279a change log level 5 to 4 2020-08-30 20:12:04 -07:00
Chris Lu
56244fb9a1 fix hard coded host address 2020-08-30 10:23:35 -07:00
Chris Lu
f2a8574448 filer and mount deletion resolves manifest chunks also 2020-08-30 02:07:14 -07:00
Chris Lu
99ecf63276 go fmt 2020-08-29 22:28:33 -07:00
Chris Lu
ef75ce8a34 use UTC for all time related folders 2020-08-29 21:01:14 -07:00
Chris Lu
b69cb74c03 read meta logs by timestamp 
pass in event ts when moving logs
meta aggregator reads in memory logs only
 2020-08-29 17:37:19 -07:00
Chris Lu
063c9ddac5 adjust logs 2020-08-29 11:56:22 -07:00
Chris Lu
ca658a97c5 add signatures to messages to avoid double processing 2020-08-28 23:48:48 -07:00
Chris Lu
63ad1abcce watch: follow changes for a pattern 2020-08-28 19:43:04 -07:00
Chris Lu
b7a654a318 file mime avoid saving application/octet-stream 2020-08-28 19:42:40 -07:00
Chris Lu
707192f966 fix help message 2020-08-27 00:13:38 -07:00
Chris Lu
0be6863c87 rename 2020-08-26 22:40:15 -07:00
Chris Lu
50f7e2e89f add log messages 2020-08-26 09:51:20 -07:00
Chris Lu
6bc618051c log message 2020-08-26 09:32:56 -07:00
Chris Lu
9dae4e9d3e fix logs 2020-08-26 09:31:11 -07:00
Chris Lu
4c1a3187bf fix log error 2020-08-26 09:25:54 -07:00
Chris Lu
ab759f0ec2 erasure coding: fix EC error if multiple disks are configured in one volume server 2020-08-26 09:16:58 -07:00
Chris Lu
d1cf39f180 fix logging 2020-08-26 08:55:15 -07:00
Chris Lu
83012f543a s: avoid possible completeMultipartUpload re-tries that can overwrite the uploaded file 2020-08-26 08:50:24 -07:00
Konstantin Lebedev
464d4c82ec stop send heartbeat before stop volume server 2020-08-26 16:16:11 +05:00
Chris Lu
1901f15cd2 volume server: remove whitelist for status checking 
Volume liveness/readiness not work if jwt enabled, I try use param
-whiteList
but jwt breaks
 2020-08-25 02:27:47 -07:00
Chris Lu
81e5124faf s3: list bucket permission change from admin to read 
fix https://github.com/chrislusf/seaweedfs/issues/1430
 2020-08-24 11:22:45 -07:00
Chris Lu
5f55a87101 close http response 2020-08-24 00:32:44 -07:00
Chris Lu
0b301b0b47 consume all response body in order to release requests 2020-08-23 21:32:29 -07:00
Chris Lu
2cdad77d82 fix checkin error 2020-08-23 18:30:11 -07:00
Chris Lu
81a8aa6581 1.91 2020-08-23 17:28:40 -07:00
Chris Lu
51346a5930 always keep the manifest list of chunks 2020-08-23 17:15:12 -07:00
Chris Lu
c1d1677a28 keep manifest chunks forever 2020-08-23 16:59:01 -07:00
Chris Lu
98175548c2 adjust deletion ordering 2020-08-23 16:27:12 -07:00
Chris Lu
f0e50e7b26 fix test 2020-08-23 16:12:34 -07:00
Chris Lu
aee27ccbe1 multiple fixes 
* adjust isOpen count
* move ContinuousDirtyPages lock to filehandle
* fix problem with MergeIntoVisibles, avoid reusing slices
* let filer delete the garbage
 2020-08-23 15:48:02 -07:00
Chris Lu
c19245886c simpler test 2020-08-23 14:20:27 -07:00
Chris Lu
df816a58fe add tests 2020-08-23 14:09:25 -07:00
Chris Lu
77393d3d30 add file handle locking when changing file entry 2020-08-23 00:35:50 -07:00
Chris Lu
d60bcbf08a sorting chunks 2020-08-23 00:00:36 -07:00
Chris Lu
5e6b714836 add random test 2020-08-22 16:33:00 -07:00
Chris Lu
42ddbcc3a2 SQL insert falls back to update 
fix https://github.com/chrislusf/seaweedfs/issues/1429
 2020-08-20 07:52:46 -07:00
Chris Lu
f48567c5c6 remove unused function 2020-08-19 22:53:49 -07:00
Chris Lu
c45ba5d7d4 fix listObjectsV2 response format 
fix https://github.com/chrislusf/seaweedfs/issues/1426 issue 1
 2020-08-19 12:08:04 -07:00
James Hartig
3ccfa4c6ad Added VolumeMarkWritable and VolumeStatus grpc methods 
This is necessary for copy to mark as read-only and then restore the
original state afterwards.
 2020-08-19 11:42:56 -04:00
Chris Lu
3b4b1d4a77 fix tests 2020-08-19 01:37:56 -07:00
Chris Lu
839634097f also do flush on release 2020-08-19 01:27:10 -07:00
Chris Lu
c27e18aa6a read possible old deleted chunks 2020-08-19 00:43:07 -07:00
Chris Lu
a78772d5ea avoid shutdown in the middle of running 2020-08-19 00:42:02 -07:00
Chris Lu
ae9bc4a508 logs 2020-08-18 23:42:26 -07:00
Chris Lu
ed4b43b419 adjust logs 2020-08-18 23:42:09 -07:00
Chris Lu
9f1e0aeef5 delete chunks in the last step 2020-08-18 23:40:53 -07:00
Chris Lu
99d05f758c adjust logs 2020-08-18 23:39:18 -07:00
Chris Lu
fe01191b5b support read option readDeleted=true 2020-08-18 19:22:16 -07:00
Chris Lu
6ccd7f0a4d refactoring 2020-08-18 18:01:37 -07:00
Chris Lu
c026eb0592 refactoring 2020-08-18 17:39:29 -07:00
Chris Lu
7e91ae592c pass in option to read deleted entries 
not working yet
 2020-08-18 17:37:26 -07:00
Chris Lu
51ecb49db3 for debugging 2020-08-18 17:35:39 -07:00
Chris Lu
ee11d98650 refactoring 2020-08-18 17:35:19 -07:00
Chris Lu
6a92f0bc7a refactoring to typed Size 
Go is amazing with refactoring!
 2020-08-18 17:04:28 -07:00
Chris Lu
618b2f6829 release resources only when needed to 2020-08-18 12:53:08 -07:00
Chris Lu
208849702d logs 2020-08-18 12:52:54 -07:00
Chris Lu
1fcd083db3 printout data size 2020-08-18 09:09:29 -07:00
Chris Lu
3e5339337a minor 2020-08-18 08:50:14 -07:00
Chris Lu
85001cbec7 properly report io.EOF 2020-08-18 08:18:54 -07:00
Chris Lu
0ca45a5cbc adjust logs 2020-08-18 00:34:15 -07:00
Chris Lu
cd43738245 fix reading when filling zeros 2020-08-18 00:32:01 -07:00
Chris Lu
ecb3ce46be adjust error logs 2020-08-18 00:31:42 -07:00
Chris Lu
30fe424469 add one more test case 2020-08-17 22:47:27 -07:00
Chris Lu
1b68ba953b fix for out of range reads 2020-08-17 22:46:32 -07:00
Chris Lu
56fbd2c211 fix reading 2020-08-17 21:17:32 -07:00
Chris Lu
09e126bae5 refactoring: use interface 2020-08-17 20:20:08 -07:00
Chris Lu
be4d42b8e2 rename 2020-08-17 20:15:53 -07:00
Chris Lu
0625e63648 count 0 as part of the reads 2020-08-17 20:14:40 -07:00
Chris Lu
0be4b6e7f0 logs 2020-08-17 16:05:40 -07:00
Chris Lu
97e54a80d4 rename variables 2020-08-17 16:05:13 -07:00
Chris Lu
abdaf9958d possibly read more 2020-08-17 16:04:56 -07:00
Chris Lu
9d46c7bc78 rename 2020-08-17 11:12:10 -07:00
Chris Lu
4ccfdaeb4d prevent nil 2020-08-17 10:07:34 -07:00
Chris Lu
24c8e6bcb4 minor optimization 2020-08-17 10:03:34 -07:00
Chris Lu
f5837b7000 report error first 2020-08-16 23:49:10 -07:00
Chris Lu
2ac27616bc fix possible out of range bytes 
avoid buff out of range resp.Data = buff[:totalRead]
 2020-08-16 23:47:34 -07:00
Chris Lu
ff200398bb 1.90 2020-08-16 21:18:50 -07:00
Chris Lu
4ceeba9e70 streaming reads 2020-08-16 21:07:46 -07:00
Chris Lu
e72953dff7 logs 2020-08-16 21:06:03 -07:00
Chris Lu
627b081b67 adjust logs 2020-08-16 16:32:22 -07:00
Chris Lu
22e5132b3a adjust log level 2020-08-16 16:25:11 -07:00
Chris Lu
ee0f92a6be reduce memory allocation 2020-08-16 16:24:40 -07:00
Chris Lu
2ba817afac read randomly written data 2020-08-16 15:16:46 -07:00
Chris Lu
6111b265e7 fix compilation 2020-08-16 01:38:16 -07:00
Chris Lu
4a77f0820a clean up logs 2020-08-16 01:37:50 -07:00
Chris Lu
8c9e6eaacd fix tests 2020-08-16 01:35:52 -07:00
Chris Lu
b71df82292 1.89 2020-08-16 00:57:28 -07:00
Chris Lu
1d9ea30b72 fix ViewFromVisibleIntervals 2020-08-16 00:49:26 -07:00
Chris Lu
aec7f32b02 fix reader_at 2020-08-16 00:49:08 -07:00
Chris Lu
5d80fc2ec7 adjust logs 2020-08-15 21:09:31 -07:00
Chris Lu
e0bfd3161a update metadata only if changed 2020-08-15 19:56:24 -07:00
Chris Lu
003d48da21 adjust logs 2020-08-15 19:55:28 -07:00
Chris Lu
6ee8d952d2 adjust log level 2020-08-15 18:24:35 -07:00
Chris Lu
3e1395b767 adjust log message 2020-08-15 17:06:16 -07:00
Chris Lu
a22ee30596 fix nil 2020-08-15 17:01:42 -07:00
Chris Lu
0d60e67816 ensure meta data changes are updated 2020-08-15 14:15:07 -07:00
Chris Lu
5b43bddf20 proper deletion ordering 
delete central file store first, then delete local cache
 2020-08-15 09:33:41 -07:00
Chris Lu
c647deace1 file size support set file length 
use Attr.FileSize and TotalChunkSize to determine file size
 2020-08-15 09:32:47 -07:00
Chris Lu
c03bb180eb fix error reporting 2020-08-14 00:44:02 -07:00
Chris Lu
eb493283dd deletion error report 2020-08-14 00:23:01 -07:00
Chris Lu
a7f669044e rename also applies to open file handle 2020-08-14 00:22:49 -07:00
Chris Lu
edfa73782f adjust log level 2020-08-14 00:22:21 -07:00
Chris Lu
0983060a90 increase default volume file size limit to 1024 
avoid possible large chunk size set on mount or filer
 2020-08-13 09:07:22 -07:00
Chris Lu
090612492c fix compilation 2020-08-13 00:07:56 -07:00
Chris Lu
d43129d27f add back handles lock 2020-08-12 23:52:13 -07:00
Chris Lu
f735d579d3 adjust comment 2020-08-12 13:22:19 -07:00
Chris Lu
8824a9755c remove directory cache 2020-08-12 13:11:04 -07:00
Chris Lu
83cad3da79 add retry file upload 2020-08-11 20:30:11 -07:00
Chris Lu
152a6cbc2b minor adjustments 2020-08-10 20:42:27 -07:00
Chris Lu
f1b40567e3
Merge pull request #1419 from robin1900/master 
fix bug:  two same volumeId in different collections
 2020-08-10 20:38:55 -07:00
cheng.li01
25fbff5d52 fix bug: two same volumeId in different collections 
1, there will be two leader when master server startup in a few seconds
2, raft server will get a leader even there is only one master, so there is no need to do hard code to set the server to be leader
 2020-08-10 16:37:47 +08:00
Chris Lu
e74dc4e4bc add back fs node cache for renaming 2020-08-09 21:56:09 -07:00
Chris Lu
f86c7d911a remove println 2020-08-09 14:42:25 -07:00
Chris Lu
9ecc1170a3 istObjects和listObjectsV2不能查询子目录 #1418 
fix https://github.com/chrislusf/seaweedfs/issues/1418
 2020-08-09 14:35:53 -07:00
Chris Lu
3f4aff5dde s3: fix delimiter in list response 2020-08-09 09:09:35 -07:00
Chris Lu
b056707770 1.88 2020-08-08 22:55:12 -07:00
Chris Lu
9832653e1d FUSE mount: proper error with deleting non empty folder 2020-08-08 21:37:36 -07:00
Chris Lu
3b1a95ac26 filer refactoring: same auto chunking logic for POST and PUT, no size limit 2020-08-08 12:02:06 -07:00
Chris Lu
67348e7b15 less noisy heartbeat logs 2020-08-08 10:53:35 -07:00
Chris Lu
bee0d7e5eb lower log priority for noisy heartbeat 2020-08-08 10:52:13 -07:00
Chris Lu
ae00cce4bd support POST and PUT auto chunking 2020-08-08 10:45:37 -07:00
Chris Lu
bd8bfdae07 refactoring 2020-08-08 10:18:43 -07:00
Chris Lu
ab6e5c0dc4 adjust error message 2020-08-08 10:18:32 -07:00
Chris Lu
828a5ae429 check signature only when auth is enabled 2020-08-08 09:11:40 -07:00
Chris Lu
20e2ac1add filer: store md5 metadata for files uploaded by filer 
fix https://github.com/chrislusf/seaweedfs/issues/1412
 2020-08-06 10:04:17 -07:00
Chris Lu
93ea0801ea volume: the variable for the master node may be stale? 
related to https://github.com/chrislusf/seaweedfs/issues/1414
 2020-08-06 09:48:54 -07:00
Chris Lu
4ecfa9879d volume: report Content-MD5 in response header 2020-08-06 05:22:53 -07:00
Chris Lu
41007ced77 remove logging 2020-08-06 04:26:29 -07:00
Chris Lu
2b74abf766 S3: configurable access for anonymous user 
fix https://github.com/chrislusf/seaweedfs/issues/1413
 2020-08-06 03:41:34 -07:00
Устюжанин Антон Александрович
9ca011e3ca ListDirectoryPrefixedEntries 2020-08-06 00:38:59 +05:00
Устюжанин Антон Александрович
b231f7bdab ListDirectoryPrefixedEntries 2020-08-06 00:37:42 +05:00
Устюжанин Антон Александрович
a457c308ad test ListDirectoryPrefixedEntries 2020-08-06 00:24:31 +05:00
Устюжанин Антон Александрович
2d3b355fb6 test ListDirectoryPrefixedEntries 2020-08-05 23:56:06 +05:00
Устюжанин Антон Александрович
67f134ebd7 test ListDirectoryPrefixedEntries 2020-08-05 23:44:48 +05:00
Устюжанин Антон Александрович
dc9fc01254 test ListDirectoryPrefixedEntries 2020-08-05 23:38:00 +05:00
Устюжанин Антон Александрович
33a9e5e2d1 test ListDirectoryPrefixedEntries 2020-08-05 22:19:16 +05:00
Chris Lu
cbd80253e3 better needle id format 2020-08-04 22:09:07 -07:00
Chris Lu
1b3a80dd3d non-fatal error 2020-08-01 13:46:52 -07:00
Chris Lu
bbbea8159c http request use gzip if possible 2020-08-01 13:20:52 -07:00
Chris Lu
2f03481cb2 in case when content is not compressed 2020-08-01 13:08:10 -07:00
Chris Lu
49199c69d4 volume: avoid special logic for .gz files 2020-08-01 11:19:57 -07:00
Chris Lu
f89186a3cf better logging 2020-08-01 11:19:57 -07:00
Chris Lu
49d6dcfbc6 dynamically change the log level for reading the disk status 2020-08-01 11:19:57 -07:00
limd
b41b7ea4d0 fix spark read s3 bug (sc.binaryFiles) 2020-08-01 01:08:30 +08:00
Chris Lu
a6b59d50f7 1.87 2020-07-30 13:52:04 -07:00
Chris Lu
eed525b717 FUSE mount: remove DirListCacheLimit 
outdated parameter
 2020-07-28 09:24:39 -07:00
Chris Lu
a566bfc6e1 s3: use bucket in the domain 
fix https://github.com/chrislusf/seaweedfs/issues/1405
 2020-07-28 08:47:27 -07:00
limd
92c32f9d46 PutObject: fix create folder bug 2020-07-28 00:58:42 +08:00
Chris Lu
54cd582653 refactor 2020-07-26 12:58:58 -07:00
Chris Lu
c50ec1599c auth for presigned put object part 2020-07-26 12:09:50 -07:00
Chris Lu
ea0180de4e auth for presigned put operation 2020-07-26 12:06:45 -07:00
Chris Lu
3faa0aa448 rename 2020-07-25 20:09:19 -07:00
Chris Lu
023a1efdf2 set filename in Content-Disposition header 2020-07-25 20:06:40 -07:00
popstk
a09fcfcf20 fix s3api auth bug 2020-07-25 17:14:49 +08:00
popstk
feb52400be fix s3api copy object handler SerializationError 2020-07-25 15:52:31 +08:00
Chris Lu
d40de39e75 FUSE: do not change crtime, uid, gid on save 2020-07-24 10:13:45 -07:00
Chris Lu
8dfeba8023 FUSE: skip changing to empty uid and gid during flush 
Skip uid and gid changes if request uid and gid are zero

mongodb lock file, and many interim files, has empty uid and gid

I0724 10:05:51 93643 filehandle.go:173] /db/diagnostic.data/metrics.interim fh 1333342842031408359 flush Flush [ID=0x3 Node=0x50 Uid=0 Gid=0 Pid=178] 0x10 fl=0x0 lk=0x0
 2020-07-24 10:06:43 -07:00
Chris Lu
dd29b8c81d fix compilation 2020-07-24 09:36:05 -07:00
Chris Lu
5788f1f6bf remove fsnode cache 
fix https://github.com/chrislusf/seaweedfs/issues/1400
 2020-07-24 09:25:54 -07:00
Chris Lu
6962ca48d3 fix compilation 2020-07-23 21:31:23 -07:00
Chris Lu
df7eee523b FUSE: implement dir fsync 
fix https://github.com/chrislusf/seaweedfs/issues/1398
 2020-07-23 21:09:40 -07:00
Chris Lu
3bbac0820b FUSE: faster rename implementation 2020-07-23 21:08:42 -07:00
Chris Lu
943fa4ea0c fix compilation error 2020-07-22 16:11:17 -07:00
Chris Lu
49446af516 simplify 2020-07-22 13:43:54 -07:00
James Hartig
229f11c660 Added VolumeNeedleStatus volume server grpc method 
This is needed for the diffing tool to get the cookie for a needle
 2020-07-22 15:02:21 -04:00
James Hartig
0be3bb4a87 idx: Switch WalkIndexFile to accept generic io.ReaderAt 
This is necessary to use it with in-memory idx files.
 2020-07-22 15:01:16 -04:00
Chris Lu
7a7e70b6cc 1.86 2020-07-20 23:39:58 -07:00
Chris Lu
885c624bce volume.fsck: follow manifest chunks 2020-07-20 22:02:05 -07:00
Chris Lu
44057a4de1 clean up chunks in manifest 2020-07-20 22:01:39 -07:00
Chris Lu
0b2e06268b use merge factor 1000 2020-07-20 18:30:09 -07:00
Chris Lu
d02c0fe0c0 refactoring 2020-07-20 03:34:06 -07:00
Chris Lu
60d14a9800 mount: fix difference with storage format in local cache 2020-07-20 00:17:50 -07:00
Chris Lu
97d97f3528 go code can read and write chunk manifest 2020-07-19 17:59:43 -07:00
李远军
4162766c46 Fix the bug of "http://xx/xx,xxx.PNG?width=100" 
If you want to do a resize a UPPER ext pic file, there is a bug caused by commit 308688c8d0 (also see https://github.com/chrislusf/seaweedfs/blob/master/weed/images/resizing.go#L43)
 2020-07-17 19:57:35 +08:00
Chris Lu
f43146b237 resolve directories if containing home directory 2020-07-16 22:50:14 -07:00
Chris Lu
befb396892 mount: resolve home directory 
fix https://github.com/chrislusf/seaweedfs/issues/1391
 2020-07-16 22:38:55 -07:00
Chris Lu
2b2ca7d1dc stop if the directory can not be found 
related to https://github.com/chrislusf/seaweedfs/issues/1391
 2020-07-16 13:24:53 -07:00
Chris Lu
2286d27730 rename 2020-07-15 14:19:15 -07:00
Chris Lu
0db4204c81 add is_file_chunks to filer.proto 2020-07-14 22:42:52 -07:00
Chris Lu
bbc8ae110f report errors 2020-07-14 11:27:00 -07:00
Chris Lu
17f2d05cde filer: ensure logs are persisted 2020-07-14 11:25:50 -07:00
Chris Lu
ddec7b2bb9 go fmt 2020-07-14 07:34:16 -07:00
Chris Lu
4f6096c7f0 add reading from persisted logs for local filer store 2020-07-13 22:55:28 -07:00
Chris Lu
87b5031714 change variable visibility 2020-07-13 08:19:48 -07:00
Chris Lu
7597831cac filer: leveldb2 supports peers also 2020-07-13 00:05:40 -07:00
Chris Lu
5ba894bb72 leveldb2: support subscribing from peers 2020-07-13 00:05:40 -07:00