mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-02 00:19:15 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,132 Commits 32 Branches 296 Tags 148 MiB
2bf5ea87d6
Commit Graph

3245 Commits

Author SHA1 Message Date
Chris Lu
94e3757c08 fix 2020-12-23 21:45:16 -08:00
Chris Lu
64c48c9724 seems compiling 2020-12-23 21:44:53 -08:00
Chris Lu
15da5834e1 Merge branch 'master' into support_ssd_volume 2020-12-23 14:37:37 -08:00
Chris Lu
3be3635799 just log error when checking empty dir failed 2020-12-23 14:36:48 -08:00
Chris Lu
0ca9d89589 s3: break loop if error 
fix #1701
 2020-12-23 14:34:59 -08:00
Chris Lu
132f275d04 s3: log errors instead of stopping when checking isDirectoryAllEmpty has error 2020-12-23 12:21:21 -08:00
Chris Lu
db62090f88 s3: break loop if error 
fix https://github.com/chrislusf/seaweedfs/issues/1701
 2020-12-23 12:14:51 -08:00
Baptiste Mille-Mathias
385a4dec30 [shell] Clarify bucket replication setting order 2020-12-23 09:35:24 +01:00
Chris Lu
1c7e1295dc Merge branch 'master' into support_ssd_volume 2020-12-22 17:44:52 -08:00
Chris Lu
da134a2eb7 minor 2020-12-22 17:43:13 -08:00
Chris Lu
beb3b8ddb0 shell: change bucket.list to s3.bucket.list, same for create and delete 2020-12-22 17:40:55 -08:00
Chris Lu
4cf3176cbb filer first, master second 2020-12-22 17:34:44 -08:00
Chris Lu
a1009e8044 filer: add -defaultStoreDir so that filer.toml can be skipped 
fix https://github.com/chrislusf/seaweedfs/issues/1659
 2020-12-22 17:33:40 -08:00
Chris Lu
a0990b929d minor 2020-12-22 17:33:40 -08:00
Baptiste Mille-Mathias
0ed5345954 [shell] Add more information to bucket.create 
State the replication setting that will be honor if not defined.
 2020-12-22 17:18:59 +01:00
Chris Lu
97e3432dfe avoid wrong error 
fix https://github.com/chrislusf/seaweedfs/issues/1691
 2020-12-22 02:46:24 -08:00
Chris Lu
4f31c1bb94 go fmt 2020-12-22 02:34:08 -08:00
Chris Lu
6c4f32d173 remove unused code 2020-12-22 02:33:31 -08:00
Chris Lu
90d785a15f filer: redis, redis cluster, cassandra support super large directory 2020-12-22 02:26:05 -08:00
Chris Lu
fe46411cd4 handle directory listing edge cases 2020-12-21 23:37:43 -08:00
Chris Lu
c74bede730 handle listing directories 2020-12-21 23:19:05 -08:00
Chris Lu
adf8cb4000 filer: path-specific stores trim out common prefixes 2020-12-21 22:57:13 -08:00
Chris Lu
0823bde8d8 refactor: split into two files 2020-12-21 22:05:15 -08:00
Chris Lu
8ab70ccf76 refactor 2020-12-21 21:46:32 -08:00
Chris Lu
65cc6dc636 refactor to dedicated function 2020-12-21 21:18:34 -08:00
Chris Lu
488c2680e8 change default filer store root 2020-12-21 21:15:22 -08:00
Chris Lu
f5a19a87be filer: report error if the path specific store is not found 
related https://github.com/chrislusf/seaweedfs/issues/1691
 2020-12-21 12:43:52 -08:00
Chris Lu
6912bf94ae 2.16 2020-12-20 15:30:12 -08:00
Chris Lu
41c0f3ad24 filer: support path-specific filer store 2020-12-19 01:27:09 -08:00
Chris Lu
2260864b45 a cleaner fix 2020-12-18 10:39:30 -08:00
Chris Lu
b7fef8f693 fix when a volume server has only hdd volumes 2020-12-18 10:34:16 -08:00
Chris Lu
0d5683fb0e todo: load path-specific store from filer.toml 2020-12-18 03:05:42 -08:00
Chris Lu
3269fd7eaf filer: use store by path 2020-12-18 02:57:49 -08:00
Chris Lu
e605f1e001 refactor, add getActualStore() by path 2020-12-18 02:55:00 -08:00
Chris Lu
23903aa95b refactoring 2020-12-18 02:35:45 -08:00
Chris Lu
4e6b316913 assert FilerStoreWrapper is VirtualFilerStore 2020-12-18 02:17:06 -08:00
Chris Lu
9abf016af1 Merge branch 'master' into support_ssd_volume 2020-12-17 22:30:19 -08:00
Chris Lu
738c4fd203 fix logging when error 
address https://github.com/chrislusf/seaweedfs/pull/1685#issuecomment-747845601
 2020-12-17 19:37:46 -08:00
Chris Lu
d9e8479c06 adjust UI max count 2020-12-17 13:47:51 -08:00
Chris Lu
3cdf5945a2 adjust UI 2020-12-17 13:37:00 -08:00
Chris Lu
f696a2b2a7 assign volumes based on disk type 2020-12-17 13:25:05 -08:00
Chris Lu
daa8157fc2 Merge branch 'master' into support_ssd_volume 2020-12-17 13:05:20 -08:00
Chris Lu
e2076201d7 volume: avoid reprocessing the same volume 
fix https://github.com/chrislusf/seaweedfs/issues/1682
 2020-12-17 13:03:39 -08:00
Chris Lu
986cbdf7d9 Revert "Merge pull request #1683 from qieqieplus/master" 
This reverts commit 8cb67952db, reversing
changes made to 200e56215a.
 2020-12-17 12:46:42 -08:00
Chris Lu
f56e6d231e Revert "Merge pull request #1683 from qieqieplus/master" 
This reverts commit 8cb67952db, reversing
changes made to 200e56215a.
 2020-12-17 12:46:20 -08:00
Chris Lu
1bf22c0b5b go fmt 2020-12-16 09:14:05 -08:00
Chris Lu
1d88865869 passing disk type along 2020-12-16 09:10:14 -08:00
Chris Lu
23014b6810 Merge branch 'master' into support_ssd_volume 2020-12-16 08:26:51 -08:00
Chris Lu
b789767cd1 filer: return http status 499 for client closed connection 
fix https://github.com/chrislusf/seaweedfs/issues/1684
 2020-12-16 08:18:00 -08:00
chenqieqie
45f902a9b7 smaller critical section 2020-12-16 20:23:51 +08:00
chenqieqie
4e58a4f24e fix race condition when loading volumes concurrently 2020-12-16 18:49:10 +08:00
Chris Lu
3c7f9633eb show error a little bit more obviously 
fix https://github.com/chrislusf/seaweedfs/issues/1678
 2020-12-14 09:39:33 -08:00
Chris Lu
97dcae827b Merge branch 'master' into support_ssd_volume 2020-12-14 01:27:31 -08:00
Chris Lu
3d47c38262 collection.delete requires _default_ as the default empty collection name 
fix https://github.com/chrislusf/seaweedfs/issues/1677
 2020-12-14 01:05:20 -08:00
Chris Lu
2e8dba571b adjust volume server UI 2020-12-14 00:51:57 -08:00
Chris Lu
bf5873022d Merge branch 'master' into support_ssd_volume 2020-12-14 00:32:17 -08:00
Chris Lu
b7e3ca9172
Merge pull request #1650 from kmlebedev/masterServerEnableUI 
security master use access.ui
 2020-12-14 00:31:51 -08:00
Chris Lu
021358749b better detect IP address 
following advice from https://github.com/chrislusf/seaweedfs/issues/1671
 2020-12-14 00:30:20 -08:00
Konstantin Lebedev
c4459249ef ui interface is always on 2020-12-14 13:24:41 +05:00
Chris Lu
5cffddebb9 add logging 2020-12-14 00:11:52 -08:00
Chris Lu
933f2f4cfd find location with matching disk type 2020-12-13 23:39:00 -08:00
Chris Lu
94525aa0fd allocate volume by disk type 2020-12-13 23:08:21 -08:00
Chris Lu
f6a419c26c disk type configurable for each folder 2020-12-13 22:49:56 -08:00
Chris Lu
7c52a35942 refactor 2020-12-13 22:29:52 -08:00
Chris Lu
c45d6a865b Merge branch 'master' into support_ssd_volume 2020-12-13 22:14:18 -08:00
Chris Lu
38fc89d041 filer store: sql update if any insert error happens 
fix https://github.com/chrislusf/seaweedfs/issues/1673
 2020-12-13 20:49:44 -08:00
Chris Lu
23280257df sql put kv: avoid unnecessary update 2020-12-13 19:45:47 -08:00
Chris Lu
a9db24cd05 master allocate volumes if ssd type runs out 2020-12-13 19:44:57 -08:00
Chris Lu
ac4d4a65af Update Makefile 2020-12-13 12:34:58 -08:00
Chris Lu
8baba93fce rename parameter name to "disk" 2020-12-13 12:06:21 -08:00
Chris Lu
51eadaf2b6 rename parameter name to "disk" 2020-12-13 12:05:31 -08:00
Chris Lu
0d2ec832e2 rename from volumeType to diskType 2020-12-13 11:59:32 -08:00
Chris Lu
715b199eeb fix tests 2020-12-13 04:14:50 -08:00
Chris Lu
66cd219b15 Merge branch 'master' into support_ssd_volume 2020-12-13 04:08:36 -08:00
Chris Lu
813453f3e1 2.15 2020-12-13 03:45:35 -08:00
Chris Lu
17372ac3da balance ssd volumes 2020-12-13 03:40:33 -08:00
Chris Lu
d156c74ec0 volume server set volume type and heartbeat to the master 2020-12-13 03:11:24 -08:00
Chris Lu
e9cd798bd3 adding volume type 2020-12-13 00:58:58 -08:00
Chris Lu
16cd6fb278 fix variable name 2020-12-12 21:19:40 -08:00
Chris Lu
5c465293e9 correctly determine whether a folder is empty 
avoid edge cases that deleting the folder if previous 32 directories are all empty

early terminate if one file is found
 2020-12-12 16:19:29 -08:00
Chris Lu
14910d035c the pagination size was too big for recursive deletion 2020-12-12 13:26:10 -08:00
Chris Lu
f930c713fc more efficient recursion 2020-12-12 13:25:19 -08:00
Chris Lu
316d1b4e69 refactor APIs 2020-12-12 12:42:53 -08:00
Chris Lu
03637d6f57 s3: move "delete-directory-if-empty" to read time 
move "delete-directory-if-empty" to read time instead of entry deletion time

the listing speed for a s3 bucket folder will slow down if it has many sub folders

related to 0d345ac97d

fix https://github.com/chrislusf/seaweedfs/issues/1647

fix https://github.com/chrislusf/seaweedfs/issues/1670
 2020-12-12 03:38:34 -08:00
Chris Lu
37075a414d adjust logs 2020-12-12 03:33:57 -08:00
Chris Lu
ae655033ac adjust logging 2020-12-11 16:57:53 -08:00
Chris Lu
d986c7196d use append time instead of filer's own modification time 
fix https://github.com/chrislusf/seaweedfs/issues/1669
 2020-12-11 16:55:18 -08:00
Chris Lu
3fedfec1e7 check cross device rename error 2020-12-10 23:50:32 -08:00
Chris Lu
c2f18a10cb minor 2020-12-10 22:23:22 -08:00
Chris Lu
4b0c2a846b add logs 2020-12-10 19:55:28 -08:00
Chris Lu
83078ac6ce filer: change to /etc/seaweedfs folder on filer 
fix https://github.com/chrislusf/seaweedfs/issues/1666
 2020-12-10 11:11:02 -08:00
Chris Lu
26731694f8 s3: use static configuration by default 
So that users can still use the previous configuration files.

If leave it empty, s3 will try to use the version from filer
 2020-12-10 00:59:04 -08:00
Chris Lu
765b3ef1be save /etc/iam/identity.json inside filer store 2020-12-10 00:15:22 -08:00
Chris Lu
97c942b5ee paginate through large directories 2020-12-09 23:23:38 -08:00
Chris Lu
02a160c3fd s3: recursively iterate all sub folders 
fix https://github.com/chrislusf/seaweedfs/issues/1656
 2020-12-09 16:47:34 -08:00
Chris Lu
a9c619c270 added error handling for saving 2020-12-09 01:34:24 -08:00
Chris Lu
4211601eab set file handle reader to nil 2020-12-08 22:48:18 -08:00
Chris Lu
b52ae9cef8 add s3 debug 2020-12-08 22:27:04 -08:00
Chris Lu
8e78187a97 add back last read chunk cache to reader and properly close the reader 2020-12-08 22:26:46 -08:00
Chris Lu
900d22c6ec mount: avoid memory leaking read buffer 
fix https://github.com/chrislusf/seaweedfs/issues/1654

the reader goes together with the file handle, which may stay for a long time.
 2020-12-08 02:38:53 -08:00
Chris Lu
06bb7bf6c0 adding /usr/local/etc/seaweedfs for bsd style config search path 
related to https://reviews.freebsd.org/D27391
 2020-12-07 16:46:48 -08:00
Chris Lu
fce8803087 break import cycle 2020-12-07 00:29:17 -08:00
Chris Lu
606051cc79 filer: cache "/etc" filder small files to filer store 2020-12-07 00:11:35 -08:00
Chris Lu
ac22f1dd26 filer subscribe: handle rename subscription 2020-12-07 00:10:49 -08:00
Chris Lu
eed87791b7 s3: subscribe to s3.configure changes 2020-12-07 00:10:29 -08:00
Chris Lu
01e2da5782 refactoring 2020-12-06 23:16:20 -08:00
Chris Lu
77286f8bea fix test 2020-12-06 21:56:13 -08:00
Chris Lu
57578a6cd1 change parameter help message 2020-12-06 21:54:55 -08:00
Chris Lu
78f3ab439c refactoring 2020-12-06 21:54:34 -08:00
Chris Lu
9acda432fe fix import cycle 2020-12-06 20:12:52 -08:00
Chris Lu
ae5eb85a06 refactoring 2020-12-06 20:05:06 -08:00
Chris Lu
5931a2f53f filer conf: support json conf only 2020-12-06 19:47:06 -08:00
Chris Lu
4c72482536 shell: fix fs.configure 2020-12-06 19:43:31 -08:00
Chris Lu
46b91228af 2.14 2020-12-06 18:56:58 -08:00
Chris Lu
263eb29e9f filer: add option to cache small files to filer store 2020-12-04 22:39:43 -08:00
Chris Lu
d171d9f988 volume: ensure the volume dat and idx files are always closed 
fix https://github.com/chrislusf/seaweedfs/issues/1646
 2020-12-04 21:50:26 -08:00
Konstantin Lebedev
eadbba5c95 security master use access.ui 2020-12-04 12:57:59 +05:00
Chris Lu
cc839f935d better unit detection 2020-12-03 19:23:59 -08:00
Chris Lu
b219ccfe68 Update volume_ttl_test.go 2020-12-03 12:55:44 -08:00
Chris Lu
a83b8e7b01 ttl fix: 3y was converted into 96m 2020-12-03 10:53:29 -08:00
Chris Lu
5fb60b7135
Merge pull request #1596 from kmlebedev/store_s3cred 
S3 credentials store in filer
 2020-12-03 00:40:20 -08:00
Konstantin Lebedev
14699dfcef use content field of entry 2020-12-02 17:19:05 +05:00
Konstantin Lebedev
a3d4b50a49 use entry content filed 2020-12-02 16:12:13 +05:00
Chris Lu
003b6245e7 fix nil 2020-12-02 00:09:19 -08:00
Chris Lu
f3bb645018 file open error 2020-12-01 23:37:49 -08:00
Chris Lu
dc0bc48257 return file open error 2020-12-01 23:36:49 -08:00
Chris Lu
45dba088cd fix tests 2020-12-01 19:37:21 -08:00
Chris Lu
04062c56c7 webdav: improve webdav upload speed 2020-12-01 15:32:27 -08:00
Konstantin Lebedev
03620776ec Merge branch 'upstream_master' into store_s3cred 2020-12-01 16:03:34 +05:00
Chris Lu
005a6123e9
Merge pull request #1643 from hilimd/master 
fix tls grpc ca path
 2020-11-30 17:16:46 -08:00
Chris Lu
f4abd01adf filer: cache small file to filer store 2020-11-30 04:34:04 -08:00
Chris Lu
a9c6be5fc3 set creation time 2020-11-30 03:11:52 -08:00
Chris Lu
141ce67c09 close http request body 2020-11-30 02:45:00 -08:00
Chris Lu
0e99531dbf 2.13 2020-11-29 17:00:03 -08:00
Chris Lu
965413c21b shell: add volume.vacuum command 2020-11-28 23:18:02 -08:00
Chris Lu
96c48bc8a8 fix test 2020-11-28 03:03:43 -08:00
Chris Lu
c57a7839ec adjust comments 2020-11-28 03:01:03 -08:00
Chris Lu
8438fb0cc3 fix ec shard spreading error 2020-11-28 02:21:16 -08:00
Chris Lu
536202e4e0 minor 2020-11-28 02:16:06 -08:00
Chris Lu
99c4e50d3d minor 2020-11-28 00:14:11 -08:00
Chris Lu
2c913dde04 volume: detect and drop volumes with disk IO error 
from Jethro in slack:
is it possible to make the assign request a bit smarter? Currently I’m in the state that a disk failed but all assign request are being send to this volume. It would be cool if the master sees this and stopped using this volume.
e=HTTP(http://x:8089/913,045a782b63176edf) not 200 but 500 Internal Server Error
Body={"size":740167,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"ee4381e202212ff3aee647704c036689"}
e=HTTP(http://x:8089/913,045a782c90240077) not 200 but 500 Internal Server Error
Body={"size":792779,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"c43463ccc11eb6eb2fc306f407a6a953"}
e=HTTP(http://x:8089/913,045a782e6b7901ea) not 200 but 500 Internal Server Error
Body={"size":3962392,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"04c91198e9b276c81f11dbf189af5d28"}
 2020-11-28 00:09:29 -08:00
Chris Lu
9ac4935f22 read from volume index file directly instead of open a separate file 
fix https://github.com/chrislusf/seaweedfs/issues/1640

read from volume index file directly instead of open a separate file,
to ensure reading latest index entries.
 2020-11-27 16:18:48 -08:00
Chris Lu
85554bea38 filer: readonly handle static resources 
fix https://github.com/chrislusf/seaweedfs/issues/1641
 2020-11-27 13:42:14 -08:00
Chris Lu
6d30b21b10 volume: add "-dir.idx" option for separate index storage 
fix https://github.com/chrislusf/seaweedfs/issues/1265
 2020-11-27 03:17:10 -08:00
Chris Lu
3c229eb677 refactor 2020-11-26 17:21:55 -08:00
Chris Lu
c0fb4a3766 add comments 2020-11-26 15:22:42 -08:00
Chris Lu
6783ae5f8f rename 2020-11-26 15:19:43 -08:00
Chris Lu
983fd90010 marked as WIP 2020-11-26 15:08:52 -08:00
Chris Lu
97f706f35b just refactoring 2020-11-26 15:08:05 -08:00
Chris Lu
b11449f955 pass in dir.idx parameter to DiskLocation 2020-11-26 14:59:03 -08:00
Chris Lu
0d345ac97d s3: remove empty parent folder on delete 
fix https://github.com/chrislusf/seaweedfs/issues/1637
 2020-11-26 12:21:58 -08:00
Chris Lu
cc2bd97ad9 refactor 2020-11-26 11:25:56 -08:00
Chris Lu
0da7ecfd29 go fmt 2020-11-26 11:22:30 -08:00
Chris Lu
effa00ed08 refactor 2020-11-26 11:21:58 -08:00
Chris Lu
513bcd6e0d filer: avoid duplicated FindEntry for deletion 2020-11-26 11:14:56 -08:00
Chris Lu
1ae108efca add debug message 2020-11-26 10:41:10 -08:00
Chris Lu
a102157020 do not retry if failed with a non transport related error 2020-11-26 01:29:30 -08:00
Konstantin Lebedev
4e55baf5b1 s3 config changes 2020-11-26 03:50:53 +05:00
Konstantin Lebedev
52c8f2fc9a s3iam test 2020-11-26 02:26:45 +05:00
Konstantin Lebedev
a26f1b2040 new pkg s3iam 2020-11-26 01:30:11 +05:00
Konstantin Lebedev
6206737df2 s3 configure 2020-11-25 21:02:31 +05:00
Chris Lu
eab53ea80d filer leveldb store: a bit more efficient directory listing with prefix 2020-11-22 21:10:41 -08:00
Chris Lu
d3be2ceedd 2.12 2020-11-22 17:17:33 -08:00
Chris Lu
c7ebadc25d avoid possible concurrent access inside ensureCorrectWritables() 2020-11-22 17:15:59 -08:00
hilimd
9aa990f80f
Merge pull request #39 from chrislusf/master 
sync
 2020-11-22 20:29:17 +08:00
limd
e8296104fc fix tls grpc ca path 2020-11-22 20:27:15 +08:00
Chris Lu
92f906b6fc remove zstd 
fix https://github.com/chrislusf/seaweedfs/issues/1629
 2020-11-21 13:06:45 -08:00
Aleksandr Demshin
176950ea78 fix typo 2020-11-20 19:50:46 +05:00
Chris Lu
7385103057 fix volume placement validating 
fix https://github.com/chrislusf/seaweedfs/issues/1626
 2020-11-20 01:12:25 -08:00
Konstantin Lebedev
27e73de797 Merge branch 'upstream_master' into store_s3cred 
# Conflicts:
#	weed/s3api/filer_util.go
 2020-11-19 18:16:44 +05:00
Chris Lu
290b5e2cd0 directly delete file chunks 
keeping current async deletions for now
 2020-11-17 17:20:21 -08:00
Chris Lu
781585b195 remove unnecessary checking 2020-11-17 17:00:06 -08:00
Chris Lu
8cb8cd4cc5 add locks 2020-11-17 16:59:48 -08:00
Chris Lu
7c9e592c2d s3: avoid nil resp when having error 
fix https://github.com/chrislusf/seaweedfs/issues/1622
 2020-11-17 11:23:17 -08:00
Chris Lu
dc304342b2 fs.configure: configurable volume growth 2020-11-17 01:00:02 -08:00
Chris Lu
b04375a3d1 use alternative assign request 2020-11-17 00:36:21 -08:00
Chris Lu
9add554feb fail fast if user configures collection for a bucket 2020-11-16 19:57:08 -08:00
Chris Lu
6204dc6e85 add comment 2020-11-16 16:57:31 -08:00
Chris Lu
ef3e068c51 deprecating "filer.options.buckets_fsync" in filer.toml 2020-11-16 16:56:49 -08:00
Chris Lu
5f19e81dab filer confi: support hierachical configuration 2020-11-16 16:50:12 -08:00
Chris Lu
ed1ce3f299 adjust help message 2020-11-16 09:54:26 -08:00
Chris Lu
c0d279c54e filere.conf: prefer to use json format 2020-11-15 21:48:17 -08:00
Chris Lu
83527a8f55 adds more help message 2020-11-15 21:31:14 -08:00
Chris Lu
af658ea970 fix typo 2020-11-15 21:16:28 -08:00
Chris Lu
98827d68d9 2.11 2020-11-15 20:25:56 -08:00
Chris Lu
ee2fa14dbe filer conf: delete location specific configuration 2020-11-15 20:15:47 -08:00
Chris Lu
2bd6fd3bbe remove unused function 2020-11-15 20:15:07 -08:00
Chris Lu
71056dae07 fs.configure: read and local add filer configuration 2020-11-15 18:09:35 -08:00
Chris Lu
0ea5c087ce go fmt 2020-11-15 16:59:28 -08:00
Chris Lu
95c0de285d refactoring 2020-11-15 16:58:48 -08:00
Chris Lu
500bcab953 refactoring 2020-11-15 14:41:56 -08:00
Chris Lu
590f02179d filer: load filer conf when starting 2020-11-15 14:06:03 -08:00
Chris Lu
0a406f652e load filer conf and match by prefix 2020-11-15 00:26:05 -08:00
Chris Lu
68043cfcac add reference implementation to detect create/update/delete/rename events 2020-11-14 21:21:58 -08:00
Chris Lu
0fc9ffc603 able to read filer.conf if updated 2020-11-14 21:21:20 -08:00
Chris Lu
442e092995 filer watch "/etc" folder for configuration changes 2020-11-14 14:26:08 -08:00
ruitao.liu
a9990a1dc6 adjust check bucket if exist or has access. 2020-11-13 17:13:20 +08:00
Chris Lu
e6333da65a enable admin to access all buckets 2020-11-12 13:57:54 -08:00
Chris Lu
7094492428 fix ErrNotFound 2020-11-12 13:30:46 -08:00
Chris Lu
c6a0704019 rename function 2020-11-12 13:30:08 -08:00
Chris Lu
ed3b0f81c6
Merge pull request #1613 from taozix/master 
check permission for bucket delete/head.
 2020-11-12 13:25:28 -08:00
ruitao.liu
c4f0fd6e1b skip if entry.Extended map is nil. 2020-11-12 17:59:31 +08:00
ruitao.liu
ab966410d2 return NoSuchBucket instead of InternalError delete non-existed bucket. 2020-11-12 16:44:16 +08:00
Chris Lu
a4dfa00030 adjust filer conf 2020-11-12 00:39:59 -08:00
Chris Lu
2251a8e472 minor 2020-11-12 00:38:58 -08:00
Chris Lu
5a16f17e47 remove unused message type 2020-11-12 00:38:23 -08:00
ruitao.liu
e06676f007 check permission for bucket delete/head. 2020-11-12 16:15:59 +08:00
Chris Lu
c32b6f4d9d prepare for path based configuration 2020-11-11 23:46:45 -08:00
Chris Lu
0fbc27b640
Merge pull request #1612 from taozix/master 
add owner attr for bucket.
 2020-11-11 22:19:47 -08:00
ruitao.liu
d7cc0498e0 check if bucket already exists. 2020-11-12 14:11:03 +08:00
Chris Lu
827930e249
Merge pull request #1610 from kmlebedev/prefer_read_in_this_dc 
prefer to read from volumes in this data center
 2020-11-11 22:09:41 -08:00
Konstantin Lebedev
0880aff224 update dc flag desc 2020-11-12 10:07:52 +05:00
ruitao.liu
dbba8cb57e fix if nil map in entry. 2020-11-12 11:50:19 +08:00
Konstantin Lebedev
aa30604cec fix tests 2020-11-12 08:49:38 +05:00
Konstantin Lebedev
1eec5c8d5d gen pb 2020-11-12 04:10:06 +05:00
Konstantin Lebedev
fc7baef5bb fiil serverUrls sorted by data center 2020-11-12 02:13:33 +05:00
Chris Lu
745ee8d8f3 avoid unnecessary error message 
fix https://github.com/chrislusf/seaweedfs/issues/1611
 2020-11-11 13:00:05 -08:00
Chris Lu
a29abd8600 add a {X-,}Idempotency-Key header for http POST 
fix https://github.com/chrislusf/seaweedfs/issues/1592

according to https://github.com/golang/go/issues/19943
 2020-11-11 12:56:53 -08:00
Chris Lu
8708e7a64d print out data size 2020-11-11 12:51:44 -08:00
Chris Lu
c6bd244ebd add TODO 2020-11-11 12:51:27 -08:00
Konstantin Lebedev
b73ef6aa95 Merge remote-tracking branch 'upstream/master' into prefer_read_in_this_dc 2020-11-11 23:29:57 +05:00
Konstantin Lebedev
dc26012a3b initial 2020-11-11 15:03:47 +05:00
Chris Lu
86cdb2a3e5 adjust logs 2020-11-11 02:01:24 -08:00
Chris Lu
73f934d5de s3: do not close reader too early 
fix https://github.com/chrislusf/seaweedfs/issues/1609
 2020-11-11 01:49:39 -08:00
ruitao.liu
5b636b3242 Add bucket owner attr. 2020-11-11 16:20:59 +08:00
Chris Lu
15956d8a8e 2.10 2020-11-10 23:17:13 -08:00
Chris Lu
de3bdd0651 delete old volume replica 
related to https://github.com/chrislusf/seaweedfs/issues/1607

old is:
* older compaction revision
* older modified time
* smaller volume size
 2020-11-10 12:26:05 -08:00
Chris Lu
1dd3a6ac36 avoid verbose logs 2020-11-09 15:56:11 -08:00
Chris Lu
6856b0d57e filer: add API to add/modify/delete tagging 2020-11-09 01:00:07 -08:00
Konstantin Lebedev
6cd6ff6962 avoid blank response for 404 2020-11-04 22:54:47 +05:00
Chris Lu
0dafcf1f5a volume: detect max volume count changes based on disk usage 
fix https://github.com/chrislusf/seaweedfs/issues/1594
 2020-11-03 14:43:17 -08:00
Chris Lu
06ee199ef3 ensure entryViewCache is in sync with the chunks 2020-11-03 12:22:19 -08:00
Chris Lu
b132cb893f in case some chunks modified earliar are added later 2020-11-03 12:22:19 -08:00
Chris Lu
0ae4d7bfdf lock "addChunks()" 2020-11-03 12:22:19 -08:00
Konstantin Lebedev
e1190b3224 load S3 config from filer 
https://github.com/chrislusf/seaweedfs/issues/1500
 2020-11-03 21:45:56 +05:00
Chris Lu
d29b787220 store together with "Seaweed-" prefix 2020-11-03 00:21:10 -08:00
Chris Lu
5448781dfc filer: Headers with "Seaweed-" prefix are stored as extended properties 2020-11-03 00:15:51 -08:00
Chris Lu
0ecff9521d avoid []byte reuse 2020-11-02 23:27:54 -08:00
Chris Lu
031aa70da6 Revert "revert bytebufferpool, seems problematic" 
This reverts commit e0fcab47c0.
 2020-11-02 19:11:25 -08:00
Chris Lu
7d5db73d98 go fmt 2020-11-02 14:21:07 -08:00
Chris Lu
a9662d44ac avoid closing chunkSaveErrChan too early 2020-11-02 14:20:38 -08:00
Chris Lu
e0fcab47c0 revert bytebufferpool, seems problematic 2020-11-02 13:49:38 -08:00
Chris Lu
e1423a83dd mount: clear possible leftover entry data in memory 2020-11-01 23:28:56 -08:00
Chris Lu
9b10d1ac33 2.08 2020-11-01 12:45:20 -08:00
Chris Lu
8750cac090 move to util.RetryWaitTime 2020-11-01 02:36:43 -08:00
Chris Lu
ef908e166b break import cycle 2020-11-01 02:28:11 -08:00
Chris Lu
4609644658 mount: remove files that may still are open 
related to https://github.com/chrislusf/seaweedfs/issues/1581
 2020-11-01 01:33:26 -08:00
Chris Lu
df8d976bb0 refactoring 2020-11-01 01:58:48 -07:00
Chris Lu
7192a378cc add retry to master clients also 2020-11-01 01:40:16 -07:00
Chris Lu
004aa9ca5d only retry transport related errors 2020-11-01 01:21:44 -07:00