mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-19 05:27:50 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,460 Commits 32 Branches 296 Tags 152 MiB
fa25d5c43a
Commit Graph

1194 Commits

Author SHA1 Message Date
Chris Lu
3be3c17f59 volume vacuum: avoid timeout with streaming progress report 
fix https://github.com/chrislusf/seaweedfs/issues/2396
 2021-10-24 01:55:34 -07:00
Konstantin Lebedev
c78220a7f2 fix object tagging https://github.com/chrislusf/seaweedfs/issues/2389 2021-10-20 16:01:06 +05:00
Chris Lu
29fa1b9fdc add logs 2021-10-17 13:50:34 -07:00
Chris Lu
cca62fdb30 mount: streaming renaming folders 2021-10-17 04:22:42 -07:00
Eng Zer Jun
a23bcbb7ec
refactor: move from io/ioutil to io and os package 
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2021-10-14 12:27:58 +08:00
Chris Lu
de1ec050e8 fier/s3: save "Content-Disposition" to extended properties 
related to https://github.com/chrislusf/seaweedfs/issues/2371
 2021-10-12 00:04:28 -07:00
Chris Lu
34f764007f escape file name only when necessary 2021-10-11 23:33:45 -07:00
Chris Lu
8e9273db99 s3: use "response-content-disposition" to overwrite default content-disposition 
fix one part of https://github.com/chrislusf/seaweedfs/issues/2371

see https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
 2021-10-11 23:29:52 -07:00
Chris Lu
bf218cd59d removing etcd sequencer 
causing go mod tidy problem. If anyone wants this, please help to resolve this first.

github.com/chrislusf/seaweedfs/weed/sequence imports
	go.etcd.io/etcd/client tested by
	go.etcd.io/etcd/client.test imports
	github.com/coreos/etcd/integration imports
	github.com/coreos/etcd/proxy/grpcproxy imports
	google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.41.0), but does not contain package google.golang.org/grpc/naming
 2021-10-10 19:10:46 -07:00
Chris Lu
332d49432d reduce concurrent volume grow requests 2021-10-05 01:58:30 -07:00
Chris Lu
96119eab00 refactor 2021-10-05 00:40:04 -07:00
Chris Lu
366f522a2d add redis3 2021-10-04 01:01:31 -07:00
Chris Lu
2e9372dcf7 volume stream read skips deleted content 2021-09-27 03:07:44 -07:00
Chris Lu
225b019fe0 stream read multiple volumes in a volume server 2021-09-27 02:51:31 -07:00
Chris Lu
1904448d4e adjust starting offset 2021-09-27 02:01:29 -07:00
Chris Lu
aa64f2ac4c send needle data instead of raw needle body 2021-09-27 01:53:41 -07:00
Chris Lu
c4d7ee6c5c volume server: read all files in a volume 2021-09-27 01:45:32 -07:00
Chris Lu
603ea2db73 avoid looping forever if there are no more metadata updates 2021-09-26 11:55:27 -07:00
Chris Lu
ede7a65a50 update log 2021-09-19 01:33:50 -07:00
Chris Lu
c1255d30ad add cache error log 2021-09-19 01:33:26 -07:00
Chris Lu
2789d10342 go fmt 2021-09-14 10:37:06 -07:00
Chris Lu
7504be58f9 Avoid xattr printed out as HTTP headers 
fix https://github.com/chrislusf/seaweedfs/issues/2336
 2021-09-13 04:00:57 -07:00
Chris Lu
f74b29416a better etag matching 2021-09-13 00:31:46 -07:00
Chris Lu
e9760f261e minor 2021-09-13 00:31:06 -07:00
Chris Lu
e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu
7591336a22 log format 2021-09-11 14:27:57 -07:00
Chris Lu
7e600bff5f adjust log level 2021-09-11 12:59:15 -07:00
Chris Lu
a243d7e047 EC: clean up if failed in the middle 2021-09-11 02:07:14 -07:00
Chris Lu
f7f01ac209 return if error 2021-09-11 02:06:43 -07:00
Chris Lu
59cb6146a3 refactor 2021-09-11 02:06:30 -07:00
Chris Lu
5496d68f6a increase counter only if not early terminated 2021-09-11 02:05:55 -07:00
Chris Lu
0207f5fe9b replicated remote.cache 2021-09-08 15:54:55 -07:00
Chris Lu
0128239c0f handle ipv6 addresses 2021-09-07 16:43:54 -07:00
Chris Lu
2b1feb732c remote.cache supports replication 2021-09-06 18:30:44 -07:00
Chris Lu
6923af7280 refactoring 2021-09-06 16:20:49 -07:00
Chris Lu
e93d4935e3 add other replica locations when assigning volumes 2021-09-05 23:32:25 -07:00
Chris Lu
7a13816e94 refactor 2021-09-05 23:17:15 -07:00
Chris Lu
006c01a519 fix format 2021-09-05 16:18:50 -07:00
Chris Lu
65af3cf4df master: disconnect only the phantom volume server 
fix https://github.com/chrislusf/seaweedfs/issues/2311
 2021-09-05 15:20:03 -07:00
Chris Lu
24fda6f47f remove QR code for mobile upload 2021-09-01 16:59:37 -07:00
Chris Lu
7ce97b59d8 go fmt 2021-09-01 02:45:42 -07:00
Chris Lu
d1a4e19a3f volume: copy file also copies modification time 
to ensure ttl can work well
 2021-09-01 02:42:57 -07:00
Chris Lu
43fd11278e support follow additional path prefixes 2021-08-31 23:23:08 -07:00
Chris Lu
05f32376eb add cluster id in filer configuration response 2021-08-29 21:02:10 -07:00
Chris Lu
6a0bb7106b cloud drive: parallelize remote storage downloading 2021-08-26 16:16:26 -07:00
Chris Lu
05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
mauroparente
c63713774b
Fix image resize 
https://github.com/chrislusf/seaweedfs/issues/2262
 2021-08-18 23:15:35 +02:00
Chris Lu
570d2eccec skip body if not allowed by http status 2021-08-18 00:56:35 -07:00
Chris Lu
e2aa3cf63b fix go test 2021-08-15 23:20:46 -07:00
Chris Lu
0b2a92d371 skip if http.StatusNoContent 2021-08-15 11:27:49 -07:00
Chris Lu
c34747c79d rename, fix wrong logic. 2021-08-14 21:46:34 -07:00
Chris Lu
889b143fa7 adjust modification detection logic 2021-08-14 15:44:47 -07:00
Chris Lu
f365af81c2 parallelize remote content fetching 2021-08-14 15:41:37 -07:00
Chris Lu
53e66980b2 add comments 2021-08-14 15:16:10 -07:00
Chris Lu
9921801e0c Revert "use default or path-specific setting for cache replication level" 
This reverts commit ba6923b223.
 2021-08-14 15:14:26 -07:00
Chris Lu
ba6923b223 use default or path-specific setting for cache replication level 2021-08-14 15:14:01 -07:00
Chris Lu
8126ab4b5d rename 2021-08-14 05:03:45 -07:00
Chris Lu
7937db52e1 Filer locationPrefix configure does not exec replication #2257 
fix https://github.com/chrislusf/seaweedfs/issues/2257
 2021-08-14 02:54:13 -07:00
Chris Lu
f4decf02df volume copying: clean up stale volume data files 
fix https://github.com/chrislusf/seaweedfs/issues/2250
 2021-08-13 03:24:21 -07:00
Chris Lu
78e8ddf910 Only when tailing volume, the zero-ed cookie should skip checking. 
This only happens when checkCookie == false and fsync == false.
 2021-08-13 02:09:35 -07:00
Chris Lu
5a0f92423e use grpc and jwt 2021-08-12 21:40:33 -07:00
Chris Lu
6238644c35 remove gateway command 2021-08-12 20:52:04 -07:00
Chris Lu
d1d1fc772c move some volume lookup operations to grpc 
jwt related lookup will come in next commit
 2021-08-12 20:33:00 -07:00
Chris Lu
5571f4f70a master: add master.follower to handle read file id lookup requests 2021-08-12 18:10:59 -07:00
Chris Lu
5469019852 adjust data type 2021-08-12 17:54:34 -07:00
Chris Lu
f2cd753bf9 fix avoid lock error 
fix https://github.com/chrislusf/seaweedfs/issues/2247
 2021-08-10 14:34:13 -07:00
Chris Lu
1154e23e2d add logs for volume creation 2021-08-10 13:04:25 -07:00
Chris Lu
69655ba8e5 mount: cache on reading remote storage 2021-08-09 22:11:57 -07:00
Chris Lu
a7012d9729 fix 2021-08-09 16:03:03 -07:00
Chris Lu
8d3e275735 remote: filer cache remote content on read 2021-08-09 15:16:45 -07:00
Chris Lu
9096f6f4f7 cache: set upper limit of chunk size 2021-08-09 15:08:53 -07:00
Chris Lu
a6be2520c9 fix 2021-08-09 14:37:25 -07:00
Chris Lu
713c035a6e shell: remote.cache remote.uncache 2021-08-09 14:35:18 -07:00
Chris Lu
734c980040 volume: support concurrent download data size limit 2021-08-08 23:25:16 -07:00
Chris Lu
96ce85f5ae rename 2021-08-08 22:33:31 -07:00
Chris Lu
c5f38c365d go fmt 2021-08-08 22:30:36 -07:00
Chris Lu
8f5170c138 remove imports 2021-08-07 16:20:17 -07:00
Chris Lu
270770d7d7 refactor 2021-08-07 14:18:53 -07:00
Chris Lu
d84c311699 refactoring 2021-08-04 12:30:18 -07:00
Chris Lu
767edd3c08 rename 2021-07-31 23:52:09 -07:00
Chris Lu
9df7d16791 read <- remote_storage 2021-07-31 22:39:38 -07:00
Chris Lu
c090d6bb25 add ReadRemote(), add read remote setup when filer starts 2021-07-28 22:43:12 -07:00
Chris Lu
10464f47d0
Merge pull request #2217 from combineads/add_webp_format 2021-07-24 09:42:41 -07:00
byunghwa.yun
7374b5b473 Add webp extension for resizing 2021-07-24 14:32:31 +09:00
Chris Lu
ac28611817 snowflake sequencer need an unique id 
fix https://github.com/chrislusf/seaweedfs/issues/2213
 2021-07-23 20:54:03 -07:00
Chris Lu
7ab389e7ec optimization: improve random range query for large files 2021-07-19 23:07:22 -07:00
Chris Lu
b938df97a2 remove unused parameter 2021-07-19 02:59:12 -07:00
Chris Lu
450222dd64 add remote to filer.Entry and filer_pb entry, add RemoteConf 2021-07-19 02:47:27 -07:00
Chris Lu
18c40686d9 s3: multipart upload miss data if file is chunked in 4MB 
fix https://github.com/chrislusf/seaweedfs/issues/2195
 2021-07-15 11:56:28 -07:00
Chris Lu
49c66e88a0 volume: change all writes to fsync during graceful stopping 
fix https://github.com/chrislusf/seaweedfs/issues/2193
 2021-07-13 01:29:57 -07:00
Chris Lu
b194f91f47 add version to filer configuration response 2021-07-12 01:23:20 -07:00
Chris Lu
da7bd62822 /etc files are stored inside metadata store 2021-07-09 02:33:14 -07:00
Chris Lu
a6d73e0a66
Merge pull request #2185 from bingoohuang/master 
show RemoteVolumes/EcVolumes only if it is not empty
 2021-07-06 00:28:07 -07:00
bingoohuang
ed57a55eae show RemoteVolumes/EcVolumes only if it is not empty 2021-07-06 15:20:18 +08:00
Chris Lu
141388367e
Merge pull request #2183 from bingoohuang/master 
extract embed html of master/volume/filer ui to separate files
 2021-07-05 03:22:26 -07:00
bingoohuang
44a2538f67 extract embed html of master/volume/filer ui to separate files 2021-07-05 18:09:44 +08:00
qieqieplus
233103f6b2 sync empty notification with timestamp 2021-07-05 16:01:16 +08:00
Chris Lu
8fe75692ee volume: address "unaligned 64-bit atomic operation" 
fix https://github.com/chrislusf/seaweedfs/issues/2177
 2021-07-02 13:57:43 -07:00
Chris Lu
c6d4c16079 S3: add metadata with multipart upload 
fix https://github.com/chrislusf/seaweedfs/issues/2173
 2021-07-01 19:12:11 -07:00
Chris Lu
2420c60fc4 log reading adds delay between retries 2021-07-01 14:01:25 -07:00
Chris Lu
067eb15e70 remove debug messages 2021-07-01 01:24:07 -07:00
Chris Lu
b624090398 go fmt 2021-07-01 01:21:14 -07:00
Chris Lu
215b169562 mount: recursively rename locally 2021-07-01 01:19:31 -07:00
zhangsong
20d33ae025 add proxy mode to read non-local volumes 2021-06-30 18:33:18 +08:00
zhangsong
7566782c2e add proxy mode to read non-local volumes 2021-06-30 17:28:37 +08:00
Chris Lu
3668d10664 range query for mp4 video play 
fix https://github.com/chrislusf/seaweedfs/issues/2156
 2021-06-29 02:13:29 -07:00
Chris Lu
17477b37d5 sleep before re-reading the messages 
If there are no more metadata changes and the client disconnects, it would go into a busy loop without this fix.
 2021-06-27 06:31:04 -07:00
Chris Lu
cc7714fdbe logging changes to debug 2021-06-27 05:54:16 -07:00
Chris Lu
c2e0a75c1f adjust logs 2021-06-24 12:46:00 -07:00
Chris Lu
78b1fb921c adjust log level 2021-06-23 20:59:54 -07:00
Chris Lu
f24bb9e688 mount: fix for deletion stopped working since 2.53 
fix https://github.com/chrislusf/seaweedfs/issues/2138

due to 4d0cbd2700
 2021-06-19 03:46:39 -07:00
Chris Lu
88d52adfdd remove unused fields 2021-06-18 15:35:22 -07:00
Chris Lu
ed6aa13520 minor 2021-06-12 18:36:25 -07:00
Chris Lu
ee6c67682c minor 2021-06-12 02:52:41 -07:00
Chris Lu
5e6dfbc25f locks for data racing 2021-06-07 12:04:50 -07:00
Chris Lu
452c6ef183 limits concurrent uploads for one file 2021-06-06 23:05:17 -07:00
Chris Lu
8295e2feb6 skip md5 checking for now because of race condition 2021-06-06 22:16:32 -07:00
Chris Lu
19caeb7b02 fix writing the small file 2021-06-06 20:57:03 -07:00
Chris Lu
bb45dea15a filer: parallel data upload 2021-06-06 20:23:36 -07:00
Chris Lu
44f1ba6894 refactor 2021-06-06 18:43:04 -07:00
Chris Lu
6c82326575 use bytes.Buffer to reduce memory allocation and gc 2021-06-06 13:42:36 -07:00
Chris Lu
9cba5cca0b optionally disable concurrent upload limit 2021-06-06 13:13:33 -07:00
Chris Lu
ab606dec2a filer: add path-specific option to enforce readonly 2021-06-04 01:03:41 -07:00
Chris Lu
fb8036385a s3: save metadata during put-object 
fix https://github.com/chrislusf/seaweedfs/issues/2092
 2021-05-29 14:14:30 -07:00
Chris Lu
ac9bf71544 use jquery 3.6.0 2021-05-26 00:21:11 -07:00
Konstantin Lebedev
b612d5aebd s3 test get w/ If-Match: bogus ETag 2021-05-24 16:59:44 +05:00
Konstantin Lebedev
69c768870b - object write cache control 
- object write expires
 2021-05-24 15:43:55 +05:00
Konstantin Lebedev
84dce32a57
Merge branch 'master' into head_check_all_chunks 2021-05-24 12:28:19 +05:00
Chris Lu
064269bb57 filer: Support sqlite as filer meta store 2021-05-23 23:58:28 -07:00
Chris Lu
e5a2bf1287 s3: deprecating filer.options.buckets_fsync 2021-05-22 00:24:23 -07:00
Konstantin Lebedev
26a4f34a57 del checks 
url err logging
stats chunk fetch
 2021-05-21 15:59:12 +05:00
Konstantin Lebedev
03d1199d5f Revert "revert PR #1903 avoid http error: superfluous response.WriteHeader" 
This reverts commit ac71117e
 2021-05-20 11:45:21 +05:00
Chris Lu
da0a4e775b Revert "Revert "Merge pull request #2027 from bingoohuang/master"" 
This reverts commit d74cdf0115.
 2021-05-10 00:13:55 -07:00
Chris Lu
7ca75347ec minor 2021-05-07 21:56:45 -07:00
Chris Lu
8f8738867f add retry to assign volume 
fix https://github.com/chrislusf/seaweedfs/issues/2056
 2021-05-07 07:29:26 -07:00
Chris Lu
aaad4b578b
Merge pull request #2049 from qieqieplus/async-assign 
ahead of time volume assignment
 2021-05-06 10:00:25 -07:00
qieqieplus
c4d32f6937 ahead of time volume assignment 2021-05-06 18:55:44 +08:00
Chris Lu
c899bdf063 a little optimization 2021-05-06 03:03:00 -07:00
Chris Lu
3eb336e0b0 report error only for the first multipart upload 
the glog.Errorf would always print for s3 multipart uploads
 2021-05-06 01:57:54 -07:00
Chris Lu
ac71117ee6 revert PR #1903 avoid http error: superfluous response.WriteHeader 2021-05-05 15:11:39 -07:00
Chris Lu
24efa31e49
Merge pull request #2045 from qieqieplus/fix-vacuum-commit 2021-05-05 07:41:38 -07:00
qieqieplus
ac26080bd2 fix concurrent vacuum & delete panic 2021-05-05 17:54:50 +08:00
Chris Lu
e24ba2aadc filer: delete specific tags 
fix https://github.com/chrislusf/seaweedfs/issues/2041
 2021-05-02 21:53:43 -07:00
Chris Lu
d74cdf0115 Revert "Merge pull request #2027 from bingoohuang/master" 
Need to revert because docker image build failed. The docker apk package only has go 1.15.
 2021-04-30 03:36:15 -07:00
Konstantin Lebedev
c2269123d3 fix aws style Etag for chunks 2021-04-28 22:28:05 +05:00
bingoohuang
7a9d27fce8 promote to go:embed instead of github.com/rakyll/statik 2021-04-27 17:22:24 +08:00
bingoohuang
cf552417a7 minFreeSpace refactored 2021-04-27 10:37:24 +08:00
bingoohuang
31f1cdeac2 minFreeSpace argument allows size like 10GiB 2021-04-26 18:48:34 +08:00
Chris Lu
ddc8643ee0 filer: directory listing adds namePatternExclude 
fix https://github.com/chrislusf/seaweedfs/issues/2023
 2021-04-24 11:49:03 -07:00
Chris Lu
f0ad172e80 shell: show which server holds the lock 
fix https://github.com/chrislusf/seaweedfs/issues/1983
 2021-04-22 23:56:35 -07:00