mauroparente
c63713774b
Fix image resize
...
https://github.com/chrislusf/seaweedfs/issues/2262
2021-08-18 23:15:35 +02:00
Chris Lu
570d2eccec
skip body if not allowed by http status
2021-08-18 00:56:35 -07:00
Chris Lu
e2aa3cf63b
fix go test
2021-08-15 23:20:46 -07:00
Chris Lu
0b2a92d371
skip if http.StatusNoContent
2021-08-15 11:27:49 -07:00
Chris Lu
c34747c79d
rename, fix wrong logic.
2021-08-14 21:46:34 -07:00
Chris Lu
889b143fa7
adjust modification detection logic
2021-08-14 15:44:47 -07:00
Chris Lu
f365af81c2
parallelize remote content fetching
2021-08-14 15:41:37 -07:00
Chris Lu
53e66980b2
add comments
2021-08-14 15:16:10 -07:00
Chris Lu
9921801e0c
Revert "use default or path-specific setting for cache replication level"
...
This reverts commit ba6923b223
2021-08-14 15:14:26 -07:00
Chris Lu
ba6923b223
use default or path-specific setting for cache replication level
2021-08-14 15:14:01 -07:00
Chris Lu
8126ab4b5d
rename
2021-08-14 05:03:45 -07:00
Chris Lu
7937db52e1
Filer locationPrefix configure does not exec replication #2257
...
fix https://github.com/chrislusf/seaweedfs/issues/2257
2021-08-14 02:54:13 -07:00
Chris Lu
f4decf02df
volume copying: clean up stale volume data files
...
fix https://github.com/chrislusf/seaweedfs/issues/2250
2021-08-13 03:24:21 -07:00
Chris Lu
78e8ddf910
Only when tailing volume, the zero-ed cookie should skip checking.
...
This only happens when checkCookie == false and fsync == false.
2021-08-13 02:09:35 -07:00
Chris Lu
5a0f92423e
use grpc and jwt
2021-08-12 21:40:33 -07:00
Chris Lu
6238644c35
remove gateway command
2021-08-12 20:52:04 -07:00
Chris Lu
d1d1fc772c
move some volume lookup operations to grpc
...
jwt related lookup will come in next commit
2021-08-12 20:33:00 -07:00
Chris Lu
5571f4f70a
master: add master.follower to handle read file id lookup requests
2021-08-12 18:10:59 -07:00
Chris Lu
5469019852
adjust data type
2021-08-12 17:54:34 -07:00
Chris Lu
f2cd753bf9
fix avoid lock error
...
fix https://github.com/chrislusf/seaweedfs/issues/2247
2021-08-10 14:34:13 -07:00
Chris Lu
1154e23e2d
add logs for volume creation
2021-08-10 13:04:25 -07:00
Chris Lu
69655ba8e5
mount: cache on reading remote storage
2021-08-09 22:11:57 -07:00
Chris Lu
a7012d9729
fix
2021-08-09 16:03:03 -07:00
Chris Lu
8d3e275735
remote: filer cache remote content on read
2021-08-09 15:16:45 -07:00
Chris Lu
9096f6f4f7
cache: set upper limit of chunk size
2021-08-09 15:08:53 -07:00
Chris Lu
a6be2520c9
fix
2021-08-09 14:37:25 -07:00
Chris Lu
713c035a6e
shell: remote.cache remote.uncache
2021-08-09 14:35:18 -07:00
Chris Lu
734c980040
volume: support concurrent download data size limit
2021-08-08 23:25:16 -07:00
Chris Lu
96ce85f5ae
rename
2021-08-08 22:33:31 -07:00
Chris Lu
c5f38c365d
go fmt
2021-08-08 22:30:36 -07:00
Chris Lu
8f5170c138
remove imports
2021-08-07 16:20:17 -07:00
Chris Lu
270770d7d7
refactor
2021-08-07 14:18:53 -07:00
Chris Lu
d84c311699
refactoring
2021-08-04 12:30:18 -07:00
Chris Lu
767edd3c08
rename
2021-07-31 23:52:09 -07:00
Chris Lu
9df7d16791
read <- remote_storage
2021-07-31 22:39:38 -07:00
Chris Lu
c090d6bb25
add ReadRemote(), add read remote setup when filer starts
2021-07-28 22:43:12 -07:00
Chris Lu
10464f47d0
Merge pull request #2217 from combineads/add_webp_format
2021-07-24 09:42:41 -07:00
byunghwa.yun
7374b5b473
Add webp extension for resizing
2021-07-24 14:32:31 +09:00
Chris Lu
ac28611817
snowflake sequencer need an unique id
...
fix https://github.com/chrislusf/seaweedfs/issues/2213
2021-07-23 20:54:03 -07:00
Chris Lu
7ab389e7ec
optimization: improve random range query for large files
2021-07-19 23:07:22 -07:00
Chris Lu
b938df97a2
remove unused parameter
2021-07-19 02:59:12 -07:00
Chris Lu
450222dd64
add remote to filer.Entry and filer_pb entry, add RemoteConf
2021-07-19 02:47:27 -07:00
Chris Lu
18c40686d9
s3: multipart upload miss data if file is chunked in 4MB
...
fix https://github.com/chrislusf/seaweedfs/issues/2195
2021-07-15 11:56:28 -07:00
Chris Lu
49c66e88a0
volume: change all writes to fsync during graceful stopping
...
fix https://github.com/chrislusf/seaweedfs/issues/2193
2021-07-13 01:29:57 -07:00
Chris Lu
b194f91f47
add version to filer configuration response
2021-07-12 01:23:20 -07:00
Chris Lu
da7bd62822
/etc files are stored inside metadata store
2021-07-09 02:33:14 -07:00
Chris Lu
a6d73e0a66
Merge pull request #2185 from bingoohuang/master
...
show RemoteVolumes/EcVolumes only if it is not empty
2021-07-06 00:28:07 -07:00
bingoohuang
ed57a55eae
show RemoteVolumes/EcVolumes only if it is not empty
2021-07-06 15:20:18 +08:00
Chris Lu
141388367e
Merge pull request #2183 from bingoohuang/master
...
extract embed html of master/volume/filer ui to separate files
2021-07-05 03:22:26 -07:00
bingoohuang
44a2538f67
extract embed html of master/volume/filer ui to separate files
2021-07-05 18:09:44 +08:00
qieqieplus
233103f6b2
sync empty notification with timestamp
2021-07-05 16:01:16 +08:00
Chris Lu
8fe75692ee
volume: address "unaligned 64-bit atomic operation"
...
fix https://github.com/chrislusf/seaweedfs/issues/2177
2021-07-02 13:57:43 -07:00
Chris Lu
c6d4c16079
S3: add metadata with multipart upload
...
fix https://github.com/chrislusf/seaweedfs/issues/2173
2021-07-01 19:12:11 -07:00
Chris Lu
2420c60fc4
log reading adds delay between retries
2021-07-01 14:01:25 -07:00
Chris Lu
067eb15e70
remove debug messages
2021-07-01 01:24:07 -07:00
Chris Lu
b624090398
go fmt
2021-07-01 01:21:14 -07:00
Chris Lu
215b169562
mount: recursively rename locally
2021-07-01 01:19:31 -07:00
zhangsong
20d33ae025
add proxy mode to read non-local volumes
2021-06-30 18:33:18 +08:00
zhangsong
7566782c2e
add proxy mode to read non-local volumes
2021-06-30 17:28:37 +08:00
Chris Lu
3668d10664
range query for mp4 video play
...
fix https://github.com/chrislusf/seaweedfs/issues/2156
2021-06-29 02:13:29 -07:00
Chris Lu
17477b37d5
sleep before re-reading the messages
...
If there are no more metadata changes and the client disconnects, it would go into a busy loop without this fix.
2021-06-27 06:31:04 -07:00
Chris Lu
cc7714fdbe
logging changes to debug
2021-06-27 05:54:16 -07:00
Chris Lu
c2e0a75c1f
adjust logs
2021-06-24 12:46:00 -07:00
Chris Lu
78b1fb921c
adjust log level
2021-06-23 20:59:54 -07:00
Chris Lu
f24bb9e688
mount: fix for deletion stopped working since 2.53
...
fix https://github.com/chrislusf/seaweedfs/issues/2138
due to 4d0cbd2700
2021-06-19 03:46:39 -07:00
Chris Lu
88d52adfdd
remove unused fields
2021-06-18 15:35:22 -07:00
Chris Lu
ed6aa13520
minor
2021-06-12 18:36:25 -07:00
Chris Lu
ee6c67682c
minor
2021-06-12 02:52:41 -07:00
Chris Lu
5e6dfbc25f
locks for data racing
2021-06-07 12:04:50 -07:00
Chris Lu
452c6ef183
limits concurrent uploads for one file
2021-06-06 23:05:17 -07:00
Chris Lu
8295e2feb6
skip md5 checking for now because of race condition
2021-06-06 22:16:32 -07:00
Chris Lu
19caeb7b02
fix writing the small file
2021-06-06 20:57:03 -07:00
Chris Lu
bb45dea15a
filer: parallel data upload
2021-06-06 20:23:36 -07:00
Chris Lu
44f1ba6894
refactor
2021-06-06 18:43:04 -07:00
Chris Lu
6c82326575
use bytes.Buffer to reduce memory allocation and gc
2021-06-06 13:42:36 -07:00
Chris Lu
9cba5cca0b
optionally disable concurrent upload limit
2021-06-06 13:13:33 -07:00
Chris Lu
ab606dec2a
filer: add path-specific option to enforce readonly
2021-06-04 01:03:41 -07:00
Chris Lu
fb8036385a
s3: save metadata during put-object
...
fix https://github.com/chrislusf/seaweedfs/issues/2092
2021-05-29 14:14:30 -07:00
Chris Lu
ac9bf71544
use jquery 3.6.0
2021-05-26 00:21:11 -07:00
Konstantin Lebedev
b612d5aebd
s3 test get w/ If-Match: bogus ETag
2021-05-24 16:59:44 +05:00
Konstantin Lebedev
69c768870b
- object write cache control
...
- object write expires
2021-05-24 15:43:55 +05:00
Konstantin Lebedev
84dce32a57
Merge branch 'master' into head_check_all_chunks
2021-05-24 12:28:19 +05:00
Chris Lu
064269bb57
filer: Support sqlite as filer meta store
2021-05-23 23:58:28 -07:00
Chris Lu
e5a2bf1287
s3: deprecating filer.options.buckets_fsync
2021-05-22 00:24:23 -07:00
Konstantin Lebedev
26a4f34a57
del checks
...
url err logging
stats chunk fetch
2021-05-21 15:59:12 +05:00
Konstantin Lebedev
03d1199d5f
Revert "revert PR #1903 avoid http error: superfluous response.WriteHeader"
...
This reverts commit ac71117e
2021-05-20 11:45:21 +05:00
Chris Lu
da0a4e775b
Revert "Revert "Merge pull request #2027 from bingoohuang/master""
...
This reverts commit d74cdf0115
2021-05-10 00:13:55 -07:00
Chris Lu
7ca75347ec
minor
2021-05-07 21:56:45 -07:00
Chris Lu
8f8738867f
add retry to assign volume
...
fix https://github.com/chrislusf/seaweedfs/issues/2056
2021-05-07 07:29:26 -07:00
Chris Lu
aaad4b578b
Merge pull request #2049 from qieqieplus/async-assign
...
ahead of time volume assignment
2021-05-06 10:00:25 -07:00
qieqieplus
c4d32f6937
ahead of time volume assignment
2021-05-06 18:55:44 +08:00
Chris Lu
c899bdf063
a little optimization
2021-05-06 03:03:00 -07:00
Chris Lu
3eb336e0b0
report error only for the first multipart upload
...
the glog.Errorf would always print for s3 multipart uploads
2021-05-06 01:57:54 -07:00
Chris Lu
ac71117ee6
revert PR #1903 avoid http error: superfluous response.WriteHeader
2021-05-05 15:11:39 -07:00
Chris Lu
24efa31e49
Merge pull request #2045 from qieqieplus/fix-vacuum-commit
2021-05-05 07:41:38 -07:00
qieqieplus
ac26080bd2
fix concurrent vacuum & delete panic
2021-05-05 17:54:50 +08:00
Chris Lu
e24ba2aadc
filer: delete specific tags
...
fix https://github.com/chrislusf/seaweedfs/issues/2041
2021-05-02 21:53:43 -07:00
Chris Lu
d74cdf0115
Revert "Merge pull request #2027 from bingoohuang/master"
...
Need to revert because docker image build failed. The docker apk package only has go 1.15.
2021-04-30 03:36:15 -07:00
Konstantin Lebedev
c2269123d3
fix aws style Etag for chunks
2021-04-28 22:28:05 +05:00
bingoohuang
7a9d27fce8
promote to go:embed instead of github.com/rakyll/statik
2021-04-27 17:22:24 +08:00
bingoohuang
cf552417a7
minFreeSpace refactored
2021-04-27 10:37:24 +08:00
bingoohuang
31f1cdeac2
minFreeSpace argument allows size like 10GiB
2021-04-26 18:48:34 +08:00
Chris Lu
ddc8643ee0
filer: directory listing adds namePatternExclude
...
fix https://github.com/chrislusf/seaweedfs/issues/2023
2021-04-24 11:49:03 -07:00
Chris Lu
f0ad172e80
shell: show which server holds the lock
...
fix https://github.com/chrislusf/seaweedfs/issues/1983
2021-04-22 23:56:35 -07:00
Chris Lu
11c120c040
master UI adds volume size limit
2021-04-22 14:22:48 -07:00
Chris Lu
6e5df901e4
adjust package names
2021-04-22 14:22:48 -07:00
Konstantin Lebedev
198688c717
revert volume etag
2021-04-16 23:22:31 +05:00
Chris Lu
1e033d45b8
simpler logic
...
related to https://github.com/chrislusf/seaweedfs/pull/1981
2021-04-12 12:04:53 -07:00
Chris Lu
52200a903b
Merge pull request #1981 from ueni-ltd/if-modified-since
...
Fix If-Modified-Since behavior
2021-04-12 12:02:54 -07:00
Chris Lu
519b0e1e49
filer: upload to a directory without "/" suffix
...
fix https://github.com/chrislusf/seaweedfs/issues/1988
2021-04-12 11:56:56 -07:00
Chris Lu
af313dff58
add gateway for easier POST and DELETE blobs
2021-04-10 23:47:47 -07:00
Chris Lu
0b82edc0d2
filer: avoid stuck uploader
...
fix https://github.com/chrislusf/seaweedfs/issues/1980
reverting the file upload batch executor
2021-04-09 13:05:15 -07:00
Merlin Gaillard
f952f979d1
filer: return 304 when If-Modified-Since == Last-Modified
2021-04-09 15:04:17 +02:00
Chris Lu
2327c0756b
fix to avoid loop
2021-04-05 23:24:26 -07:00
Chris Lu
c131764c34
ensure error is sent back in channel
...
fix https://github.com/chrislusf/seaweedfs/issues/1966
avoid shared readErr variable
2021-04-05 19:40:12 -07:00
Chris Lu
8251d1140e
refactor
2021-04-04 18:38:33 -07:00
Chris Lu
8e404a1433
go fmt
2021-04-02 02:22:26 -07:00
Chris Lu
67e019d54b
add missing changes
2021-04-02 01:11:44 -07:00
Chris Lu
7e8edc3c4a
refactoring
2021-04-02 01:10:24 -07:00
Chris Lu
cefe66f159
dedicated upload processor
...
avoid thundering effect of overloading volume servers
2021-04-01 02:21:40 -07:00
Chris Lu
b5880334fc
refactor
2021-03-30 21:07:34 -07:00
Chris Lu
e79e2ddeed
remove unused MoveEvents
2021-03-30 20:42:44 -07:00
Chris Lu
ac875976c0
filer, volume: add concurrent upload size limit to avoid OOM
...
add some back pressure when writes are slow
2021-03-30 02:10:53 -07:00
李海
06be5dc6c3
log snowflake sequencer nodeid's hex when start, it'll be in part of new assigned fid
2021-03-25 18:59:40 +08:00
李海
69b2dab9c6
add a snowflake sequencer as more robust fid generator, but less compressable than small auto-inc id
2021-03-25 18:49:26 +08:00
Chris Lu
a801332b0d
filer: return 409 if file conflicts with a directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1938
2021-03-24 01:03:11 -07:00
Chris Lu
5d931eff27
avoid possible nil
...
fix https://github.com/chrislusf/seaweedfs/issues/1928
The nil was because of `dn.Parent().UnlinkChildNode(dn.Id())` in topo.UnRegisterDataNode() function, when the dn leaves the cluster.
2021-03-22 13:24:07 -07:00
Chris Lu
f315eb2bb8
fix build error
2021-03-22 00:12:53 -07:00
Chris Lu
b465095db1
shell: add volume.check.disk to fix inconsistency for replicated volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/1923
2021-03-22 00:03:16 -07:00
Chris Lu
b1a86cf808
s3: copy object to itself
...
fix https://github.com/chrislusf/seaweedfs/issues/1922
2021-03-19 01:31:49 -07:00
Konstantin Lebedev
06da02739d
CheckAllChunkViews() for HEAD requests only
2021-03-16 14:15:17 +05:00
Konstantin Lebedev
90510e3137
If WriteHeader is not called explicitly, the first call to Write
...
will trigger an implicit WriteHeader(http.StatusOK).
WriteHeader: Only one header may be written. Go does not currently!
2021-03-15 21:22:59 +05:00
Konstantin Lebedev
46b9f5cff4
add debug logging
2021-03-15 19:33:33 +05:00
Chris Lu
5d446673a6
fix error printing
2021-03-10 17:13:04 -08:00
Chris Lu
df72dc206d
stats collect chunk upload retry count
2021-03-07 11:26:15 -08:00
Chris Lu
8363be8548
trim out trailing return character
2021-03-06 14:26:27 -08:00
Chris Lu
400de380f4
volume server: support tcp direct put/get/delete
2021-03-05 02:29:38 -08:00
Chris Lu
e982b06bcd
support IPv6
2021-03-02 20:59:39 -08:00
Chris Lu
543c8daa6d
Merge pull request #1844 from kmlebedev/processRangeRequest
2021-02-26 09:18:22 -08:00
Patrick Schmidt
5f7b024891
Show the real disk usage in stats calls
...
Currently the file size of only one volume location is taken into
account in the stats. This commit multiplies the disk usages by the
amount of nodes holding a replica of the volume.
This will yield the expected amount of disk usage and matches the
total size calculations from before.
2021-02-26 13:58:40 +01:00
Konstantin Lebedev
ee21c0042e
log error write entry
2021-02-26 14:18:01 +05:00
Chris Lu
7ba75e3d5a
filer: do not return no content for empty files
...
fix https://github.com/chrislusf/seaweedfs/issues/1831
fix https://github.com/chrislusf/seaweedfs/issues/1830
2021-02-23 12:25:27 -08:00
Chris Lu
30b30b8fe0
volume.tier.move: passing non-empty disk type
2021-02-22 01:59:03 -08:00
Chris Lu
73958e357d
add descriptive error if no free volumes
2021-02-18 19:10:20 -08:00
Chris Lu
c2ad6f1047
webdav add replication setting
...
fix https://github.com/chrislusf/seaweedfs/issues/1817
2021-02-18 12:15:09 -08:00
bingoohuang
352ac2f271
Merge remote-tracking branch 'origin/master'
2021-02-18 14:05:51 +08:00
bingoohuang
eab6e31d34
use backticks instead of double quotes to avoid escaped additionally in regex
2021-02-18 14:05:28 +08:00
Chris Lu
3575d41009
go fmt
2021-02-17 20:57:08 -08:00
Chris Lu
6daa932f5c
refactoring to get master function, instead of passing master values directly
...
this will enable retrying later
2021-02-17 20:55:55 -08:00
Chris Lu
aa17311063
s3: set filename when uploading file chunks
2021-02-17 20:54:53 -08:00
