Chris Lu
11c120c040
master UI adds volume size limit
2021-04-22 14:22:48 -07:00
Chris Lu
6e5df901e4
adjust package names
2021-04-22 14:22:48 -07:00
Konstantin Lebedev
198688c717
revert volume etag
2021-04-16 23:22:31 +05:00
Chris Lu
1e033d45b8
simpler logic
...
related to https://github.com/chrislusf/seaweedfs/pull/1981
2021-04-12 12:04:53 -07:00
Chris Lu
52200a903b
Merge pull request #1981 from ueni-ltd/if-modified-since
...
Fix If-Modified-Since behavior
2021-04-12 12:02:54 -07:00
Chris Lu
519b0e1e49
filer: upload to a directory without "/" suffix
...
fix https://github.com/chrislusf/seaweedfs/issues/1988
2021-04-12 11:56:56 -07:00
Chris Lu
af313dff58
add gateway for easier POST and DELETE blobs
2021-04-10 23:47:47 -07:00
Chris Lu
0b82edc0d2
filer: avoid stuck uploader
...
fix https://github.com/chrislusf/seaweedfs/issues/1980
reverting the file upload batch executor
2021-04-09 13:05:15 -07:00
Merlin Gaillard
f952f979d1
filer: return 304 when If-Modified-Since == Last-Modified
2021-04-09 15:04:17 +02:00
Chris Lu
2327c0756b
fix to avoid loop
2021-04-05 23:24:26 -07:00
Chris Lu
c131764c34
ensure error is sent back in channel
...
fix https://github.com/chrislusf/seaweedfs/issues/1966
avoid shared readErr variable
2021-04-05 19:40:12 -07:00
Chris Lu
8251d1140e
refactor
2021-04-04 18:38:33 -07:00
Chris Lu
8e404a1433
go fmt
2021-04-02 02:22:26 -07:00
Chris Lu
67e019d54b
add missing changes
2021-04-02 01:11:44 -07:00
Chris Lu
7e8edc3c4a
refactoring
2021-04-02 01:10:24 -07:00
Chris Lu
cefe66f159
dedicated upload processor
...
avoid thundering effect of overloading volume servers
2021-04-01 02:21:40 -07:00
Chris Lu
b5880334fc
refactor
2021-03-30 21:07:34 -07:00
Chris Lu
e79e2ddeed
remove unused MoveEvents
2021-03-30 20:42:44 -07:00
Chris Lu
ac875976c0
filer, volume: add concurrent upload size limit to avoid OOM
...
add some back pressure when writes are slow
2021-03-30 02:10:53 -07:00
李海
06be5dc6c3
log snowflake sequencer nodeid's hex when start, it'll be in part of new assigned fid
2021-03-25 18:59:40 +08:00
李海
69b2dab9c6
add a snowflake sequencer as more robust fid generator, but less compressable than small auto-inc id
2021-03-25 18:49:26 +08:00
Chris Lu
a801332b0d
filer: return 409 if file conflicts with a directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1938
2021-03-24 01:03:11 -07:00
Chris Lu
5d931eff27
avoid possible nil
...
fix https://github.com/chrislusf/seaweedfs/issues/1928
The nil was because of `dn.Parent().UnlinkChildNode(dn.Id())` in topo.UnRegisterDataNode() function, when the dn leaves the cluster.
2021-03-22 13:24:07 -07:00
Chris Lu
f315eb2bb8
fix build error
2021-03-22 00:12:53 -07:00
Chris Lu
b465095db1
shell: add volume.check.disk to fix inconsistency for replicated volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/1923
2021-03-22 00:03:16 -07:00
Chris Lu
b1a86cf808
s3: copy object to itself
...
fix https://github.com/chrislusf/seaweedfs/issues/1922
2021-03-19 01:31:49 -07:00
Konstantin Lebedev
06da02739d
CheckAllChunkViews() for HEAD requests only
2021-03-16 14:15:17 +05:00
Konstantin Lebedev
90510e3137
If WriteHeader is not called explicitly, the first call to Write
...
will trigger an implicit WriteHeader(http.StatusOK).
WriteHeader: Only one header may be written. Go does not currently!
2021-03-15 21:22:59 +05:00
Konstantin Lebedev
46b9f5cff4
add debug logging
2021-03-15 19:33:33 +05:00
Chris Lu
5d446673a6
fix error printing
2021-03-10 17:13:04 -08:00
Chris Lu
df72dc206d
stats collect chunk upload retry count
2021-03-07 11:26:15 -08:00
Chris Lu
8363be8548
trim out trailing return character
2021-03-06 14:26:27 -08:00
Chris Lu
400de380f4
volume server: support tcp direct put/get/delete
2021-03-05 02:29:38 -08:00
Chris Lu
e982b06bcd
support IPv6
2021-03-02 20:59:39 -08:00
Chris Lu
543c8daa6d
Merge pull request #1844 from kmlebedev/processRangeRequest
2021-02-26 09:18:22 -08:00
Patrick Schmidt
5f7b024891
Show the real disk usage in stats calls
...
Currently the file size of only one volume location is taken into
account in the stats. This commit multiplies the disk usages by the
amount of nodes holding a replica of the volume.
This will yield the expected amount of disk usage and matches the
total size calculations from before.
2021-02-26 13:58:40 +01:00
Konstantin Lebedev
ee21c0042e
log error write entry
2021-02-26 14:18:01 +05:00
Chris Lu
7ba75e3d5a
filer: do not return no content for empty files
...
fix https://github.com/chrislusf/seaweedfs/issues/1831
fix https://github.com/chrislusf/seaweedfs/issues/1830
2021-02-23 12:25:27 -08:00
Chris Lu
30b30b8fe0
volume.tier.move: passing non-empty disk type
2021-02-22 01:59:03 -08:00
Chris Lu
73958e357d
add descriptive error if no free volumes
2021-02-18 19:10:20 -08:00
Chris Lu
c2ad6f1047
webdav add replication setting
...
fix https://github.com/chrislusf/seaweedfs/issues/1817
2021-02-18 12:15:09 -08:00
bingoohuang
352ac2f271
Merge remote-tracking branch 'origin/master'
2021-02-18 14:05:51 +08:00
bingoohuang
eab6e31d34
use backticks instead of double quotes to avoid escaped additionally in regex
2021-02-18 14:05:28 +08:00
Chris Lu
3575d41009
go fmt
2021-02-17 20:57:08 -08:00
Chris Lu
6daa932f5c
refactoring to get master function, instead of passing master values directly
...
this will enable retrying later
2021-02-17 20:55:55 -08:00
Chris Lu
aa17311063
s3: set filename when uploading file chunks
2021-02-17 20:54:53 -08:00
Chris Lu
cd866664a8
skip JWT if fileId is empty
...
related to https://github.com/chrislusf/seaweedfs/issues/1808
2021-02-16 15:39:12 -08:00
Chris Lu
3fe628f04e
use hdd instead of empty string
2021-02-16 03:03:00 -08:00
Chris Lu
f8446b42ab
this can compile now!!!
2021-02-16 02:47:02 -08:00
Chris Lu
7ce647f27e
support customizable disk type
2021-02-13 15:42:42 -08:00
Chris Lu
4bd8a692d8
disk type can be generic tags
2021-02-13 13:50:14 -08:00
Chris Lu
487e435679
adjust http max idle connections per host
...
related to https://github.com/chrislusf/seaweedfs/issues/1802
2021-02-12 03:47:15 -08:00
Chris Lu
770393a48c
volume: add capability to change disk type when moving a volume
2021-02-09 23:58:08 -08:00
Chris Lu
821c46edf1
Merge branch 'master' into support_ssd_volume
2021-02-09 11:37:07 -08:00
bingoohuang
7256902fb0
fix typo offset.ToAcutalOffset to offset.ToActualOffset
2021-02-07 12:11:51 +08:00
bingoohuang
94ea3bd3a5
renaming NeedleMapType to NeedleMapKind
2021-02-07 09:00:03 +08:00
Chris Lu
a331bbb3ae
filer: should return 204 on DELETE to nonexistent file
...
related to
https://github.com/chrislusf/seaweedfs/issues/1776
https://github.com/chrislusf/seaweedfs/issues/1160
2021-02-03 00:40:31 -08:00
Chris Lu
d67ccb66c0
webdav: can start together with "weed server" or "weed filer"
2021-01-31 22:16:52 -08:00
Chris Lu
76430790b9
webdav: cache to version specific folder
2021-01-31 22:02:03 -08:00
Chris Lu
d475c89fcc
go fmt
2021-01-28 15:23:46 -08:00
Chris Lu
990fa69bfe
add back AdjustedUrl() related code
2021-01-28 14:36:29 -08:00
Chris Lu
00707ec00f
mount: outsideContainerClusterMode proxy through filer
...
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
2021-01-24 19:01:58 -08:00
Chris Lu
6ca10725b8
Revert "mount: when outside cluster network, use filer as proxy to access volume servers"
...
This reverts commit 096e088d7b
.
2021-01-24 03:15:19 -08:00
Chris Lu
096e088d7b
mount: when outside cluster network, use filer as proxy to access volume servers
2021-01-24 01:41:38 -08:00
Chris Lu
80b8692688
filer.sync: replicate outside of either cluster, only need to see filers
2021-01-24 00:01:44 -08:00
Chris Lu
f8dbb03bdd
filer: append operation returns final file size
2021-01-23 04:19:49 -08:00
Chris Lu
759482e2ac
filer: proxy all http methods to volume servers
2021-01-23 03:52:38 -08:00
Chris Lu
937cfacc01
filer: add "proxyToFileId" to reverse proxy to a volume server
2021-01-23 03:43:48 -08:00
Chris Lu
711c3f3939
filer: fix http status 206 setting
...
fix https://github.com/chrislusf/seaweedfs/issues/1753
2021-01-22 00:26:30 -08:00
Chris Lu
f98817cfe6
filer: support appending to a file
2021-01-20 13:40:32 -08:00
Chris Lu
1b8e3da295
refactoring
2021-01-20 12:59:54 -08:00
Chris Lu
d5add83e85
filer store: add postgres2
2021-01-19 18:07:29 -08:00
Chris Lu
52a8f1470e
filer store: add mysql2
2021-01-19 17:21:50 -08:00
Chris Lu
a4063a5437
add stream list directory entries
2021-01-15 23:56:24 -08:00
Chris Lu
f002e668de
change limit to int64 in case of overflow
2021-01-14 23:10:37 -08:00
Chris Lu
698f58f7c4
filer, s3: add http status 206 as late as possible
...
fix https://github.com/chrislusf/seaweedfs/issues/1746
2021-01-14 02:59:40 -08:00
Chris Lu
0ef43a23a7
go fmt
2021-01-12 02:30:12 -08:00
Chris Lu
629c9962e7
filer: add leveldb3
...
support dedicated leveldb instance for each bucket
2021-01-12 02:29:44 -08:00
Chris Lu
394513f598
filer: ensure seamless meta data updates
2021-01-11 02:08:55 -08:00
Chris Lu
1efb51ba84
filer: change to saveToFilerLimit from cacheToFilerLimit
...
short circuit saving small files to volume server
2021-01-10 23:14:46 -08:00
henry
b42c5b71c1
Seaweed custom header are not visible to Vue or javascript
2021-01-05 13:01:29 +08:00
henry
1309dfe612
Seaweed custom header are not visible to Vue or javascript
2021-01-05 12:58:46 +08:00
henry
97a94eddab
Merge branch 'master' of https://github.com/fuyouyshengwu/seaweedfs
2021-01-04 14:50:01 +08:00
Chris Lu
f4d87f8b9d
works
...
performance seems about the same as leveldb for insert entries
2021-01-02 23:32:58 -08:00
Chris Lu
2c3c2c27d7
separate prefix from namePattern
...
fix https://github.com/chrislusf/seaweedfs/issues/1722
2021-01-01 20:23:23 -08:00
Peter Cai
cd2a9865ed
filer: respect Content-Type on PUT
...
This is needed for the S3 gateway to conform to the S3 protocol -- the
server should always respect the Content-Type set by client.
2020-12-31 11:32:45 +08:00
henry
14ddd15508
1 Distinguish between folders and common files, folders are listed first
...
2 Add some custom headers that need to be exposed
2020-12-31 00:03:22 +08:00
Chris Lu
2e6cf541e7
filer: avoid deletion error when item is not found
2020-12-28 00:18:37 -08:00
henry
7458ff9523
Solve the Chinese name disorder of download file
2020-12-28 11:28:33 +08:00
Chris Lu
0a7c5f85a9
filer: add namePattern to search in current folder
2020-12-26 15:05:31 -08:00
Chris Lu
0a067944cc
filer: add retries during volume moving
...
fix https://github.com/chrislusf/seaweedfs/issues/1704
2020-12-25 02:32:55 -08:00
Chris Lu
c3d1b3b5aa
hook up
2020-12-23 21:49:01 -08:00
Chris Lu
1c7e1295dc
Merge branch 'master' into support_ssd_volume
2020-12-22 17:44:52 -08:00
Chris Lu
a1009e8044
filer: add -defaultStoreDir so that filer.toml can be skipped
...
fix https://github.com/chrislusf/seaweedfs/issues/1659
2020-12-22 17:33:40 -08:00
Chris Lu
f696a2b2a7
assign volumes based on disk type
2020-12-17 13:25:05 -08:00
Chris Lu
1bf22c0b5b
go fmt
2020-12-16 09:14:05 -08:00
Chris Lu
1d88865869
passing disk type along
2020-12-16 09:10:14 -08:00
Chris Lu
23014b6810
Merge branch 'master' into support_ssd_volume
2020-12-16 08:26:51 -08:00
Chris Lu
b789767cd1
filer: return http status 499 for client closed connection
...
fix https://github.com/chrislusf/seaweedfs/issues/1684
2020-12-16 08:18:00 -08:00
Chris Lu
2e8dba571b
adjust volume server UI
2020-12-14 00:51:57 -08:00
Chris Lu
bf5873022d
Merge branch 'master' into support_ssd_volume
2020-12-14 00:32:17 -08:00
Chris Lu
b7e3ca9172
Merge pull request #1650 from kmlebedev/masterServerEnableUI
...
security master use access.ui
2020-12-14 00:31:51 -08:00
Konstantin Lebedev
c4459249ef
ui interface is always on
2020-12-14 13:24:41 +05:00
Chris Lu
94525aa0fd
allocate volume by disk type
2020-12-13 23:08:21 -08:00
Chris Lu
f6a419c26c
disk type configurable for each folder
2020-12-13 22:49:56 -08:00
Chris Lu
7c52a35942
refactor
2020-12-13 22:29:52 -08:00
Chris Lu
a9db24cd05
master allocate volumes if ssd type runs out
2020-12-13 19:44:57 -08:00
Chris Lu
8baba93fce
rename parameter name to "disk"
2020-12-13 12:06:21 -08:00
Chris Lu
51eadaf2b6
rename parameter name to "disk"
2020-12-13 12:05:31 -08:00
Chris Lu
0d2ec832e2
rename from volumeType to diskType
2020-12-13 11:59:32 -08:00
Chris Lu
d156c74ec0
volume server set volume type and heartbeat to the master
2020-12-13 03:11:24 -08:00
Chris Lu
e9cd798bd3
adding volume type
2020-12-13 00:58:58 -08:00
Chris Lu
16cd6fb278
fix variable name
2020-12-12 21:19:40 -08:00
Chris Lu
3fedfec1e7
check cross device rename error
2020-12-10 23:50:32 -08:00
Chris Lu
83078ac6ce
filer: change to /etc/seaweedfs folder on filer
...
fix https://github.com/chrislusf/seaweedfs/issues/1666
2020-12-10 11:11:02 -08:00
Chris Lu
765b3ef1be
save /etc/iam/identity.json inside filer store
2020-12-10 00:15:22 -08:00
Chris Lu
606051cc79
filer: cache "/etc" filder small files to filer store
2020-12-07 00:11:35 -08:00
Chris Lu
ac22f1dd26
filer subscribe: handle rename subscription
2020-12-07 00:10:49 -08:00
Chris Lu
263eb29e9f
filer: add option to cache small files to filer store
2020-12-04 22:39:43 -08:00
Konstantin Lebedev
eadbba5c95
security master use access.ui
2020-12-04 12:57:59 +05:00
Chris Lu
45dba088cd
fix tests
2020-12-01 19:37:21 -08:00
Chris Lu
04062c56c7
webdav: improve webdav upload speed
2020-12-01 15:32:27 -08:00
Chris Lu
f4abd01adf
filer: cache small file to filer store
2020-11-30 04:34:04 -08:00
Chris Lu
a9c6be5fc3
set creation time
2020-11-30 03:11:52 -08:00
Chris Lu
141ce67c09
close http request body
2020-11-30 02:45:00 -08:00
Chris Lu
965413c21b
shell: add volume.vacuum command
2020-11-28 23:18:02 -08:00
Chris Lu
c57a7839ec
adjust comments
2020-11-28 03:01:03 -08:00
Chris Lu
8438fb0cc3
fix ec shard spreading error
2020-11-28 02:21:16 -08:00
Chris Lu
85554bea38
filer: readonly handle static resources
...
fix https://github.com/chrislusf/seaweedfs/issues/1641
2020-11-27 13:42:14 -08:00
Chris Lu
6d30b21b10
volume: add "-dir.idx" option for separate index storage
...
fix https://github.com/chrislusf/seaweedfs/issues/1265
2020-11-27 03:17:10 -08:00
Chris Lu
b11449f955
pass in dir.idx parameter to DiskLocation
2020-11-26 14:59:03 -08:00
Chris Lu
0da7ecfd29
go fmt
2020-11-26 11:22:30 -08:00
Chris Lu
92f906b6fc
remove zstd
...
fix https://github.com/chrislusf/seaweedfs/issues/1629
2020-11-21 13:06:45 -08:00
Chris Lu
dc304342b2
fs.configure: configurable volume growth
2020-11-17 01:00:02 -08:00
Chris Lu
6204dc6e85
add comment
2020-11-16 16:57:31 -08:00
Chris Lu
ef3e068c51
deprecating "filer.options.buckets_fsync" in filer.toml
2020-11-16 16:56:49 -08:00
Chris Lu
0ea5c087ce
go fmt
2020-11-15 16:59:28 -08:00
Chris Lu
95c0de285d
refactoring
2020-11-15 16:58:48 -08:00
Chris Lu
500bcab953
refactoring
2020-11-15 14:41:56 -08:00
Chris Lu
590f02179d
filer: load filer conf when starting
2020-11-15 14:06:03 -08:00
Konstantin Lebedev
1eec5c8d5d
gen pb
2020-11-12 04:10:06 +05:00
Konstantin Lebedev
fc7baef5bb
fiil serverUrls sorted by data center
2020-11-12 02:13:33 +05:00
Konstantin Lebedev
dc26012a3b
initial
2020-11-11 15:03:47 +05:00
Chris Lu
6856b0d57e
filer: add API to add/modify/delete tagging
2020-11-09 01:00:07 -08:00
Chris Lu
0dafcf1f5a
volume: detect max volume count changes based on disk usage
...
fix https://github.com/chrislusf/seaweedfs/issues/1594
2020-11-03 14:43:17 -08:00
Chris Lu
d29b787220
store together with "Seaweed-" prefix
2020-11-03 00:21:10 -08:00
Chris Lu
5448781dfc
filer: Headers with "Seaweed-" prefix are stored as extended properties
2020-11-03 00:15:51 -08:00
Chris Lu
de86945aeb
go fmt
2020-10-31 16:45:38 -07:00
Chris Lu
306062b4e7
filer: add CORS support
2020-10-31 16:44:03 -07:00
Chris Lu
05acc33294
volume: add CORS support
2020-10-31 16:31:39 -07:00
Konstantin Lebedev
d2a4ffdd60
glog
2020-10-31 03:21:54 +05:00
Konstantin Lebedev
46303c36bf
When the volume server is stopped, the master server immediately sees the deletion of volumes
2020-10-31 02:48:25 +05:00
Chris Lu
eb32af25d4
make space for the QR code
2020-10-30 10:55:31 -07:00
ruitao.liu
22a9ea0512
adjust s3 header file.
2020-10-29 16:05:40 +08:00
ruitao.liu
b917be7955
S3 bucket list, response with uploaded storageclass.
2020-10-29 14:57:19 +08:00
ruitao.liu
2dcc178d0a
save s3 metadata to filer.
2020-10-28 18:16:05 +08:00
Chris Lu
53c3aad875
volume: add a note file to avoid incomplete volume files
...
fix https://github.com/chrislusf/seaweedfs/issues/1567
2020-10-27 15:56:49 -07:00
Chris Lu
24bf142596
copy large file first
2020-10-27 15:31:15 -07:00
ruitao.liu
72f6b77dea
fix tag prefix.
2020-10-27 18:01:37 +08:00
ruitao.liu
8766ca1b95
rename s3 tag prefix.
2020-10-27 17:33:24 +08:00
ruitao.liu
19026ae55d
return x-amz-tag-count header when GET object.
2020-10-27 16:49:31 +08:00
Chris Lu
e219c57849
passing full path when assign volume locations
2020-10-25 15:46:29 -07:00
Chris Lu
f375b93aef
renaming
2020-10-25 15:32:43 -07:00
Chris Lu
e9d40b80b2
less verbose logs
2020-10-24 09:42:54 -07:00
Chris Lu
fb124f2d4a
conditionally display volume server public url link
2020-10-23 23:58:46 -07:00
Chris Lu
1b17f71939
adjust election timeout to 10 seconds
2020-10-23 23:06:44 -07:00
Chris Lu
fc689319ae
add volume public url
2020-10-23 18:03:23 -07:00
Chris Lu
6c92fa349d
filer: support default rack
...
fix https://github.com/chrislusf/seaweedfs/issues/1546
2020-10-20 17:41:39 -07:00
Chris Lu
69f0da5d73
mount: pass along replication and collection parameters
2020-10-19 00:12:42 -07:00
Chris Lu
37f165d743
webdav: return io.EOF when at end of a file
...
fix https://github.com/chrislusf/seaweedfs/issues/1344
2020-10-16 12:53:37 -07:00
Chris Lu
ace0ea3d28
s3: avoid duplicated bucket
2020-10-15 10:52:20 -07:00
Chris Lu
f022aff289
add back http.StatusPartialContent
...
revert e7c04af1d0
2020-10-12 12:26:25 -07:00
Chris Lu
723ae11db4
refactoring in order to adjust volume server url later
2020-10-11 20:15:10 -07:00
Chris Lu
9b0e8ef026
filer: added QR code to transfer files with mobile devices
2020-10-10 18:00:21 -07:00
Chris Lu
e1a8e3f900
fix logging
2020-10-08 18:40:38 -07:00
Chris Lu
bd8b9b0c2e
some changes to prepare fix after failed reads
2020-10-08 18:33:06 -07:00
Chris Lu
ec08a1670b
volume: avoid hanging when stopping a volume server
2020-10-07 23:48:24 -07:00
Chris Lu
62d3d3aea0
go fmt
2020-10-07 23:30:54 -07:00
Chris Lu
a8624c2e4f
read from alternative replica
...
related to https://github.com/chrislusf/seaweedfs/issues/1512
2020-10-07 22:49:04 -07:00
Chris Lu
da4edf3651
master: check peers for existing leader before starting a leader election
...
fix https://github.com/chrislusf/seaweedfs/issues/1509
2020-10-07 01:25:39 -07:00
Konstantin Lebedev
e4f2d9eb4a
We return etag using the same algorithm as aws s3
...
https://teppen.io/2018/06/23/aws_s3_etags/
2020-10-05 14:43:32 +05:00
Устюжанин Антон Александрович
702b1cb876
fix: remove deleted peers if resumeState = true
2020-10-04 21:56:17 +05:00
Устюжанин Антон Александрович
dc31b19469
fix: restore raft state
2020-10-03 14:03:41 +05:00
Устюжанин Антон Александрович
8c82fb7e5f
fix: restore raft state
2020-10-02 23:01:20 +05:00
Chris Lu
55cb68c09c
filer: replication follows master config if not specified
2020-09-30 09:15:55 -07:00
Chris Lu
31fc7bb2e1
refactor
...
adjust for faster test
2020-09-27 10:41:29 -07:00
Chris Lu
41be7a4c8f
filer: upload also set file size
...
ensure works same as fuse mount
2020-09-25 12:02:06 -07:00
Chris Lu
1012df7bb5
switch hardlink id from int64 to bytes
2020-09-24 11:11:42 -07:00
Chris Lu
4856bce0ee
adjust for metrics port
2020-09-24 10:21:23 -07:00
Chris Lu
2e9099369e
Merge pull request #1489 from kmlebedev/promhttp
...
Promhttp
2020-09-24 09:50:48 -07:00
Konstantin Lebedev
324e44d4b3
add start metrics server
2020-09-24 17:45:39 +05:00
Chris Lu
5e239afdfc
hardlink works now
2020-09-24 03:06:48 -07:00
James Hartig
91e4eca1e9
Fix deadlock with KeepConnected and SendHeartbeat
...
There's the potential where we're writing to a clientConn and it goes away
and we're stuck keeping a read lock on clientChansLock. This causes
KeepConnected to not be able to remove the client since it requires a write
lock on clientChansLock. This ends up backing up SendHeartbeat because it
can't get a read lock.
2020-09-21 22:41:38 -04:00
Chris Lu
63373a9f9f
filer: add file extended properties
2020-09-21 11:08:34 -07:00
Chris Lu
9a3b564508
adjust printout
2020-09-20 16:01:56 -07:00
Chris Lu
62563a895a
refactoring
2020-09-20 16:00:01 -07:00
Chris Lu
d013d09a9b
adjust logging
2020-09-20 15:38:59 -07:00
Chris Lu
41d508edfd
go fmt
2020-09-19 14:10:26 -07:00
Chris Lu
2c21eb1971
volume: get metrics configuration from master
...
fix https://github.com/chrislusf/seaweedfs/issues/1354
2020-09-19 00:03:00 -07:00
Chris Lu
cb427d48fa
filer report metrics configuration
2020-09-17 06:46:51 -07:00
Chris Lu
e4e0234998
refactoring
2020-09-17 06:43:54 -07:00
Chris Lu
e861a6a3ab
simplify metrics configuration logic
2020-09-16 01:39:30 -07:00
Chris Lu
be54eeb364
passing value by checkWithMaster() instead
2020-09-16 01:33:45 -07:00
Chris Lu
4d21de63ee
go fmt
2020-09-16 01:27:24 -07:00
Chris Lu
913a16268d
volume: load configuration from master at the start
...
fix https://github.com/chrislusf/seaweedfs/issues/1469
2020-09-16 01:27:05 -07:00
Chris Lu
c9202c4b3d
add storage backend to GetMasterConfigurationResponse
2020-09-16 01:12:22 -07:00
Chris Lu
8a0710cb73
handle more than 2GB files
...
related to https://github.com/chrislusf/seaweedfs/issues/1468
2020-09-16 00:41:50 -07:00
Chris Lu
854007bc98
minor
2020-09-16 00:37:57 -07:00
Chris Lu
ada996fe55
filer: support createing empty folder
...
fix https://github.com/chrislusf/seaweedfs/issues/1161
2020-09-14 13:07:46 -07:00
Chris Lu
5d6753fb98
shell: add volumeServer.leave command
2020-09-13 21:25:51 -07:00
Chris Lu
cc5fe6f5ee
handle special characters in html link
2020-09-10 19:46:02 -07:00
Chris Lu
b183ae54b5
master: changing 301 to 308 on redirect to volume servers
...
fix https://github.com/chrislusf/seaweedfs/issues/1454
2020-09-09 22:36:10 -07:00
Chris Lu
daf0a449f7
properly cancel context for streaming grpc
2020-09-09 12:07:15 -07:00
Chris Lu
387ab6796f
filer: cross cluster synchronization
2020-09-09 11:21:23 -07:00
Chris Lu
5e13bc878c
adjust log level
2020-09-09 03:49:26 -07:00
Chris Lu
8acd714600
apply meta changes only if store is different
2020-09-05 23:25:03 -07:00
Chris Lu
bba90ff3c8
read filer signature
2020-09-05 22:52:15 -07:00
ruitao.liu
798280e980
change filer2 to filer.
2020-09-03 17:05:26 +08:00
ruitao.liu
a93d27d1e8
new filer option to es v7.
2020-09-03 16:34:58 +08:00
James Hartig
8e54e34576
volume: Don't unmount before deleting volume in copy
...
If we unmount first and then delete, the delete fails because the volume
was unmounted. Delete ends up doing the same thing as the unmount anyways.
2020-09-01 22:00:07 -04:00
Chris Lu
eb7929a971
rename filer2 to filer
2020-09-01 00:21:19 -07:00
Chris Lu
97733c3dff
typo
2020-08-31 10:49:17 -07:00
Chris Lu
f0c89cfacd
go fmt
2020-08-31 10:23:31 -07:00
Chris Lu
edb9d65e05
Merge pull request #1431 from kmlebedev/wip-prefix-search
...
Wip prefix search
2020-08-31 10:22:14 -07:00
Chris Lu
a41588279a
change log level 5 to 4
2020-08-30 20:12:04 -07:00
Chris Lu
56244fb9a1
fix hard coded host address
2020-08-30 10:23:35 -07:00
Chris Lu
99ecf63276
go fmt
2020-08-29 22:28:33 -07:00
Chris Lu
ca658a97c5
add signatures to messages to avoid double processing
2020-08-28 23:48:48 -07:00
Konstantin Lebedev
464d4c82ec
stop send heartbeat before stop volume server
2020-08-26 16:16:11 +05:00
Chris Lu
1901f15cd2
volume server: remove whitelist for status checking
...
Volume liveness/readiness not work if jwt enabled, I try use param
-whiteList
but jwt breaks
2020-08-25 02:27:47 -07:00
Chris Lu
51346a5930
always keep the manifest list of chunks
2020-08-23 17:15:12 -07:00
James Hartig
3ccfa4c6ad
Added VolumeMarkWritable and VolumeStatus grpc methods
...
This is necessary for copy to mark as read-only and then restore the
original state afterwards.
2020-08-19 11:42:56 -04:00
Chris Lu
a78772d5ea
avoid shutdown in the middle of running
2020-08-19 00:42:02 -07:00
Chris Lu
c026eb0592
refactoring
2020-08-18 17:39:29 -07:00
Chris Lu
7e91ae592c
pass in option to read deleted entries
...
not working yet
2020-08-18 17:37:26 -07:00
Chris Lu
6a92f0bc7a
refactoring to typed Size
...
Go is amazing with refactoring!
2020-08-18 17:04:28 -07:00
Chris Lu
be4d42b8e2
rename
2020-08-17 20:15:53 -07:00
Chris Lu
aec7f32b02
fix reader_at
2020-08-16 00:49:08 -07:00
Chris Lu
c647deace1
file size support set file length
...
use Attr.FileSize and TotalChunkSize to determine file size
2020-08-15 09:32:47 -07:00
Chris Lu
3b1a95ac26
filer refactoring: same auto chunking logic for POST and PUT, no size limit
2020-08-08 12:02:06 -07:00
Chris Lu
67348e7b15
less noisy heartbeat logs
2020-08-08 10:53:35 -07:00
Chris Lu
bee0d7e5eb
lower log priority for noisy heartbeat
2020-08-08 10:52:13 -07:00
Chris Lu
ae00cce4bd
support POST and PUT auto chunking
2020-08-08 10:45:37 -07:00
Chris Lu
bd8bfdae07
refactoring
2020-08-08 10:18:43 -07:00
Chris Lu
20e2ac1add
filer: store md5 metadata for files uploaded by filer
...
fix https://github.com/chrislusf/seaweedfs/issues/1412
2020-08-06 10:04:17 -07:00
Chris Lu
93ea0801ea
volume: the variable for the master node may be stale?
...
related to https://github.com/chrislusf/seaweedfs/issues/1414
2020-08-06 09:48:54 -07:00
Chris Lu
4ecfa9879d
volume: report Content-MD5 in response header
2020-08-06 05:22:53 -07:00
Устюжанин Антон Александрович
2d3b355fb6
test ListDirectoryPrefixedEntries
2020-08-05 23:56:06 +05:00
Устюжанин Антон Александрович
33a9e5e2d1
test ListDirectoryPrefixedEntries
2020-08-05 22:19:16 +05:00
Chris Lu
49199c69d4
volume: avoid special logic for .gz files
2020-08-01 11:19:57 -07:00
Chris Lu
3faa0aa448
rename
2020-07-25 20:09:19 -07:00
Chris Lu
023a1efdf2
set filename in Content-Disposition header
2020-07-25 20:06:40 -07:00
James Hartig
229f11c660
Added VolumeNeedleStatus volume server grpc method
...
This is needed for the diffing tool to get the cookie for a needle
2020-07-22 15:02:21 -04:00
Chris Lu
97d97f3528
go code can read and write chunk manifest
2020-07-19 17:59:43 -07:00
李远军
4162766c46
Fix the bug of " http://xx/xx,xxx.PNG?width=100 "
...
If you want to do a resize a UPPER ext pic file, there is a bug caused by commit 308688c8d0
(also see https://github.com/chrislusf/seaweedfs/blob/master/weed/images/resizing.go#L43 )
2020-07-17 19:57:35 +08:00
Chris Lu
ddec7b2bb9
go fmt
2020-07-14 07:34:16 -07:00
Chris Lu
4f6096c7f0
add reading from persisted logs for local filer store
2020-07-13 22:55:28 -07:00
Chris Lu
1dd2c76532
refactoring
2020-07-12 17:31:24 -07:00
Chris Lu
e7c04af1d0
skip http.StatusPartialContent in case of error and superfluous response.WriteHeader
2020-07-10 08:56:23 -07:00
李远军
6608cb5f43
Revert "remove fixJpgOrientation"
...
This reverts commit de5ca9b2
2020-07-10 10:08:36 +08:00
Chris Lu
a2eb680f34
filer: default filer store directory
...
1. set default filer store directory
2. set peers, avoiding empty string counted as 1.
2020-07-07 23:06:48 -07:00
Chris Lu
3c269da37f
fix compilation
2020-07-06 09:46:24 -07:00
Chris Lu
d198e8c6d6
filer: support cross filer meta data sync if sharing the same store
2020-07-06 09:24:54 -07:00
Chris Lu
0b8cc9b626
aggregate multiple filer metadata chagne events
2020-07-05 23:05:02 -07:00
Chris Lu
49929e0869
rename
2020-07-05 15:52:36 -07:00
Chris Lu
70d8a3a1d3
add SubscribeLocalMetadata without checking persisted meta logs
2020-07-05 15:50:07 -07:00
Chris Lu
55e40b08fc
refactoring
2020-07-05 15:43:06 -07:00
Chris Lu
881e0fde2e
rename
2020-07-05 08:55:09 -07:00
Chris Lu
4d1484628a
refactoring
2020-07-03 16:41:30 -07:00
Chris Lu
31e23e9783
filer: support active<=>active filer replication
2020-06-30 22:53:57 -07:00
Chris Lu
df75b5c98d
passing ctx for metadata logging
2020-06-28 14:34:51 -07:00
Chris Lu
a4dc25cd22
volume: generate ec shard first, before generating the ec index file.
...
possible fix https://github.com/chrislusf/seaweedfs/issues/1364
the ec index file is used to determine whether the ec files are ready or not
2020-06-25 11:06:58 -07:00
Chris Lu
3b638d3994
add more ec encoding logging
2020-06-25 09:43:38 -07:00
Chris Lu
fe60db404a
prepare for zstd
2020-06-23 09:12:02 -07:00
Chris Lu
a4e3cffe0a
S3: DeleteObject API also support deleting a folder recursively
2020-06-22 10:01:00 -07:00
Chris Lu
de5ca9b258
remove fixJpgOrientation
2020-06-20 12:50:40 -07:00
Chris Lu
3080c197e3
rename UnCompressData to DecompressData
2020-06-20 08:16:16 -07:00
Chris Lu
ca3516ac6d
adjust protoc
2020-06-20 08:00:25 -07:00
Chris Lu
e912fd15e3
renaming
2020-06-19 22:45:27 -07:00
Chris Lu
16fe132a20
detect mime type before replicating to other volume servers
2020-06-19 22:11:36 -07:00
Chris Lu
044841c885
master: always clear previous master meta data directory
2020-06-19 20:42:16 -07:00
Chris Lu
ae1994cbc1
erasure coding: fix cases where there are no .ecj files
2020-06-18 09:52:35 -07:00
Chris Lu
c66f357715
report error correctly
2020-06-10 10:59:25 -07:00
Chris Lu
c38066be89
also add it for webdav
2020-06-09 18:07:03 -07:00
Evgenii Kozlov
2410ffbd79
Name fix
2020-06-05 19:00:38 +03:00
Evgenii Kozlov
0e0db70f55
Set volumes ReadOnly if low free disk space
2020-06-05 18:18:15 +03:00
Chris Lu
393ab07c7c
disable pulse seconds
...
there are a few other hard coded timing settings, which will cause disconnected volume servers.
2020-06-04 10:52:01 -07:00
Chris Lu
bc2ec6774d
inject git version into build
2020-06-02 00:10:38 -07:00
bingoohuang
1a642b9876
add Volume Ids column only for max 100 volumes for convenience in the master ui.
2020-05-29 15:37:58 +08:00
bingoohuang
accb4964b7
util.BytesToHumanReadable
2020-05-29 10:00:07 +08:00
bingoohuang
21d0a013d5
rename 'needle' which collides with imported package name
2020-05-28 17:00:00 +08:00
bingoohuang
41cab5d69f
log JSON response if httpStatus >= 400
2020-05-28 08:56:55 +08:00
Chris Lu
d4235afe4d
support multiple locks
2020-05-25 18:39:53 -07:00
bingoohuang
ecdeef8c66
simplify func(w http.ResponseWriter, r *http.Request) to http.HandlerFunc
2020-05-25 21:00:12 +08:00
Chris Lu
91b91d6cb7
add error to avoid copying not found volume
...
fix https://github.com/chrislusf/seaweedfs/issues/1317
2020-05-17 20:20:12 -07:00
Chris Lu
3fbf635308
Merge branch 'master' into msg_channel
2020-05-13 03:54:15 -07:00
Chris Lu
3a5e4769cf
prevent empty locations returned
...
fix https://github.com/chrislusf/seaweedfs/issues/1313
2020-05-13 03:46:38 -07:00
Chris Lu
dfccc3c263
able to read chan and write chan
2020-05-08 02:47:22 -07:00
Chris Lu
1e3e4b3072
add broker connects to filer
2020-05-05 02:05:28 -07:00
Chris Lu
871efa4fc1
refactoring
...
some previous chunk etag was using md5, which should be wrong.
2020-04-30 17:20:47 -07:00