Konstantin Lebedev
5021bea698
GetUserPolicy
2021-04-10 23:57:45 +05:00
Chris Lu
0b82edc0d2
filer: avoid stuck uploader
...
fix https://github.com/chrislusf/seaweedfs/issues/1980
reverting the file upload batch executor
2021-04-09 13:05:15 -07:00
Chris Lu
93f4146ffa
properly release the view cache
2021-04-09 12:36:39 -07:00
Merlin Gaillard
f952f979d1
filer: return 304 when If-Modified-Since == Last-Modified
2021-04-09 15:04:17 +02:00
Merlin Gaillard
4d4acc715e
s3api: handle 304 response code from filer
2021-04-09 12:13:19 +02:00
Chris Lu
6deb647a8f
mount: fix possible memory leak
...
if many files are read repeatedly, their metadata are accumulated in memory. This fix cleared the metadata after the file is read.
2021-04-08 19:47:31 -07:00
Konstantin Lebedev
ba175f81b5
add auth aws signV4
2021-04-08 17:40:47 +05:00
Konstantin Lebedev
995ae91007
add DeleteUserPolicy
2021-04-08 11:16:36 +05:00
Philippe Pepiot
42a761ee20
Fix typo in weed filer long help
2021-04-07 22:47:23 +02:00
Chris Lu
3be061994f
skip connection reset error
...
fix https://github.com/chrislusf/seaweedfs/issues/1971
this is because the connections are pooled but the volume server has reset the connection
2021-04-07 00:54:13 -07:00
Chris Lu
c5b08bac1b
remove mac specific mount options
2021-04-07 00:54:13 -07:00
Konstantin Lebedev
f5f8eec8e2
fix get filerGrpcAddress
2021-04-06 13:53:56 +05:00
Konstantin Lebedev
011e6e90ee
Merge branch 'upstreamMaster' into iamapipr
2021-04-06 13:50:33 +05:00
Konstantin Lebedev
ed79baa30f
add tests
2021-04-06 13:43:08 +05:00
Chris Lu
2327c0756b
fix to avoid loop
2021-04-05 23:24:26 -07:00
Chris Lu
a37eca78cd
2.38
2021-04-05 19:41:54 -07:00
Chris Lu
c131764c34
ensure error is sent back in channel
...
fix https://github.com/chrislusf/seaweedfs/issues/1966
avoid shared readErr variable
2021-04-05 19:40:12 -07:00
Chris Lu
0f64f5b9c8
mount: add readOnly option
...
fix https://github.com/chrislusf/seaweedfs/issues/1961
2021-04-04 21:40:58 -07:00
Chris Lu
6eee200c13
2.37
2021-04-04 18:45:48 -07:00
Chris Lu
fbb82a5c9c
skip limiting if limit is zero
2021-04-04 18:38:33 -07:00
Chris Lu
8251d1140e
refactor
2021-04-04 18:38:33 -07:00
Chris Lu
5dca98fe71
fix typo
2021-04-03 11:37:13 -07:00
Chris Lu
5e64f65632
ensure tables are created
...
fix https://github.com/chrislusf/seaweedfs/issues/1957
2021-04-02 12:02:26 -07:00
Chris Lu
319743d330
go fmt
2021-04-02 02:22:59 -07:00
Chris Lu
8e404a1433
go fmt
2021-04-02 02:22:26 -07:00
Chris Lu
bdf2ddddfd
revert to same implementation as before
...
This reverts commit 7e8edc3c4a
.
2021-04-02 02:21:38 -07:00
Chris Lu
67e019d54b
add missing changes
2021-04-02 01:11:44 -07:00
Chris Lu
7e8edc3c4a
refactoring
2021-04-02 01:10:24 -07:00
Chris Lu
cc0df36a9e
smaller file chunks are faster
2021-04-01 02:21:59 -07:00
Chris Lu
cefe66f159
dedicated upload processor
...
avoid thundering effect of overloading volume servers
2021-04-01 02:21:40 -07:00
Chris Lu
1f984d2645
refactor buffer pool
2021-04-01 02:20:00 -07:00
Chris Lu
b5880334fc
refactor
2021-03-30 21:07:34 -07:00
Chris Lu
e79e2ddeed
remove unused MoveEvents
2021-03-30 20:42:44 -07:00
Chris Lu
e06807a55b
adjust logs
2021-03-30 20:36:06 -07:00
Chris Lu
ac875976c0
filer, volume: add concurrent upload size limit to avoid OOM
...
add some back pressure when writes are slow
2021-03-30 02:10:53 -07:00
Chris Lu
a1e18a1384
server add memory profiling
2021-03-30 02:10:53 -07:00
LazyDBA247-Anyvision
9385e2224d
fix postgres2 upsertQuery syntax also
2021-03-30 10:51:52 +03:00
Chris Lu
a5662c9e5e
adjust ordering
2021-03-30 00:26:57 -07:00
Chris Lu
bec74c3e6a
Merge pull request #1948 from LazyDBA247-Anyvision/master
...
add enableUpsert=true
2021-03-30 00:25:01 -07:00
LazyDBA247-Anyvision
9f1cab179c
use a compatible pg syntax for upsert
2021-03-30 10:21:27 +03:00
Chris Lu
a95929e53c
reduce default concurrentWriters to 32
2021-03-30 00:17:52 -07:00
LazyDBA247-Anyvision
7f44d953b5
fix GetBool
2021-03-30 01:36:02 +03:00
LazyDBA247-Anyvision
4c51e6a660
add enableUpsert=true
...
and rename config to upsertQuery
2021-03-30 00:32:03 +03:00
Chris Lu
b61e9f6a1a
adjust ordering
2021-03-29 00:49:50 -07:00
Chris Lu
1ebdbad222
Merge pull request #1946 from LazyDBA247-Anyvision/master
...
Adding Customisation - insertQuery for postgres/mysql
2021-03-29 00:42:48 -07:00
Konstantin Lebedev
8a95f9c10c
iam GetUser
2021-03-29 12:01:44 +05:00
LazyDBA247-Anyvision
96c62bd34d
scaffold - insertQuery
...
Example for upsert query for mysql & postgres variants
2021-03-29 10:01:12 +03:00
LazyDBA247-Anyvision
4a02389eb0
Adding custom insertQuery support for postgres/2 mysql/2
2021-03-29 09:58:13 +03:00
Chris Lu
6b7aa9633f
2.36
2021-03-28 19:09:06 -07:00
Chris Lu
bd7471d877
refactor
2021-03-25 12:05:59 -07:00
李海
06be5dc6c3
log snowflake sequencer nodeid's hex when start, it'll be in part of new assigned fid
2021-03-25 18:59:40 +08:00
李海
69b2dab9c6
add a snowflake sequencer as more robust fid generator, but less compressable than small auto-inc id
2021-03-25 18:49:26 +08:00
Konstantin Lebedev
40938d6a47
SaveInsideFiler S3 Configuration
2021-03-25 12:34:11 +05:00
Konstantin Lebedev
9f26f2815c
SaveAs S3 Configuration
2021-03-25 12:34:11 +05:00
Konstantin Lebedev
82b0463fac
handler PutUserPolicy
...
https://docs.aws.amazon.com/IAM/latest/APIReference/API_PutUserPolicy.html
2021-03-25 12:34:11 +05:00
Konstantin Lebedev
d7719d0542
base handlers
2021-03-25 12:34:11 +05:00
Konstantin Lebedev
03c7953254
init Iam Api Server
2021-03-25 12:34:11 +05:00
Chris Lu
82dfe06066
Merge pull request #1933 from kmlebedev/missingKeysThreshold
...
add missingKeysThreshold for fsck
2021-03-24 10:56:25 -07:00
Konstantin Lebedev
020a5d40c3
avoid counter is null
2021-03-24 22:51:22 +05:00
Konstantin Lebedev
df6cf0a2fa
nonRepairThreshold
2021-03-24 22:07:13 +05:00
Konstantin Lebedev
9ce82a9cd2
GOMAXPROCS use defaults to the value of runtime.NumCPU
...
https://github.com/chrislusf/seaweedfs/issues/1942
2021-03-24 18:45:58 +05:00
Konstantin Lebedev
60972f1c97
rename option repairThreshold
2021-03-24 13:24:49 +05:00
Chris Lu
a801332b0d
filer: return 409 if file conflicts with a directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1938
2021-03-24 01:03:11 -07:00
Chris Lu
c42b95c596
more help message
...
fix https://github.com/chrislusf/seaweedfs/issues/1937
2021-03-23 17:27:57 -07:00
Konstantin Lebedev
40a9e88c07
rename var
2021-03-23 21:42:35 +05:00
Konstantin Lebedev
d21a5bf5d0
add missingKeysThreshold for fsck
2021-03-23 15:04:07 +05:00
Chris Lu
85cff10787
mysql: ensure name column is case sensitive
...
fix https://github.com/chrislusf/seaweedfs/issues/1931
2021-03-23 00:46:50 -07:00
Chris Lu
4abb511db3
make a local copy of the in memory cached data
2021-03-22 22:33:07 -07:00
Chris Lu
3cbc40fa48
avoid creating multiple reader
2021-03-22 22:32:47 -07:00
Chris Lu
2f7c7afdec
refactor: remove unused parameter
2021-03-22 22:13:19 -07:00
Chris Lu
d48dd0c738
use []byte directly instead of bytes.Buffer
2021-03-22 22:12:57 -07:00
Chris Lu
8eb14967ef
shell: volume.check.disk adds optional slow mode
...
fix https://github.com/chrislusf/seaweedfs/issues/1930
2021-03-22 21:01:43 -07:00
Chris Lu
288369cfc7
mount: release resources when Forget() is called
...
address https://github.com/chrislusf/seaweedfs/issues/1929
2021-03-22 16:22:19 -07:00
Chris Lu
5d931eff27
avoid possible nil
...
fix https://github.com/chrislusf/seaweedfs/issues/1928
The nil was because of `dn.Parent().UnlinkChildNode(dn.Id())` in topo.UnRegisterDataNode() function, when the dn leaves the cluster.
2021-03-22 13:24:07 -07:00
Chris Lu
f315eb2bb8
fix build error
2021-03-22 00:12:53 -07:00
Chris Lu
1dd5bc134c
2.35
2021-03-22 00:05:09 -07:00
Chris Lu
b465095db1
shell: add volume.check.disk to fix inconsistency for replicated volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/1923
2021-03-22 00:03:16 -07:00
Chris Lu
df461402cc
ensure entry attribute is not nill
...
fix https://github.com/chrislusf/seaweedfs/issues/1926
2021-03-21 22:27:30 -07:00
Chris Lu
102a951377
refactor, split into 2 files
2021-03-21 13:05:53 -07:00
Chris Lu
f6e2566ee5
better logs for cookie mismatch
2021-03-20 12:02:57 -07:00
Chris Lu
27e24162f2
minor
2021-03-20 06:37:03 -07:00
Chris Lu
41cc6e661b
fs.mv ensures there are 2 arguments
2021-03-20 06:34:13 -07:00
Chris Lu
b2c1c209a5
refactor
2021-03-20 06:34:13 -07:00
Chris Lu
e9b1853d63
go fmt
2021-03-19 01:31:56 -07:00
Chris Lu
b1a86cf808
s3: copy object to itself
...
fix https://github.com/chrislusf/seaweedfs/issues/1922
2021-03-19 01:31:49 -07:00
Konstantin Lebedev
bf94149920
add line
2021-03-17 23:41:34 +05:00
Konstantin Lebedev
c5705e7a35
dir + slash key does not exist
...
https://github.com/chrislusf/seaweedfs/issues/1917
2021-03-17 23:40:42 +05:00
Konstantin Lebedev
eb54993a4e
Merge branch 'upstreamMaster' into check_chunkviews_mr
...
# Conflicts:
# weed/filer/filechunk_manifest.go
# weed/filer/stream.go
# weed/replication/repl_util/replication_util.go
# weed/util/fasthttp_util.go
2021-03-16 15:29:49 +05:00
Chris Lu
9672f9e1b2
2.34
2021-03-16 03:01:15 -07:00
Chris Lu
69694a17be
reverting 7d57664c2d
2021-03-16 02:59:26 -07:00
Chris Lu
c00dd5e62e
report error if entry not found
2021-03-16 02:55:49 -07:00
Konstantin Lebedev
06da02739d
CheckAllChunkViews() for HEAD requests only
2021-03-16 14:15:17 +05:00
Chris Lu
91a3ac9731
2.33
2021-03-16 00:36:06 -07:00
Chris Lu
4b1ed227d1
revert fasthttp changes
...
related to https://github.com/chrislusf/seaweedfs/issues/1907
2021-03-16 00:33:14 -07:00
Chris Lu
10164d0386
Merge pull request #1905 from wuh-fnst/s3ListCounterErr
...
s3: count correction for the number of files in the dir
2021-03-15 20:54:58 -07:00
Chris Lu
22a07a12d2
[mount] [regression] filer.path
option results in empty mount in 2.32
...
fix https://github.com/chrislusf/seaweedfs/issues/1906
2021-03-15 20:46:25 -07:00
wuh-fnst
050a7f9599
s3: count correction for the number of files in the dir
2021-03-16 09:22:03 +08:00
Konstantin Lebedev
3a3699867b
Status PartialContent for Content-Range response
2021-03-15 23:30:22 +05:00
Konstantin Lebedev
90510e3137
If WriteHeader is not called explicitly, the first call to Write
...
will trigger an implicit WriteHeader(http.StatusOK).
WriteHeader: Only one header may be written. Go does not currently!
2021-03-15 21:22:59 +05:00
Konstantin Lebedev
46b9f5cff4
add debug logging
2021-03-15 19:33:33 +05:00
Konstantin Lebedev
7194a5e7bf
avoid http error: superfluous response.WriteHeader
...
https://github.com/chrislusf/seaweedfs/issues/1838
2021-03-15 18:52:59 +05:00
Chris Lu
9f00f95bfb
2.32
2021-03-14 21:32:00 -07:00
Chris Lu
a4cfffc264
shell: fix moving volume, volume server evacuate
...
fix https://github.com/chrislusf/seaweedfs/issues/1534
2021-03-14 21:29:55 -07:00
Chris Lu
ca100568f7
fix parsing
2021-03-14 20:50:14 -07:00
Chris Lu
352ba23f83
revert previous change
...
revert 29e62aba00
2021-03-14 20:49:56 -07:00
Chris Lu
29e62aba00
possible fix for volume balance
...
address https://github.com/chrislusf/seaweedfs/issues/1534
2021-03-14 20:23:19 -07:00
Chris Lu
35d939a1d2
go fmt
2021-03-14 13:21:02 -07:00
Chris Lu
2d4c2db81d
filer: leveldb, rocksdb auto create store directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1901
2021-03-14 13:20:14 -07:00
Chris Lu
6d3a96eb56
filer: mysql2, postgres2 trigger actions on bucket creation and deletion
...
fix https://github.com/chrislusf/seaweedfs/issues/1877
2021-03-13 22:07:39 -08:00
Chris Lu
cb423312a4
prevent nil volume message
2021-03-13 11:05:29 -08:00
Chris Lu
972327f966
prevent nil volume nm
2021-03-13 11:04:51 -08:00
Chris Lu
2a68ddb963
default to empty host to bind to tcp4 and tcp6
2021-03-12 15:32:51 -08:00
Chris Lu
5fbcaaf37a
fullpath() test is not needed
2021-03-12 09:45:25 -08:00
Chris Lu
7d57664c2d
mount: internals switch to filer.Entry instead of protobuf
2021-03-12 00:36:38 -08:00
wuh-fnst
4c1d945e46
make List correctly judge whether it is the last file
2021-03-12 11:22:26 +08:00
Chris Lu
cca66c7fbe
print out the url
2021-03-11 18:39:45 -08:00
Chris Lu
7f887c78d2
fix
2021-03-11 18:18:16 -08:00
Chris Lu
fcc52d1e4e
shell: add s3.clean.uploads to clean up stale multipart uploads
...
fix https://github.com/chrislusf/seaweedfs/issues/1855
2021-03-11 18:05:24 -08:00
Chris Lu
40dc5ac904
mount: add a bit retry when connecting to filer during startup
2021-03-11 16:36:43 -08:00
Chris Lu
aba47fd9e8
reduce repeated calls to dir.FullPath()
2021-03-11 14:08:20 -08:00
Chris Lu
d084334ffd
Merge pull request #1883 from kmlebedev/passDelNonEmptyFolder
...
s3 delete-objects pass OK for fail to delete non-empty folder
2021-03-11 11:01:44 -08:00
Chris Lu
0e35836cc1
Merge pull request #1890 from kmlebedev/ZeroDataSize
...
error if read chunk zero data size
2021-03-11 10:51:48 -08:00
Konstantin Lebedev
58cdcc6d6e
error if urls not found
2021-03-11 23:34:36 +05:00
Chris Lu
79280e1513
escape special characters
...
fix https://github.com/chrislusf/seaweedfs/issues/1884
2021-03-11 09:49:40 -08:00
Konstantin Lebedev
4db21012c1
error if read chunk zero data size
2021-03-11 22:38:59 +05:00
wuheng
828fbf3fb7
s3: "isLast" returns true when the file does not exist
2021-03-11 15:20:50 +08:00
Chris Lu
5d446673a6
fix error printing
2021-03-10 17:13:04 -08:00
Chris Lu
ca546f47e1
s3: escape object key if containing special characters
...
fix https://github.com/chrislusf/seaweedfs/issues/1884
2021-03-10 13:19:28 -08:00
Chris Lu
105fcd7bfc
Merge pull request #1872 from kmlebedev/grpc_auth
...
TLS allowed CommonNames
2021-03-10 11:17:49 -08:00
Konstantin Lebedev
bb3af2d70b
no directoriesWithDeletion
2021-03-10 19:52:41 +05:00
Konstantin Lebedev
15b018da34
s3 delete-objects pass OK for fail to delete non-empty folder
2021-03-10 19:41:35 +05:00
Konstantin Lebedev
348e21a08c
add comments
2021-03-10 14:42:39 +05:00
Konstantin Lebedev
831953c55c
allowed wildcard domain
2021-03-10 14:02:13 +05:00
Konstantin Lebedev
4bf93d6e63
comma-separated
2021-03-10 12:43:13 +05:00
Konstantin Lebedev
0e02f7e258
comma-separated SSL certificate common names
2021-03-10 12:42:44 +05:00
Chris Lu
737bde5ab7
refactoring, remove duplicated code
2021-03-09 23:23:01 -08:00
Chris Lu
5ba4b479f8
properly lock file.entry object
...
fix https://github.com/chrislusf/seaweedfs/issues/1882
2021-03-09 23:08:38 -08:00
Chris Lu
5be83dd4a9
add comments
2021-03-09 14:13:48 -08:00
Chris Lu
9d8ca513d9
refactor
2021-03-09 13:21:26 -08:00
Chris Lu
f5b5e4da2d
2.31
2021-03-09 12:52:16 -08:00
Chris Lu
828f6e9f4d
volume: auto add missing vif files
...
fix https://github.com/chrislusf/seaweedfs/issues/1878
2021-03-09 12:09:32 -08:00
Chris Lu
387c6f4218
Revert "delete the folder object in multi_object_delete"
...
This reverts commit 2e89c8c9ae
.
2021-03-09 10:07:27 -08:00
qieqieplus
5b16820924
fix: collection stats won't update if all volumes expired at same iteration
2021-03-09 18:49:45 +08:00
Konstantin Lebedev
190fada1ef
TLS allowed commonNames
2021-03-08 21:39:44 +05:00
Konstantin Lebedev
c6d3735605
permitCommonNames
...
https://github.com/chrislusf/seaweedfs/issues/1841
https://jbrandhorst.com/post/grpc-auth/
2021-03-08 13:16:17 +05:00
Chris Lu
d888e9a90f
2.30
2021-03-07 15:01:19 -08:00
Chris Lu
60686a0bb7
make tcp optional
2021-03-07 14:45:36 -08:00
Chris Lu
3b2a9c98ef
fix nil
2021-03-07 11:45:35 -08:00
Chris Lu
df72dc206d
stats collect chunk upload retry count
2021-03-07 11:26:15 -08:00
Chris Lu
726edab054
avoid nil when closing an index
...
fix https://github.com/chrislusf/seaweedfs/issues/1870
2021-03-07 11:03:09 -08:00
Chris Lu
bdfed16d42
avoid nil exception
...
fix https://github.com/chrislusf/seaweedfs/issues/1869
2021-03-07 01:49:06 -08:00
Chris Lu
1bd880dcdb
adds tcp writes benchmark
2021-03-06 14:26:27 -08:00
Chris Lu
9d402ebe9f
refactoring
2021-03-06 14:26:27 -08:00
Chris Lu
e9eddfb8a2
remove wrong writes
2021-03-06 14:26:27 -08:00
Chris Lu
8363be8548
trim out trailing return character
2021-03-06 14:26:27 -08:00
Chris Lu
38fc200e56
CRCWriter consistent with CRC
2021-03-06 14:26:27 -08:00
Chris Lu
1444e9d275
migrated multi host connection pool from godropbox package
...
removing unneeded dependencies, which involved etcd versions.
2021-03-06 14:26:27 -08:00
Patrick Schmidt
7413d59750
Fix EC shard count logic
...
This fixes the calculation of the amount of EC shards a node holds.
Previously a global counter was increased, but also used inside the
loop to apply disk usage deltas. This led to wrong absolute numbers.
The fix is to apply only deltas of single EC shards per iteration.
2021-03-05 12:50:58 +01:00
Chris Lu
400de380f4
volume server: support tcp direct put/get/delete
2021-03-05 02:29:38 -08:00
Chris Lu
2e89c8c9ae
delete the folder object in multi_object_delete
...
fix https://github.com/chrislusf/seaweedfs/issues/1857
2021-03-04 14:03:40 -08:00
Chris Lu
c5cb346a08
fix compilation error
2021-03-04 12:36:20 -08:00
Chris Lu
f5cc96f289
skip checking master peers if not starting master
2021-03-04 12:31:26 -08:00
Chris Lu
40ff30b83f
adjust help message
2021-03-03 02:12:46 -08:00
Chris Lu
c0842fe99f
add filer.meta.backup command
2021-03-03 02:02:29 -08:00
Chris Lu
0c5c51eb98
adjust help message
2021-03-02 23:07:39 -08:00
Chris Lu
bd727b7b53
go fmt
2021-03-02 20:59:56 -08:00
Chris Lu
e982b06bcd
support IPv6
2021-03-02 20:59:39 -08:00
Chris Lu
bcf32591b7
reduce possibility of nil entry
2021-03-02 13:33:56 -08:00
Chris Lu
be9c7c21ec
reduce possibility of nil file entry
2021-03-02 11:17:17 -08:00
Chris Lu
02e146f3e1
return err
2021-03-02 08:54:18 -08:00
Chris Lu
5511722420
s3: list permission is needed to list my buckets
...
fix https://github.com/chrislusf/seaweedfs/issues/1837
2021-03-01 12:41:55 -08:00
Chris Lu
c46d16b2f3
skip network error when remote volumes start
2021-03-01 01:20:06 -08:00
Chris Lu
6e43e8ce17
add logs for remote file loading
2021-03-01 00:48:59 -08:00
Chris Lu
015d16f43f
add vif file versions in case loading superblock fails
2021-03-01 00:48:30 -08:00
Chris Lu
d680676d45
skip already loaded backends
2021-03-01 00:47:03 -08:00
Chris Lu
540441fd38
go fmt
2021-02-28 20:34:14 -08:00
Chris Lu
c9722dceb2
adjust log level
2021-02-28 20:26:55 -08:00
Chris Lu
1b1c018165
adjust text
2021-02-28 19:02:43 -08:00
Chris Lu
2cca07b44c
cloud tier to non-AWS s3 gateways
2021-02-28 18:59:09 -08:00
Chris Lu
f29ae0db76
2.29
2021-02-28 18:08:26 -08:00
bingoohuang
5cdff56731
set default env prefix to WEED_
2021-03-01 09:35:23 +08:00
Chris Lu
e52c94640e
filer.backup: added to replace filer.replicate
2021-02-28 16:22:27 -08:00
Chris Lu
014a31d11a
minor
2021-02-28 16:21:09 -08:00
Chris Lu
f2fcb77808
local file sink: create backup files with permission 0755
2021-02-28 16:20:47 -08:00
Chris Lu
984fdd6192
always use non bucket prefixing url
2021-02-28 16:20:13 -08:00
Chris Lu
9abb041763
filer source: support filerProxy mode
2021-02-28 16:19:47 -08:00
Chris Lu
678c54d705
data sink: add incremental mode
2021-02-28 16:19:03 -08:00
Chris Lu
4ff2c5c4c9
rename file
2021-02-28 16:14:21 -08:00
Chris Lu
3b76a51f5f
add help message
2021-02-27 12:15:49 -08:00
Chris Lu
70434df105
use "options"
2021-02-27 12:12:53 -08:00
Chris Lu
d3c31c69a7
avoid confusion: conf and config are too similar
2021-02-27 12:08:09 -08:00
Chris Lu
543c8daa6d
Merge pull request #1844 from kmlebedev/processRangeRequest
2021-02-26 09:18:22 -08:00
Patrick Schmidt
5f7b024891
Show the real disk usage in stats calls
...
Currently the file size of only one volume location is taken into
account in the stats. This commit multiplies the disk usages by the
amount of nodes holding a replica of the volume.
This will yield the expected amount of disk usage and matches the
total size calculations from before.
2021-02-26 13:58:40 +01:00
Konstantin Lebedev
ee21c0042e
log error write entry
2021-02-26 14:18:01 +05:00
Chris Lu
7ba75e3d5a
filer: do not return no content for empty files
...
fix https://github.com/chrislusf/seaweedfs/issues/1831
fix https://github.com/chrislusf/seaweedfs/issues/1830
2021-02-23 12:25:27 -08:00
Chris Lu
9edd964627
volume.tier.move: avoid repeated move for replicated volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/1792#issuecomment-784139348
2021-02-23 03:49:14 -08:00
Chris Lu
37f104f88f
2.28
2021-02-22 22:54:34 -08:00
Chris Lu
90cdf9dcac
avoid conflict with "weed scaffold -config=xxx"
2021-02-22 16:57:27 -08:00
Chris Lu
72b0d9d8c4
avoid unnecessary user home checking
2021-02-22 16:50:56 -08:00
Chris Lu
2270737344
volume: avoid fixed vacuum timeout for large volumes
...
1GB for 3 minutes, about 5.7MB/s
2021-02-22 12:52:37 -08:00
Chris Lu
44bdfb2d15
filer: avoid encryption and compression at the same time
...
fix https://github.com/chrislusf/seaweedfs/issues/1828
2021-02-22 12:22:49 -08:00
Chris Lu
62191b08ea
disk type support custom tags
2021-02-22 02:03:12 -08:00
Chris Lu
30b30b8fe0
volume.tier.move: passing non-empty disk type
2021-02-22 01:59:03 -08:00
Chris Lu
5da63e045e
avoid moving to another server with the same volume id
2021-02-22 01:44:18 -08:00
Chris Lu
6a4546d2c0
shell: add volume.tier.move
2021-02-22 01:30:07 -08:00
Chris Lu
1c233ad986
refactoring
2021-02-22 00:28:42 -08:00
Chris Lu
151c281f36
2.27
2021-02-21 19:29:27 -08:00
Chris Lu
258e93bc86
Merge pull request #1823 from bingoohuang/master
...
feature: support command line arguments in a configuration file
2021-02-20 22:35:18 -08:00
Chris Lu
03c643aa6b
fix test
...
fileSize need to be divided by 8
2021-02-20 21:19:21 -08:00
Chris Lu
7635f6b9fa
disk file avoid file.Stat()
2021-02-20 20:06:06 -08:00
bingoo
1af6c96cd0
fix log help
2021-02-21 10:46:01 +08:00
Chris Lu
a2383b3b12
Revert "Revert "avoid file.Stat()""
...
This reverts commit 7ef4c24f18
.
2021-02-20 12:42:09 -08:00
Chris Lu
7ef4c24f18
Revert "avoid file.Stat()"
...
This reverts commit 98c93ca465
.
2021-02-20 12:39:33 -08:00
Chris Lu
1ad3200094
skip seek() when index file writes
2021-02-20 12:39:25 -08:00
Chris Lu
98c93ca465
avoid file.Stat()
...
avoid one Syscall, but did not help on performance though
2021-02-20 09:44:17 -08:00
bingoohuang
ee082ae402
revert comment temporarily
2021-02-20 16:59:38 +08:00
bingoohuang
ee7cdf3668
revert ErrXyz to ErrorXyz temporarily.
2021-02-20 16:58:08 +08:00
bingoohuang
50df484d86
support command line arguments in a configuration file so we can weed master -config=master.conf
, weed volume -config=volume.conf
and etc.
2021-02-20 16:52:57 +08:00
bingoohuang
f69356f589
Merge pull request #3 from chrislusf/master
2021-02-20 16:45:02 +08:00
Chris Lu
565f7a6e72
Update data_node.go
2021-02-19 14:22:36 -08:00
Chris Lu
a37473ae60
add back volume ids
...
address https://github.com/chrislusf/seaweedfs/issues/1792#issuecomment-782339576
2021-02-19 14:22:12 -08:00
Chris Lu
a1210d1e8d
Revert "volume: avoid file.stat(), file.seek() if possible during writes"
...
This reverts commit c78409a598
.
2021-02-19 03:56:27 -08:00
Chris Lu
b961cd6208
add WIP message
2021-02-19 03:39:19 -08:00
Chris Lu
f465d63b5d
grpc should fail when heart beating to master
...
fix https://github.com/chrislusf/seaweedfs/issues/1820
2021-02-19 03:37:29 -08:00
Chris Lu
c576ad04ac
fix volume server display for volumes
2021-02-19 01:38:56 -08:00
bingoohuang
7ffe736d20
renaming to ErrXyz
2021-02-19 15:59:55 +08:00
Chris Lu
c78409a598
volume: avoid file.stat(), file.seek() if possible during writes
2021-02-18 23:22:54 -08:00
Chris Lu
29575dadc5
remove extra files
2021-02-18 19:14:58 -08:00
Chris Lu
73958e357d
add descriptive error if no free volumes
2021-02-18 19:10:20 -08:00
Chris Lu
776f497469
filer: fs.configure should try to read from entry.content also
...
related to https://github.com/chrislusf/seaweedfs/issues/1792
2021-02-18 17:07:02 -08:00
Chris Lu
e1992c83de
webdav adds disktype
2021-02-18 15:45:44 -08:00
Chris Lu
c2ad6f1047
webdav add replication setting
...
fix https://github.com/chrislusf/seaweedfs/issues/1817
2021-02-18 12:15:09 -08:00
Chris Lu
816ef0ed30
add more help message
...
fix https://github.com/chrislusf/seaweedfs/issues/1816
2021-02-18 10:34:49 -08:00
bingoohuang
352ac2f271
Merge remote-tracking branch 'origin/master'
2021-02-18 14:05:51 +08:00
bingoohuang
eab6e31d34
use backticks instead of double quotes to avoid escaped additionally in regex
2021-02-18 14:05:28 +08:00
Chris Lu
3575d41009
go fmt
2021-02-17 20:57:08 -08:00
Chris Lu
6daa932f5c
refactoring to get master function, instead of passing master values directly
...
this will enable retrying later
2021-02-17 20:55:55 -08:00
Chris Lu
dd9f3a0104
add sleep between upload retries
2021-02-17 20:55:11 -08:00
Chris Lu
aa17311063
s3: set filename when uploading file chunks
2021-02-17 20:54:53 -08:00
Chris Lu
3f8b0da677
filer: do not print password on error
...
fix https://github.com/chrislusf/seaweedfs/issues/1809
2021-02-17 02:13:52 -08:00
Chris Lu
cd866664a8
skip JWT if fileId is empty
...
related to https://github.com/chrislusf/seaweedfs/issues/1808
2021-02-16 15:39:12 -08:00
Chris Lu
632c94d438
Merge branch 'extend_to_disk_type'
2021-02-16 11:06:13 -08:00
Chris Lu
0611233f16
adjust printing
2021-02-16 10:55:30 -08:00
Chris Lu
68775d29e3
fix tests
2021-02-16 10:51:03 -08:00
Chris Lu
7403cd43c5
only use "" for hdd to avoid two values for the same thing
2021-02-16 10:50:42 -08:00
Chris Lu
b314d78e97
fix print
2021-02-16 10:48:28 -08:00
Chris Lu
53ca7e66ef
avoid dead lock
2021-02-16 10:48:16 -08:00
Chris Lu
38efc6f572
simplify
2021-02-16 10:47:32 -08:00
Chris Lu
ad70107eb4
ec balance among nodes with HDD
2021-02-16 06:22:50 -08:00
Chris Lu
3097b9a9b7
fix existence checking
2021-02-16 05:59:43 -08:00
Chris Lu
ebf320ec22
avoid divided by zero
2021-02-16 05:59:24 -08:00
Chris Lu
a0c6db361c
avoid nil
2021-02-16 05:33:38 -08:00
Chris Lu
36f95e50a9
avoid possible nil disk info
2021-02-16 05:13:48 -08:00
Chris Lu
43101ccea0
move to the empty nodes first
2021-02-16 04:27:16 -08:00
Chris Lu
38bbef7ec1
avoid nil
2021-02-16 04:16:46 -08:00
Chris Lu
cb9cc29518
volume.list display; fix updating maxVolumeCount for disk
2021-02-16 03:55:24 -08:00
Chris Lu
3fe628f04e
use hdd instead of empty string
2021-02-16 03:03:00 -08:00
Chris Lu
f8446b42ab
this can compile now!!!
2021-02-16 02:47:02 -08:00
Chris Lu
b9b5b932c5
filer: postgres2 mysql2 avoid repeatedly creating tables
2021-02-15 16:19:24 -08:00
Chris Lu
71f0c19515
2.26
2021-02-15 13:38:38 -08:00
Chris Lu
8b169aa47c
s3: Critical - S3 Delete Multiple Objects, delete the bucket
...
fix https://github.com/chrislusf/seaweedfs/issues/1806
2021-02-15 13:38:29 -08:00
Chris Lu
50202d1391
Merge pull request #1805 from LazyDBA247-Anyvision/master
...
better postgresql connection pool management
2021-02-14 22:26:29 -08:00
Chris Lu
8726ef34c1
2.25
2021-02-14 22:23:43 -08:00
LazyDBA247-Anyvision
7f458d5e78
better postgres connection pool management
...
adding SetConnMaxLifetime configuration (https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime )
to enable refresh of stale connections.
2021-02-15 07:45:09 +02:00
Chris Lu
0bc3a1f9e8
disk type only supports hdd and ssd, not ready for random tags yet
2021-02-14 11:38:43 -08:00
LazyDBA247-Anyvision
51b4963e2e
postgres2 & memsql2
...
add escape (quote identifiers) for the dynamic sql
so tables (collections) with special characters will work.
2021-02-14 13:14:36 +02:00
Chris Lu
876dbe3d26
collect disk types
2021-02-13 23:25:16 -08:00
Chris Lu
861f8fab9b
refactoring
2021-02-13 22:34:12 -08:00
Chris Lu
881f05f5a0
adjust help message
2021-02-13 16:13:52 -08:00
Chris Lu
ef76365ec2
adjust help message
2021-02-13 15:47:08 -08:00
Chris Lu
7ce647f27e
support customizable disk type
2021-02-13 15:42:42 -08:00
Chris Lu
4bd8a692d8
disk type can be generic tags
2021-02-13 13:50:14 -08:00
Chris Lu
712b3e9e53
mount: release need to avoid nil reader
...
fix https://github.com/chrislusf/seaweedfs/issues/1803
2021-02-13 13:39:37 -08:00
Chris Lu
4ce56bac08
fix format error
2021-02-12 06:44:26 -08:00
Chris Lu
0604afcda7
mount: remove unnecessary flush on release
2021-02-12 06:01:04 -08:00
Chris Lu
7d9dc3c6a2
use fasthttp lib to read
2021-02-12 05:59:52 -08:00
Chris Lu
487e435679
adjust http max idle connections per host
...
related to https://github.com/chrislusf/seaweedfs/issues/1802
2021-02-12 03:47:15 -08:00
Chris Lu
0f426ce34d
Merge pull request #1801 from kmlebedev/recoveringRabbitMQ
...
Do reconnect to RabbitMQ
2021-02-11 01:05:03 -08:00
Konstantin Lebedev
94eac4f00e
Do reconnect to RabbitMQ
...
https://github.com/google/go-cloud/issues/2958
https://github.com/chrislusf/seaweedfs/issues/1773
2021-02-11 13:59:36 +05:00
Chris Lu
885ca34748
volume: fail fast if idx files are missing
...
fix https://github.com/chrislusf/seaweedfs/issues/1796
2021-02-11 00:44:40 -08:00
Chris Lu
a0e84c4fbc
go fmt
2021-02-10 23:41:05 -08:00
Chris Lu
c959aaa19b
mount: load directory based on correct filer path
...
fix https://github.com/chrislusf/seaweedfs/issues/1799
2021-02-10 23:14:18 -08:00
Konstantin Lebedev
0cfed8c3cb
restart replication if RabbitMQ connection closed
...
https://github.com/google/go-cloud/issues/2958
2021-02-10 15:29:05 +05:00
Chris Lu
770393a48c
volume: add capability to change disk type when moving a volume
2021-02-09 23:58:08 -08:00
Chris Lu
821c46edf1
Merge branch 'master' into support_ssd_volume
2021-02-09 11:37:07 -08:00
Chris Lu
a833021132
fix refactoring left over
2021-02-07 23:03:03 -08:00
Chris Lu
5e4b5109dd
2.24
2021-02-07 22:29:43 -08:00
Chris Lu
15c60cbb26
close the grpc connection after 10 hours
...
related to https://github.com/chrislusf/seaweedfs/issues/1782
2021-02-07 03:50:01 -08:00
bingoohuang
7256902fb0
fix typo offset.ToAcutalOffset to offset.ToActualOffset
2021-02-07 12:11:51 +08:00
bingoohuang
94ea3bd3a5
renaming NeedleMapType to NeedleMapKind
2021-02-07 09:00:03 +08:00
Chris Lu
c3af72d950
fier store: fix elastic search regression
...
fix https://github.com/chrislusf/seaweedfs/issues/1774
2021-02-04 01:30:14 -08:00
Konstantin Lebedev
2fb4c60540
fix log warn
2021-02-03 21:34:32 +05:00
Konstantin Lebedev
24ec17219b
RabbitMQ delay retry with Dead Letter Exchange
...
https://github.com/chrislusf/seaweedfs/issues/1773
https://github.com/google/go-cloud/issues/2952
2021-02-03 21:32:30 +05:00
Chris Lu
a331bbb3ae
filer: should return 204 on DELETE to nonexistent file
...
related to
https://github.com/chrislusf/seaweedfs/issues/1776
https://github.com/chrislusf/seaweedfs/issues/1160
2021-02-03 00:40:31 -08:00
Chris Lu
5353e38469
S3: should return 204 on DELETE to nonexistent file
...
fix https://github.com/chrislusf/seaweedfs/issues/1776
2021-02-03 00:35:44 -08:00
Chris Lu
2396ac234c
filer: fix elastic search pagination
...
possible fix for https://github.com/chrislusf/seaweedfs/issues/1774
2021-02-02 11:34:12 -08:00
Chris Lu
609daaf387
s3: DeleteMultipleObjectsHandler clean up leftover empty folders
...
fix https://github.com/chrislusf/seaweedfs/issues/1772
2021-02-01 10:49:17 -08:00
Chris Lu
d67ccb66c0
webdav: can start together with "weed server" or "weed filer"
2021-01-31 22:16:52 -08:00
Chris Lu
76430790b9
webdav: cache to version specific folder
2021-01-31 22:02:03 -08:00
Chris Lu
318a3d2efc
2.23
2021-01-31 20:18:06 -08:00
Chris Lu
4be51c0701
filer: leveldb and hbase may miss files when listing large directories more than 1024
...
fix https://github.com/chrislusf/seaweedfs/issues/1768
2021-01-31 20:11:44 -08:00
Chris Lu
1102ae32c4
fix concurrent map reads
2021-01-31 18:26:26 -08:00
Chris Lu
314dc1c957
filer: etcd store fix listing
...
fix https://github.com/chrislusf/seaweedfs/issues/1767
2021-01-30 13:38:44 -08:00
Chris Lu
d475c89fcc
go fmt
2021-01-28 15:23:46 -08:00
Chris Lu
19295600f9
mount: change option name to volumeServerAccess, with publicUrl and filerProxy modes
2021-01-28 15:23:16 -08:00
Chris Lu
990fa69bfe
add back AdjustedUrl() related code
2021-01-28 14:36:29 -08:00
Chris Lu
9292796ec2
fmt
2021-01-28 14:29:14 -08:00
Chris Lu
c2bf1a88ac
delete from the deepest directory first when checking empty folders
2021-01-28 14:28:40 -08:00
Chris Lu
e9d8201925
s3: batch purge empty folders
2021-01-28 13:20:06 -08:00
Chris Lu
cf252fc0cd
mount: report error when Flush()
...
The error in Release() is not working.
See https://github.com/jaderhs/libfuse/blob/master/FAQ
related to https://github.com/chrislusf/seaweedfs/issues/1765
2021-01-28 04:46:37 -08:00
Chris Lu
9a06c35da4
replicate: incremental sink only contains new and updated files
...
address da08402ba2
2021-01-28 02:39:22 -08:00
Chris Lu
da08402ba2
replicate: use creation time for local incremental file sink
...
related to https://github.com/chrislusf/seaweedfs/pull/1762
2021-01-28 02:17:41 -08:00
Chris Lu
822f1ade9d
Merge pull request #1762 from kmlebedev/backupsink
...
replication to create time date directory
2021-01-28 02:03:33 -08:00
Konstantin Lebedev
be1062b7fc
rename in scaffold
2021-01-28 14:59:20 +05:00
Konstantin Lebedev
02fdc0a333
rename backup to local_incremental and use mtime
2021-01-28 14:56:13 +05:00
Chris Lu
5f72c388bc
Merge pull request #1764 from danlsgiga/handle-umask
...
Use provided umask on mount
2021-01-28 01:09:20 -08:00
Daniel Santos
d38613c49a
match upstream
2021-01-27 20:56:41 -07:00
Chris Lu
b81956bcb5
mount: invalidate kernel cache when mounted to a filer path
...
fix https://github.com/chrislusf/seaweedfs/issues/1752#issuecomment-768178422
2021-01-27 10:28:37 -08:00
Konstantin Lebedev
6b54ff9912
replication to create time date directory
2021-01-27 15:01:33 +05:00
Chris Lu
f20ec82a28
update help message
...
see https://github.com/chrislusf/seaweedfs/issues/1758#issuecomment-768123224
2021-01-27 00:48:31 -08:00
Chris Lu
1045ab8b07
avoid error message
2021-01-27 00:21:59 -08:00
Chris Lu
0488c84bea
adjust help message
2021-01-27 00:17:52 -08:00
Konstantin Lebedev
3634811408
replication to todays date directory
2021-01-27 12:45:58 +05:00
Chris Lu
5e07afb0f0
Merge pull request #1759 from kmlebedev/sink.local
...
replication to local disk storage
2021-01-26 22:31:54 -08:00
Chris Lu
b3f66199db
shell: volume.fix.replication adds collectionPattern with wildcard characters
...
fix https://github.com/chrislusf/seaweedfs/issues/1758
2021-01-26 22:30:39 -08:00
Daniel Santos
0c64d60fdb
Fix log message with correct mode
2021-01-26 21:45:52 -07:00
Daniel Santos
0aaaa9b470
Fix var assignment
2021-01-26 21:06:08 -07:00
Daniel Santos
5a761bb7a4
Enforce umask correctly
2021-01-26 20:42:10 -07:00
Chris Lu
ad2a20c8a5
notification add ack and nack
2021-01-26 11:08:44 -08:00
Konstantin Lebedev
612b7975a1
replication to local disk storage
2021-01-26 22:50:25 +05:00
Chris Lu
3a1d3d3413
mount: properly invalidate kernel node cache entry
...
fix https://github.com/chrislusf/seaweedfs/issues/1752
2021-01-26 02:50:53 -08:00
Chris Lu
cc1f3907ff
filer.sync: default filerProxy to true
2021-01-25 10:14:42 -08:00
Chris Lu
62ebb917e0
mount: avoid the need to specify different cache directory for different mount
...
fix https://github.com/seaweedfs/seaweedfs-csi-driver/issues/11
2021-01-25 09:10:20 -08:00
Chris Lu
17f7c1c43f
Merge branch 'mount_from_outside_cluster'
2021-01-24 19:03:02 -08:00
Chris Lu
00707ec00f
mount: outsideContainerClusterMode proxy through filer
...
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
2021-01-24 19:01:58 -08:00
Chris Lu
9bcb28a3ea
avoid possible nil reader
...
fix https://github.com/chrislusf/seaweedfs/issues/1754
2021-01-24 17:19:35 -08:00
Chris Lu
2c5eac5705
2.22
2021-01-24 14:06:09 -08:00
Chris Lu
6ca10725b8
Revert "mount: when outside cluster network, use filer as proxy to access volume servers"
...
This reverts commit 096e088d7b
.
2021-01-24 03:15:19 -08:00
Chris Lu
096e088d7b
mount: when outside cluster network, use filer as proxy to access volume servers
2021-01-24 01:41:38 -08:00
Chris Lu
ccbdb38c89
s3: fix for listing objects if more than 1000 in the folder
...
many existing tools have max key set to 1000
2021-01-24 00:27:52 -08:00
Chris Lu
80b8692688
filer.sync: replicate outside of either cluster, only need to see filers
2021-01-24 00:01:44 -08:00
Chris Lu
f8dbb03bdd
filer: append operation returns final file size
2021-01-23 04:19:49 -08:00
Chris Lu
759482e2ac
filer: proxy all http methods to volume servers
2021-01-23 03:52:38 -08:00
Chris Lu
937cfacc01
filer: add "proxyToFileId" to reverse proxy to a volume server
2021-01-23 03:43:48 -08:00
Chris Lu
20ef3bb8d4
mount: use direct_io to avoid OS page cache
...
fix https://github.com/chrislusf/seaweedfs/issues/1752
2021-01-22 22:39:46 -08:00
Chris Lu
711c3f3939
filer: fix http status 206 setting
...
fix https://github.com/chrislusf/seaweedfs/issues/1753
2021-01-22 00:26:30 -08:00
Chris Lu
024c186a48
filer.meta.tail: output json format for easier parsing
2021-01-21 22:39:25 -08:00
Chris Lu
84f05787f8
mount: limit background requests and congestion threshold
2021-01-20 23:30:14 -08:00
Chris Lu
f98817cfe6
filer: support appending to a file
2021-01-20 13:40:32 -08:00
Chris Lu
1b8e3da295
refactoring
2021-01-20 12:59:54 -08:00
Chris Lu
1a3f859c58
filer store: postgres configuration adjust text
2021-01-19 18:12:07 -08:00
Chris Lu
b434f7e4e0
filer store: postgres adjust default config, add optioanl schema
2021-01-19 18:10:36 -08:00
Chris Lu
d5add83e85
filer store: add postgres2
2021-01-19 18:07:29 -08:00
Chris Lu
52a8f1470e
filer store: add mysql2
2021-01-19 17:21:50 -08:00
Chris Lu
93b3adba98
fix bucket creation
2021-01-19 15:55:51 -08:00
Chris Lu
fa0c8d5283
fix error
2021-01-19 14:05:48 -08:00
Chris Lu
4c5b752b04
restructuring sql stores
2021-01-19 13:53:16 -08:00
Chris Lu
96354208c5
scaffold: add YugabyteDB
2021-01-19 12:35:14 -08:00
Chris Lu
ca8f793978
refactor: prepare for bucket specific tables
2021-01-19 12:34:58 -08:00
Chris Lu
90ce1eec19
add more input params
2021-01-19 11:32:55 -08:00
Chris Lu
f13c082583
add memsql to comments
2021-01-19 11:31:55 -08:00
Chris Lu
e439b65e38
fix test
2021-01-18 01:48:00 -08:00
Chris Lu
c7197470fd
2.21
2021-01-18 01:25:04 -08:00
Chris Lu
5a3386e39b
Revert "Revert "mount: fake support for socket/block/character/fifo devices""
...
This reverts commit 3d5cb7eb86
.
2021-01-18 01:15:07 -08:00
Chris Lu
67faa56920
Revert "Revert "mount: add more entry type""
...
This reverts commit 4085b79d38
.
2021-01-18 01:14:58 -08:00
Chris Lu
7119e7b491
mount: ensure reading from dirty pages
2021-01-18 01:14:42 -08:00
Chris Lu
de876c795d
minor fix
2021-01-18 01:14:27 -08:00
Chris Lu
4085b79d38
Revert "mount: add more entry type"
...
This reverts commit d7d907be27
.
2021-01-18 00:19:19 -08:00
Chris Lu
3d5cb7eb86
Revert "mount: fake support for socket/block/character/fifo devices"
...
This reverts commit 61ef2d8658
.
2021-01-18 00:18:57 -08:00
Chris Lu
61ef2d8658
mount: fake support for socket/block/character/fifo devices
...
to pass pjdfstest. A distributed file system can not really support these things anyway.
2021-01-17 23:51:10 -08:00
Chris Lu
d7d907be27
mount: add more entry type
2021-01-17 23:46:39 -08:00
Chris Lu
389426bbb7
s3: listing may repeat on the edge
2021-01-17 22:59:31 -08:00
Chris Lu
09f49d1c04
refactoring
2021-01-16 19:52:15 -08:00
Chris Lu
a4063a5437
add stream list directory entries
2021-01-15 23:56:24 -08:00
Chris Lu
01dc8a43ba
fix rocksdb
2021-01-15 18:31:29 -08:00
Chris Lu
16ad74f477
go fmt
2021-01-14 23:11:27 -08:00
Chris Lu
f002e668de
change limit to int64 in case of overflow
2021-01-14 23:10:37 -08:00
Chris Lu
19e52fd414
pass along hasMore
2021-01-14 22:49:29 -08:00
Chris Lu
c0bcf6f6e1
implement sql changes
2021-01-14 22:44:22 -08:00
Chris Lu
5d4568b91f
implement elastic changes
2021-01-14 22:42:25 -08:00
Chris Lu
893cbc8482
implement c* changes
2021-01-14 22:38:34 -08:00
Chris Lu
5ef43b9b09
implement etcd changes
2021-01-14 22:35:56 -08:00
Chris Lu
54527f0326
implement hbase changes
2021-01-14 22:34:35 -08:00
Chris Lu
b5ceffe188
implement leveldb changes
2021-01-14 22:33:05 -08:00
Chris Lu
c64bfb0e2e
implement mongodb changes
2021-01-14 22:28:51 -08:00
Chris Lu
34a846009d
implement rocksdb changes
2021-01-14 22:24:26 -08:00
Chris Lu
d2ece29fff
implement redis changes
2021-01-14 22:21:31 -08:00
Chris Lu
9a50dbcda0
chagned api
2021-01-14 22:18:38 -08:00
Chris Lu
e1c7bc66f9
change interface
2021-01-14 22:17:49 -08:00
Chris Lu
4aaa97c789
skip not found error in case of race condition
2021-01-14 19:56:14 -08:00
Chris Lu
d0826c2f62
mount: avoid showing "topics" folder
...
fix https://github.com/chrislusf/seaweedfs/issues/1325
2021-01-14 19:03:15 -08:00
Chris Lu
698f58f7c4
filer, s3: add http status 206 as late as possible
...
fix https://github.com/chrislusf/seaweedfs/issues/1746
2021-01-14 02:59:40 -08:00
Chris Lu
fd1d8a2a09
fix compilation
2021-01-14 00:06:15 -08:00
Chris Lu
ae812c07d6
Merge pull request #1743 from LazyDBA247-Anyvision/master
...
MYSQL - Better Connection Pool Control
2021-01-14 00:04:44 -08:00
Chris Lu
f20743bb38
remove WIP status
2021-01-13 22:21:34 -08:00
LazyDBA247-Anyvision
8eed763b97
better mysql connection pool management
...
adding SetConnMaxLifetime configuration (https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime )
to enable refresh of connections.
2021-01-14 08:14:21 +02:00
Chris Lu
f17aa1d06c
skip directory loop if dropping the bucket
2021-01-13 13:49:04 -08:00
Chris Lu
ca73013453
leveldb3: ensure repeatable adding removing buckets
2021-01-13 13:48:48 -08:00
Chris Lu
e2c7e3fe6d
better locks
2021-01-13 13:20:33 -08:00
Chris Lu
819a85f59a
avoid viper concurrent access
2021-01-13 13:14:52 -08:00
Chris Lu
3fb2ed9093
filer.meta.tail: optionally submit metadata changes to ElasticSearch
2021-01-13 03:31:37 -08:00
Chris Lu
c5df2577f5
rename
2021-01-13 00:31:19 -08:00
Chris Lu
90c5077610
rename "weed watch" to "weed filer.meta.tail"
2021-01-12 18:48:01 -08:00
Chris Lu
5cd9c9e642
remove println
2021-01-12 18:47:38 -08:00
Chris Lu
def6d8e565
fix test
2021-01-12 12:32:54 -08:00
Chris Lu
1c7faf1100
Revert "close opened file"
...
This reverts commit 2880160113
.
2021-01-12 11:07:14 -08:00
Chris Lu
2880160113
close opened file
2021-01-12 11:03:08 -08:00
Chris Lu
0ef43a23a7
go fmt
2021-01-12 02:30:12 -08:00
Chris Lu
629c9962e7
filer: add leveldb3
...
support dedicated leveldb instance for each bucket
2021-01-12 02:29:44 -08:00
Chris Lu
361043e6c1
filer store: leveldb2 fix nil entry error if not found
2021-01-12 02:28:57 -08:00
Chris Lu
cfb9342a15
avoid concurrent map updates to viper
2021-01-12 02:28:13 -08:00
Chris Lu
38d516251e
Revert "mount: avoid fatal error"
...
This reverts commit a4f0bac9b7
.
2021-01-11 13:50:15 -08:00
Chris Lu
a4f0bac9b7
mount: avoid fatal error
2021-01-11 13:26:59 -08:00
Chris Lu
8a7302ce37
s3: KeyCount is zero
...
fix https://github.com/chrislusf/seaweedfs/issues/1741
2021-01-11 03:50:45 -08:00
Chris Lu
2c7148cd55
redis cluster: do not enable route by latency by default
2021-01-11 02:43:53 -08:00
Chris Lu
acc0d5cd2c
Revert "Revert "filer store: redis module upgrade""
...
This reverts commit dd0171e35c
.
2021-01-11 02:30:19 -08:00
Chris Lu
dd0171e35c
Revert "filer store: redis module upgrade"
...
This reverts commit 40161c9cec
.
2021-01-11 02:09:18 -08:00
Chris Lu
394513f598
filer: ensure seamless meta data updates
2021-01-11 02:08:55 -08:00
Chris Lu
6e12a3a490
refactoring
2021-01-11 02:08:26 -08:00
Chris Lu
62ee484d12
refactoring
2021-01-11 00:03:13 -08:00
Chris Lu
1efb51ba84
filer: change to saveToFilerLimit from cacheToFilerLimit
...
short circuit saving small files to volume server
2021-01-10 23:14:46 -08:00
Chris Lu
f0d3b3bf93
2.20
2021-01-09 22:55:22 -08:00
Chris Lu
d3d3f2fb9b
mount: default to 128 concurrent writers
2021-01-09 22:53:37 -08:00
Chris Lu
90b117acf1
update ccache version
2021-01-08 02:17:43 -08:00
Chris Lu
e2e0e30a51
filer: redis default to false for useReadOnly and routeByLatency
2021-01-08 01:12:44 -08:00
Chris Lu
40161c9cec
filer store: redis module upgrade
2021-01-08 01:08:44 -08:00
Chris Lu
30efaa66c6
server: add option to disable master
2021-01-07 02:35:47 -08:00
Chris Lu
07f4703bfc
Merge pull request #1738 from qieqieplus/rocksdb
...
fix rocksdb crash when list directory
2021-01-07 02:26:41 -08:00
qieqieplus
0f18592315
alter style
2021-01-07 18:21:48 +08:00
qieqieplus
c7e7431fd3
fix seek lastkey may reach EOF
2021-01-07 17:07:56 +08:00
Chris Lu
e327385644
go fmt
2021-01-06 04:22:00 -08:00
Chris Lu
2b76854641
add "weed filer.cat" to read files directly from volume servers
2021-01-06 04:22:00 -08:00
Chris Lu
95ecf0c72f
Merge pull request #1735 from qieqieplus/rocksdb
...
ignore decode error for non-entry data
2021-01-06 02:22:18 -08:00
qieqieplus
0764fccde7
ignore decode error for non-entry data
2021-01-06 17:42:43 +08:00
Chris Lu
ab57767d4b
Merge pull request #1730 from qieqieplus/rocksdb
...
impl: TTL per entry for rocksdb; fix package name
2021-01-05 13:24:45 -08:00
qieqieplus
b2e50f602f
fix time comparison
2021-01-05 19:36:40 +08:00
qieqieplus
bbae13e415
impl: TTL per entry for rocksdb; fix package name
2021-01-05 17:58:40 +08:00
henry
b42c5b71c1
Seaweed custom header are not visible to Vue or javascript
2021-01-05 13:01:29 +08:00
henry
1309dfe612
Seaweed custom header are not visible to Vue or javascript
2021-01-05 12:58:46 +08:00
henry
942a30f04c
remove sort
2021-01-05 12:24:32 +08:00
henry
a3c432dadc
Merge branch 'master' of https://github.com/fuyouyshengwu/seaweedfs
2021-01-05 11:17:29 +08:00
qieqieplus
ebb223c190
fix for rocksdb kv
2021-01-04 21:21:46 +08:00
chenqieqie
40b3207f18
fix #1726
2021-01-04 18:48:55 +08:00
henry
97a94eddab
Merge branch 'master' of https://github.com/fuyouyshengwu/seaweedfs
2021-01-04 14:50:01 +08:00
Chris Lu
2ce86f308e
2.19
2021-01-03 17:24:02 -08:00
Chris Lu
9c9ba3c209
nil related
...
related to https://github.com/chrislusf/seaweedfs/issues/1676
2021-01-03 12:25:58 -08:00
Chris Lu
4b5adc60b4
upload result changes to the right output variable
2021-01-03 01:44:22 -08:00
Chris Lu
039ad101b1
manifest file also need to detect compression
...
fix https://github.com/chrislusf/seaweedfs/issues/1724#issuecomment-753585046
2021-01-03 01:42:21 -08:00
Chris Lu
5f538c5f9d
Merge branch 'add_rocksdb'
2021-01-03 00:48:56 -08:00
Chris Lu
bdd166c031
adjust readme
2021-01-03 00:47:58 -08:00
Chris Lu
a04d8927a8
add scaffold
2021-01-03 00:44:52 -08:00
Chris Lu
4250a3a675
upload: add read default collection from master
...
fix https://github.com/chrislusf/seaweedfs/issues/1724#issuecomment-753576071
2021-01-02 23:43:01 -08:00
Chris Lu
f4d87f8b9d
works
...
performance seems about the same as leveldb for insert entries
2021-01-02 23:32:58 -08:00
Chris Lu
2c3c2c27d7
separate prefix from namePattern
...
fix https://github.com/chrislusf/seaweedfs/issues/1722
2021-01-01 20:23:23 -08:00
Peter Cai
cd2a9865ed
filer: respect Content-Type on PUT
...
This is needed for the S3 gateway to conform to the S3 protocol -- the
server should always respect the Content-Type set by client.
2020-12-31 11:32:45 +08:00
Chris Lu
3433accb1b
weed upload: add TTL
2020-12-30 15:46:07 -08:00
henry
14ddd15508
1 Distinguish between folders and common files, folders are listed first
...
2 Add some custom headers that need to be exposed
2020-12-31 00:03:22 +08:00
Chris Lu
adef4ddc87
mount: load security keys
...
fix https://github.com/chrislusf/seaweedfs/issues/1714
2020-12-29 11:51:38 -08:00
Chris Lu
89977123b6
shell: add shell.toml support
...
fix https://github.com/chrislusf/seaweedfs/issues/1664
2020-12-28 15:07:18 -08:00
Chris Lu
d2ead72918
2.18
2020-12-28 12:37:10 -08:00
Chris Lu
2e6cf541e7
filer: avoid deletion error when item is not found
2020-12-28 00:18:37 -08:00
Chris Lu
6d2c979999
s3: fix regression
...
fix https://github.com/chrislusf/seaweedfs/issues/1707
2020-12-27 21:09:45 -08:00
Chris Lu
90df52d001
s3: add command option "allowEmptyFolder" to filer and server
2020-12-27 21:09:06 -08:00
Chris Lu
da7e5aaa65
filer: optimize for less number of directory lookup
...
bottom up directory lookup
2020-12-27 20:46:17 -08:00
henry
7458ff9523
Solve the Chinese name disorder of download file
2020-12-28 11:28:33 +08:00
Chris Lu
561a2ff0bc
s3: add option for "alllowEmptyFolder"
2020-12-27 10:50:27 -08:00
Chris Lu
aa020ee7e7
s3: restore V2 auth type
...
The added headers were included in calculating the signatures, failing the comparison
2020-12-26 22:01:16 -08:00
Chris Lu
da87f6b265
remove unused code
2020-12-26 15:21:12 -08:00
Chris Lu
0a7c5f85a9
filer: add namePattern to search in current folder
2020-12-26 15:05:31 -08:00
Chris Lu
515b7632d7
2.17
2020-12-25 23:58:22 -08:00
Chris Lu
0a067944cc
filer: add retries during volume moving
...
fix https://github.com/chrislusf/seaweedfs/issues/1704
2020-12-25 02:32:55 -08:00
Chris Lu
090f85be4b
s3: support config action Admin:bucket
2020-12-25 00:38:56 -08:00
Chris Lu
75613b2cc7
Merge branch 'add_hbase'
2020-12-24 12:13:05 -08:00
Chris Lu
0e016bc7bd
hbase add ttl
2020-12-24 12:10:35 -08:00
Chris Lu
8e48a235e2
s3: avoid looping if the directory is empty
...
fix https://github.com/chrislusf/seaweedfs/issues/1701
2020-12-24 11:34:52 -08:00
Chris Lu
1620de08ae
added more logs
2020-12-24 01:52:06 -08:00
Chris Lu
a09cd28986
add debug messages
2020-12-24 01:42:15 -08:00
Chris Lu
75c6edba9e
filer: hbase add enabled flag
2020-12-24 00:19:16 -08:00
Chris Lu
9bf6c10505
fix prefix
2020-12-24 00:18:59 -08:00
Chris Lu
2fea8cfa0f
fix compilation
2020-12-23 23:54:15 -08:00
Chris Lu
53bc1ea25b
fix compilation
2020-12-23 23:53:46 -08:00
Chris Lu
b5e2be635a
adjust for directory listing
2020-12-23 23:49:22 -08:00
Chris Lu
c4a202ec41
fix wrong column family
2020-12-23 23:23:05 -08:00
Chris Lu
c3d1b3b5aa
hook up
2020-12-23 21:49:01 -08:00
Chris Lu
94e3757c08
fix
2020-12-23 21:45:16 -08:00
Chris Lu
64c48c9724
seems compiling
2020-12-23 21:44:53 -08:00
Chris Lu
15da5834e1
Merge branch 'master' into support_ssd_volume
2020-12-23 14:37:37 -08:00
Chris Lu
3be3635799
just log error when checking empty dir failed
2020-12-23 14:36:48 -08:00
Chris Lu
0ca9d89589
s3: break loop if error
...
fix #1701
2020-12-23 14:34:59 -08:00
Chris Lu
132f275d04
s3: log errors instead of stopping when checking isDirectoryAllEmpty has error
2020-12-23 12:21:21 -08:00
Chris Lu
db62090f88
s3: break loop if error
...
fix https://github.com/chrislusf/seaweedfs/issues/1701
2020-12-23 12:14:51 -08:00
Baptiste Mille-Mathias
385a4dec30
[shell] Clarify bucket replication setting order
2020-12-23 09:35:24 +01:00
Chris Lu
1c7e1295dc
Merge branch 'master' into support_ssd_volume
2020-12-22 17:44:52 -08:00
Chris Lu
da134a2eb7
minor
2020-12-22 17:43:13 -08:00
Chris Lu
beb3b8ddb0
shell: change bucket.list to s3.bucket.list, same for create and delete
2020-12-22 17:40:55 -08:00
Chris Lu
4cf3176cbb
filer first, master second
2020-12-22 17:34:44 -08:00
Chris Lu
a1009e8044
filer: add -defaultStoreDir so that filer.toml can be skipped
...
fix https://github.com/chrislusf/seaweedfs/issues/1659
2020-12-22 17:33:40 -08:00
Chris Lu
a0990b929d
minor
2020-12-22 17:33:40 -08:00
Baptiste Mille-Mathias
0ed5345954
[shell] Add more information to bucket.create
...
State the replication setting that will be honor if not defined.
2020-12-22 17:18:59 +01:00
Chris Lu
97e3432dfe
avoid wrong error
...
fix https://github.com/chrislusf/seaweedfs/issues/1691
2020-12-22 02:46:24 -08:00
Chris Lu
4f31c1bb94
go fmt
2020-12-22 02:34:08 -08:00
Chris Lu
6c4f32d173
remove unused code
2020-12-22 02:33:31 -08:00
Chris Lu
90d785a15f
filer: redis, redis cluster, cassandra support super large directory
2020-12-22 02:26:05 -08:00
Chris Lu
fe46411cd4
handle directory listing edge cases
2020-12-21 23:37:43 -08:00
Chris Lu
c74bede730
handle listing directories
2020-12-21 23:19:05 -08:00
Chris Lu
adf8cb4000
filer: path-specific stores trim out common prefixes
2020-12-21 22:57:13 -08:00
Chris Lu
0823bde8d8
refactor: split into two files
2020-12-21 22:05:15 -08:00
Chris Lu
8ab70ccf76
refactor
2020-12-21 21:46:32 -08:00
Chris Lu
65cc6dc636
refactor to dedicated function
2020-12-21 21:18:34 -08:00
Chris Lu
488c2680e8
change default filer store root
2020-12-21 21:15:22 -08:00
Chris Lu
f5a19a87be
filer: report error if the path specific store is not found
...
related https://github.com/chrislusf/seaweedfs/issues/1691
2020-12-21 12:43:52 -08:00
Chris Lu
6912bf94ae
2.16
2020-12-20 15:30:12 -08:00
Chris Lu
41c0f3ad24
filer: support path-specific filer store
2020-12-19 01:27:09 -08:00
Chris Lu
2260864b45
a cleaner fix
2020-12-18 10:39:30 -08:00
Chris Lu
b7fef8f693
fix when a volume server has only hdd volumes
2020-12-18 10:34:16 -08:00
Chris Lu
0d5683fb0e
todo: load path-specific store from filer.toml
2020-12-18 03:05:42 -08:00
Chris Lu
3269fd7eaf
filer: use store by path
2020-12-18 02:57:49 -08:00
Chris Lu
e605f1e001
refactor, add getActualStore() by path
2020-12-18 02:55:00 -08:00
Chris Lu
23903aa95b
refactoring
2020-12-18 02:35:45 -08:00
Chris Lu
4e6b316913
assert FilerStoreWrapper is VirtualFilerStore
2020-12-18 02:17:06 -08:00
Chris Lu
9abf016af1
Merge branch 'master' into support_ssd_volume
2020-12-17 22:30:19 -08:00
Chris Lu
738c4fd203
fix logging when error
...
address https://github.com/chrislusf/seaweedfs/pull/1685#issuecomment-747845601
2020-12-17 19:37:46 -08:00
Chris Lu
d9e8479c06
adjust UI max count
2020-12-17 13:47:51 -08:00
Chris Lu
3cdf5945a2
adjust UI
2020-12-17 13:37:00 -08:00
Chris Lu
f696a2b2a7
assign volumes based on disk type
2020-12-17 13:25:05 -08:00
Chris Lu
daa8157fc2
Merge branch 'master' into support_ssd_volume
2020-12-17 13:05:20 -08:00
Chris Lu
e2076201d7
volume: avoid reprocessing the same volume
...
fix https://github.com/chrislusf/seaweedfs/issues/1682
2020-12-17 13:03:39 -08:00
Chris Lu
986cbdf7d9
Revert "Merge pull request #1683 from qieqieplus/master"
...
This reverts commit 8cb67952db
, reversing
changes made to 200e56215a
.
2020-12-17 12:46:42 -08:00
Chris Lu
f56e6d231e
Revert "Merge pull request #1683 from qieqieplus/master"
...
This reverts commit 8cb67952db
, reversing
changes made to 200e56215a
.
2020-12-17 12:46:20 -08:00
Chris Lu
1bf22c0b5b
go fmt
2020-12-16 09:14:05 -08:00
Chris Lu
1d88865869
passing disk type along
2020-12-16 09:10:14 -08:00
Chris Lu
23014b6810
Merge branch 'master' into support_ssd_volume
2020-12-16 08:26:51 -08:00
Chris Lu
b789767cd1
filer: return http status 499 for client closed connection
...
fix https://github.com/chrislusf/seaweedfs/issues/1684
2020-12-16 08:18:00 -08:00
chenqieqie
45f902a9b7
smaller critical section
2020-12-16 20:23:51 +08:00
chenqieqie
4e58a4f24e
fix race condition when loading volumes concurrently
2020-12-16 18:49:10 +08:00
Chris Lu
3c7f9633eb
show error a little bit more obviously
...
fix https://github.com/chrislusf/seaweedfs/issues/1678
2020-12-14 09:39:33 -08:00
Chris Lu
97dcae827b
Merge branch 'master' into support_ssd_volume
2020-12-14 01:27:31 -08:00
Chris Lu
3d47c38262
collection.delete requires _default_ as the default empty collection name
...
fix https://github.com/chrislusf/seaweedfs/issues/1677
2020-12-14 01:05:20 -08:00
Chris Lu
2e8dba571b
adjust volume server UI
2020-12-14 00:51:57 -08:00
Chris Lu
bf5873022d
Merge branch 'master' into support_ssd_volume
2020-12-14 00:32:17 -08:00
Chris Lu
b7e3ca9172
Merge pull request #1650 from kmlebedev/masterServerEnableUI
...
security master use access.ui
2020-12-14 00:31:51 -08:00
Chris Lu
021358749b
better detect IP address
...
following advice from https://github.com/chrislusf/seaweedfs/issues/1671
2020-12-14 00:30:20 -08:00
Konstantin Lebedev
c4459249ef
ui interface is always on
2020-12-14 13:24:41 +05:00
Chris Lu
5cffddebb9
add logging
2020-12-14 00:11:52 -08:00
Chris Lu
933f2f4cfd
find location with matching disk type
2020-12-13 23:39:00 -08:00
Chris Lu
94525aa0fd
allocate volume by disk type
2020-12-13 23:08:21 -08:00
Chris Lu
f6a419c26c
disk type configurable for each folder
2020-12-13 22:49:56 -08:00
Chris Lu
7c52a35942
refactor
2020-12-13 22:29:52 -08:00
Chris Lu
c45d6a865b
Merge branch 'master' into support_ssd_volume
2020-12-13 22:14:18 -08:00
Chris Lu
38fc89d041
filer store: sql update if any insert error happens
...
fix https://github.com/chrislusf/seaweedfs/issues/1673
2020-12-13 20:49:44 -08:00
Chris Lu
23280257df
sql put kv: avoid unnecessary update
2020-12-13 19:45:47 -08:00
Chris Lu
a9db24cd05
master allocate volumes if ssd type runs out
2020-12-13 19:44:57 -08:00
Chris Lu
ac4d4a65af
Update Makefile
2020-12-13 12:34:58 -08:00
Chris Lu
8baba93fce
rename parameter name to "disk"
2020-12-13 12:06:21 -08:00
Chris Lu
51eadaf2b6
rename parameter name to "disk"
2020-12-13 12:05:31 -08:00
Chris Lu
0d2ec832e2
rename from volumeType to diskType
2020-12-13 11:59:32 -08:00
Chris Lu
715b199eeb
fix tests
2020-12-13 04:14:50 -08:00
Chris Lu
66cd219b15
Merge branch 'master' into support_ssd_volume
2020-12-13 04:08:36 -08:00
Chris Lu
813453f3e1
2.15
2020-12-13 03:45:35 -08:00
Chris Lu
17372ac3da
balance ssd volumes
2020-12-13 03:40:33 -08:00
Chris Lu
d156c74ec0
volume server set volume type and heartbeat to the master
2020-12-13 03:11:24 -08:00
Chris Lu
e9cd798bd3
adding volume type
2020-12-13 00:58:58 -08:00
Chris Lu
16cd6fb278
fix variable name
2020-12-12 21:19:40 -08:00
Chris Lu
5c465293e9
correctly determine whether a folder is empty
...
avoid edge cases that deleting the folder if previous 32 directories are all empty
early terminate if one file is found
2020-12-12 16:19:29 -08:00
Chris Lu
14910d035c
the pagination size was too big for recursive deletion
2020-12-12 13:26:10 -08:00
Chris Lu
f930c713fc
more efficient recursion
2020-12-12 13:25:19 -08:00
Chris Lu
316d1b4e69
refactor APIs
2020-12-12 12:42:53 -08:00
Chris Lu
03637d6f57
s3: move "delete-directory-if-empty" to read time
...
move "delete-directory-if-empty" to read time instead of entry deletion time
the listing speed for a s3 bucket folder will slow down if it has many sub folders
related to 0d345ac97d
fix https://github.com/chrislusf/seaweedfs/issues/1647
fix https://github.com/chrislusf/seaweedfs/issues/1670
2020-12-12 03:38:34 -08:00
Chris Lu
37075a414d
adjust logs
2020-12-12 03:33:57 -08:00
Chris Lu
ae655033ac
adjust logging
2020-12-11 16:57:53 -08:00
Chris Lu
d986c7196d
use append time instead of filer's own modification time
...
fix https://github.com/chrislusf/seaweedfs/issues/1669
2020-12-11 16:55:18 -08:00
Chris Lu
3fedfec1e7
check cross device rename error
2020-12-10 23:50:32 -08:00
Chris Lu
c2f18a10cb
minor
2020-12-10 22:23:22 -08:00
Chris Lu
4b0c2a846b
add logs
2020-12-10 19:55:28 -08:00
Chris Lu
83078ac6ce
filer: change to /etc/seaweedfs folder on filer
...
fix https://github.com/chrislusf/seaweedfs/issues/1666
2020-12-10 11:11:02 -08:00
Chris Lu
26731694f8
s3: use static configuration by default
...
So that users can still use the previous configuration files.
If leave it empty, s3 will try to use the version from filer
2020-12-10 00:59:04 -08:00
Chris Lu
765b3ef1be
save /etc/iam/identity.json inside filer store
2020-12-10 00:15:22 -08:00
Chris Lu
97c942b5ee
paginate through large directories
2020-12-09 23:23:38 -08:00
Chris Lu
02a160c3fd
s3: recursively iterate all sub folders
...
fix https://github.com/chrislusf/seaweedfs/issues/1656
2020-12-09 16:47:34 -08:00
Chris Lu
a9c619c270
added error handling for saving
2020-12-09 01:34:24 -08:00
Chris Lu
4211601eab
set file handle reader to nil
2020-12-08 22:48:18 -08:00
Chris Lu
b52ae9cef8
add s3 debug
2020-12-08 22:27:04 -08:00
Chris Lu
8e78187a97
add back last read chunk cache to reader and properly close the reader
2020-12-08 22:26:46 -08:00
Chris Lu
900d22c6ec
mount: avoid memory leaking read buffer
...
fix https://github.com/chrislusf/seaweedfs/issues/1654
the reader goes together with the file handle, which may stay for a long time.
2020-12-08 02:38:53 -08:00
Chris Lu
06bb7bf6c0
adding /usr/local/etc/seaweedfs for bsd style config search path
...
related to https://reviews.freebsd.org/D27391
2020-12-07 16:46:48 -08:00
Chris Lu
fce8803087
break import cycle
2020-12-07 00:29:17 -08:00
Chris Lu
606051cc79
filer: cache "/etc" filder small files to filer store
2020-12-07 00:11:35 -08:00
Chris Lu
ac22f1dd26
filer subscribe: handle rename subscription
2020-12-07 00:10:49 -08:00
Chris Lu
eed87791b7
s3: subscribe to s3.configure changes
2020-12-07 00:10:29 -08:00
Chris Lu
01e2da5782
refactoring
2020-12-06 23:16:20 -08:00
Chris Lu
77286f8bea
fix test
2020-12-06 21:56:13 -08:00
Chris Lu
57578a6cd1
change parameter help message
2020-12-06 21:54:55 -08:00
Chris Lu
78f3ab439c
refactoring
2020-12-06 21:54:34 -08:00
Chris Lu
9acda432fe
fix import cycle
2020-12-06 20:12:52 -08:00
Chris Lu
ae5eb85a06
refactoring
2020-12-06 20:05:06 -08:00
Chris Lu
5931a2f53f
filer conf: support json conf only
2020-12-06 19:47:06 -08:00
Chris Lu
4c72482536
shell: fix fs.configure
2020-12-06 19:43:31 -08:00
Chris Lu
46b91228af
2.14
2020-12-06 18:56:58 -08:00
Chris Lu
263eb29e9f
filer: add option to cache small files to filer store
2020-12-04 22:39:43 -08:00
Chris Lu
d171d9f988
volume: ensure the volume dat and idx files are always closed
...
fix https://github.com/chrislusf/seaweedfs/issues/1646
2020-12-04 21:50:26 -08:00
Konstantin Lebedev
eadbba5c95
security master use access.ui
2020-12-04 12:57:59 +05:00
Chris Lu
cc839f935d
better unit detection
2020-12-03 19:23:59 -08:00
Chris Lu
b219ccfe68
Update volume_ttl_test.go
2020-12-03 12:55:44 -08:00
Chris Lu
a83b8e7b01
ttl fix: 3y was converted into 96m
2020-12-03 10:53:29 -08:00
Chris Lu
5fb60b7135
Merge pull request #1596 from kmlebedev/store_s3cred
...
S3 credentials store in filer
2020-12-03 00:40:20 -08:00
Konstantin Lebedev
14699dfcef
use content field of entry
2020-12-02 17:19:05 +05:00
Konstantin Lebedev
a3d4b50a49
use entry content filed
2020-12-02 16:12:13 +05:00
Chris Lu
003b6245e7
fix nil
2020-12-02 00:09:19 -08:00
Chris Lu
f3bb645018
file open error
2020-12-01 23:37:49 -08:00
Chris Lu
dc0bc48257
return file open error
2020-12-01 23:36:49 -08:00
Chris Lu
45dba088cd
fix tests
2020-12-01 19:37:21 -08:00
Chris Lu
04062c56c7
webdav: improve webdav upload speed
2020-12-01 15:32:27 -08:00
Konstantin Lebedev
03620776ec
Merge branch 'upstream_master' into store_s3cred
2020-12-01 16:03:34 +05:00
Chris Lu
005a6123e9
Merge pull request #1643 from hilimd/master
...
fix tls grpc ca path
2020-11-30 17:16:46 -08:00
Chris Lu
f4abd01adf
filer: cache small file to filer store
2020-11-30 04:34:04 -08:00
Chris Lu
a9c6be5fc3
set creation time
2020-11-30 03:11:52 -08:00
Chris Lu
141ce67c09
close http request body
2020-11-30 02:45:00 -08:00
Chris Lu
0e99531dbf
2.13
2020-11-29 17:00:03 -08:00
Chris Lu
965413c21b
shell: add volume.vacuum command
2020-11-28 23:18:02 -08:00
Chris Lu
96c48bc8a8
fix test
2020-11-28 03:03:43 -08:00
Chris Lu
c57a7839ec
adjust comments
2020-11-28 03:01:03 -08:00
Chris Lu
8438fb0cc3
fix ec shard spreading error
2020-11-28 02:21:16 -08:00
Chris Lu
536202e4e0
minor
2020-11-28 02:16:06 -08:00
Chris Lu
99c4e50d3d
minor
2020-11-28 00:14:11 -08:00
Chris Lu
2c913dde04
volume: detect and drop volumes with disk IO error
...
from Jethro in slack:
is it possible to make the assign request a bit smarter? Currently I’m in the state that a disk failed but all assign request are being send to this volume. It would be cool if the master sees this and stopped using this volume.
e=HTTP(http://x:8089/913,045a782b63176edf ) not 200 but 500 Internal Server Error
Body={"size":740167,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"ee4381e202212ff3aee647704c036689"}
e=HTTP(http://x:8089/913,045a782c90240077 ) not 200 but 500 Internal Server Error
Body={"size":792779,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"c43463ccc11eb6eb2fc306f407a6a953"}
e=HTTP(http://x:8089/913,045a782e6b7901ea ) not 200 but 500 Internal Server Error
Body={"size":3962392,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"04c91198e9b276c81f11dbf189af5d28"}
2020-11-28 00:09:29 -08:00
Chris Lu
9ac4935f22
read from volume index file directly instead of open a separate file
...
fix https://github.com/chrislusf/seaweedfs/issues/1640
read from volume index file directly instead of open a separate file,
to ensure reading latest index entries.
2020-11-27 16:18:48 -08:00
Chris Lu
85554bea38
filer: readonly handle static resources
...
fix https://github.com/chrislusf/seaweedfs/issues/1641
2020-11-27 13:42:14 -08:00
Chris Lu
6d30b21b10
volume: add "-dir.idx" option for separate index storage
...
fix https://github.com/chrislusf/seaweedfs/issues/1265
2020-11-27 03:17:10 -08:00
Chris Lu
3c229eb677
refactor
2020-11-26 17:21:55 -08:00
Chris Lu
c0fb4a3766
add comments
2020-11-26 15:22:42 -08:00
Chris Lu
6783ae5f8f
rename
2020-11-26 15:19:43 -08:00
Chris Lu
983fd90010
marked as WIP
2020-11-26 15:08:52 -08:00
Chris Lu
97f706f35b
just refactoring
2020-11-26 15:08:05 -08:00
Chris Lu
b11449f955
pass in dir.idx parameter to DiskLocation
2020-11-26 14:59:03 -08:00
Chris Lu
0d345ac97d
s3: remove empty parent folder on delete
...
fix https://github.com/chrislusf/seaweedfs/issues/1637
2020-11-26 12:21:58 -08:00
Chris Lu
cc2bd97ad9
refactor
2020-11-26 11:25:56 -08:00
Chris Lu
0da7ecfd29
go fmt
2020-11-26 11:22:30 -08:00
Chris Lu
effa00ed08
refactor
2020-11-26 11:21:58 -08:00
Chris Lu
513bcd6e0d
filer: avoid duplicated FindEntry for deletion
2020-11-26 11:14:56 -08:00
Chris Lu
1ae108efca
add debug message
2020-11-26 10:41:10 -08:00
Chris Lu
a102157020
do not retry if failed with a non transport related error
2020-11-26 01:29:30 -08:00
Konstantin Lebedev
4e55baf5b1
s3 config changes
2020-11-26 03:50:53 +05:00
Konstantin Lebedev
52c8f2fc9a
s3iam test
2020-11-26 02:26:45 +05:00
Konstantin Lebedev
a26f1b2040
new pkg s3iam
2020-11-26 01:30:11 +05:00
Konstantin Lebedev
6206737df2
s3 configure
2020-11-25 21:02:31 +05:00
Chris Lu
eab53ea80d
filer leveldb store: a bit more efficient directory listing with prefix
2020-11-22 21:10:41 -08:00
Chris Lu
d3be2ceedd
2.12
2020-11-22 17:17:33 -08:00
Chris Lu
c7ebadc25d
avoid possible concurrent access inside ensureCorrectWritables()
2020-11-22 17:15:59 -08:00
hilimd
9aa990f80f
Merge pull request #39 from chrislusf/master
...
sync
2020-11-22 20:29:17 +08:00
limd
e8296104fc
fix tls grpc ca path
2020-11-22 20:27:15 +08:00
Chris Lu
92f906b6fc
remove zstd
...
fix https://github.com/chrislusf/seaweedfs/issues/1629
2020-11-21 13:06:45 -08:00
Aleksandr Demshin
176950ea78
fix typo
2020-11-20 19:50:46 +05:00
Chris Lu
7385103057
fix volume placement validating
...
fix https://github.com/chrislusf/seaweedfs/issues/1626
2020-11-20 01:12:25 -08:00
Konstantin Lebedev
27e73de797
Merge branch 'upstream_master' into store_s3cred
...
# Conflicts:
# weed/s3api/filer_util.go
2020-11-19 18:16:44 +05:00
Chris Lu
290b5e2cd0
directly delete file chunks
...
keeping current async deletions for now
2020-11-17 17:20:21 -08:00
Chris Lu
781585b195
remove unnecessary checking
2020-11-17 17:00:06 -08:00
Chris Lu
8cb8cd4cc5
add locks
2020-11-17 16:59:48 -08:00
Chris Lu
7c9e592c2d
s3: avoid nil resp when having error
...
fix https://github.com/chrislusf/seaweedfs/issues/1622
2020-11-17 11:23:17 -08:00
Chris Lu
dc304342b2
fs.configure: configurable volume growth
2020-11-17 01:00:02 -08:00
Chris Lu
b04375a3d1
use alternative assign request
2020-11-17 00:36:21 -08:00
Chris Lu
9add554feb
fail fast if user configures collection for a bucket
2020-11-16 19:57:08 -08:00
Chris Lu
6204dc6e85
add comment
2020-11-16 16:57:31 -08:00
Chris Lu
ef3e068c51
deprecating "filer.options.buckets_fsync" in filer.toml
2020-11-16 16:56:49 -08:00
Chris Lu
5f19e81dab
filer confi: support hierachical configuration
2020-11-16 16:50:12 -08:00
Chris Lu
ed1ce3f299
adjust help message
2020-11-16 09:54:26 -08:00
Chris Lu
c0d279c54e
filere.conf: prefer to use json format
2020-11-15 21:48:17 -08:00
Chris Lu
83527a8f55
adds more help message
2020-11-15 21:31:14 -08:00
Chris Lu
af658ea970
fix typo
2020-11-15 21:16:28 -08:00
Chris Lu
98827d68d9
2.11
2020-11-15 20:25:56 -08:00
Chris Lu
ee2fa14dbe
filer conf: delete location specific configuration
2020-11-15 20:15:47 -08:00
Chris Lu
2bd6fd3bbe
remove unused function
2020-11-15 20:15:07 -08:00
Chris Lu
71056dae07
fs.configure: read and local add filer configuration
2020-11-15 18:09:35 -08:00
Chris Lu
0ea5c087ce
go fmt
2020-11-15 16:59:28 -08:00
Chris Lu
95c0de285d
refactoring
2020-11-15 16:58:48 -08:00
Chris Lu
500bcab953
refactoring
2020-11-15 14:41:56 -08:00
Chris Lu
590f02179d
filer: load filer conf when starting
2020-11-15 14:06:03 -08:00
Chris Lu
0a406f652e
load filer conf and match by prefix
2020-11-15 00:26:05 -08:00
Chris Lu
68043cfcac
add reference implementation to detect create/update/delete/rename events
2020-11-14 21:21:58 -08:00
Chris Lu
0fc9ffc603
able to read filer.conf if updated
2020-11-14 21:21:20 -08:00
Chris Lu
442e092995
filer watch "/etc" folder for configuration changes
2020-11-14 14:26:08 -08:00
ruitao.liu
a9990a1dc6
adjust check bucket if exist or has access.
2020-11-13 17:13:20 +08:00
Chris Lu
e6333da65a
enable admin to access all buckets
2020-11-12 13:57:54 -08:00
Chris Lu
7094492428
fix ErrNotFound
2020-11-12 13:30:46 -08:00
Chris Lu
c6a0704019
rename function
2020-11-12 13:30:08 -08:00
Chris Lu
ed3b0f81c6
Merge pull request #1613 from taozix/master
...
check permission for bucket delete/head.
2020-11-12 13:25:28 -08:00
ruitao.liu
c4f0fd6e1b
skip if entry.Extended map is nil.
2020-11-12 17:59:31 +08:00
ruitao.liu
ab966410d2
return NoSuchBucket instead of InternalError delete non-existed bucket.
2020-11-12 16:44:16 +08:00
Chris Lu
a4dfa00030
adjust filer conf
2020-11-12 00:39:59 -08:00
Chris Lu
2251a8e472
minor
2020-11-12 00:38:58 -08:00
Chris Lu
5a16f17e47
remove unused message type
2020-11-12 00:38:23 -08:00
ruitao.liu
e06676f007
check permission for bucket delete/head.
2020-11-12 16:15:59 +08:00
Chris Lu
c32b6f4d9d
prepare for path based configuration
2020-11-11 23:46:45 -08:00
Chris Lu
0fbc27b640
Merge pull request #1612 from taozix/master
...
add owner attr for bucket.
2020-11-11 22:19:47 -08:00
ruitao.liu
d7cc0498e0
check if bucket already exists.
2020-11-12 14:11:03 +08:00
Chris Lu
827930e249
Merge pull request #1610 from kmlebedev/prefer_read_in_this_dc
...
prefer to read from volumes in this data center
2020-11-11 22:09:41 -08:00
Konstantin Lebedev
0880aff224
update dc flag desc
2020-11-12 10:07:52 +05:00
ruitao.liu
dbba8cb57e
fix if nil map in entry.
2020-11-12 11:50:19 +08:00
Konstantin Lebedev
aa30604cec
fix tests
2020-11-12 08:49:38 +05:00
Konstantin Lebedev
1eec5c8d5d
gen pb
2020-11-12 04:10:06 +05:00
Konstantin Lebedev
fc7baef5bb
fiil serverUrls sorted by data center
2020-11-12 02:13:33 +05:00
Chris Lu
745ee8d8f3
avoid unnecessary error message
...
fix https://github.com/chrislusf/seaweedfs/issues/1611
2020-11-11 13:00:05 -08:00
Chris Lu
a29abd8600
add a {X-,}Idempotency-Key header for http POST
...
fix https://github.com/chrislusf/seaweedfs/issues/1592
according to https://github.com/golang/go/issues/19943
2020-11-11 12:56:53 -08:00
Chris Lu
8708e7a64d
print out data size
2020-11-11 12:51:44 -08:00
Chris Lu
c6bd244ebd
add TODO
2020-11-11 12:51:27 -08:00
Konstantin Lebedev
b73ef6aa95
Merge remote-tracking branch 'upstream/master' into prefer_read_in_this_dc
2020-11-11 23:29:57 +05:00
Konstantin Lebedev
dc26012a3b
initial
2020-11-11 15:03:47 +05:00
Chris Lu
86cdb2a3e5
adjust logs
2020-11-11 02:01:24 -08:00
Chris Lu
73f934d5de
s3: do not close reader too early
...
fix https://github.com/chrislusf/seaweedfs/issues/1609
2020-11-11 01:49:39 -08:00
ruitao.liu
5b636b3242
Add bucket owner attr.
2020-11-11 16:20:59 +08:00
Chris Lu
15956d8a8e
2.10
2020-11-10 23:17:13 -08:00
Chris Lu
de3bdd0651
delete old volume replica
...
related to https://github.com/chrislusf/seaweedfs/issues/1607
old is:
* older compaction revision
* older modified time
* smaller volume size
2020-11-10 12:26:05 -08:00
Chris Lu
1dd3a6ac36
avoid verbose logs
2020-11-09 15:56:11 -08:00
Chris Lu
6856b0d57e
filer: add API to add/modify/delete tagging
2020-11-09 01:00:07 -08:00
Konstantin Lebedev
6cd6ff6962
avoid blank response for 404
2020-11-04 22:54:47 +05:00
Chris Lu
0dafcf1f5a
volume: detect max volume count changes based on disk usage
...
fix https://github.com/chrislusf/seaweedfs/issues/1594
2020-11-03 14:43:17 -08:00
Chris Lu
06ee199ef3
ensure entryViewCache is in sync with the chunks
2020-11-03 12:22:19 -08:00
Chris Lu
b132cb893f
in case some chunks modified earliar are added later
2020-11-03 12:22:19 -08:00
Chris Lu
0ae4d7bfdf
lock "addChunks()"
2020-11-03 12:22:19 -08:00
Konstantin Lebedev
e1190b3224
load S3 config from filer
...
https://github.com/chrislusf/seaweedfs/issues/1500
2020-11-03 21:45:56 +05:00
Chris Lu
d29b787220
store together with "Seaweed-" prefix
2020-11-03 00:21:10 -08:00
Chris Lu
5448781dfc
filer: Headers with "Seaweed-" prefix are stored as extended properties
2020-11-03 00:15:51 -08:00
Chris Lu
0ecff9521d
avoid []byte reuse
2020-11-02 23:27:54 -08:00
Chris Lu
031aa70da6
Revert "revert bytebufferpool, seems problematic"
...
This reverts commit e0fcab47c0
.
2020-11-02 19:11:25 -08:00
Chris Lu
7d5db73d98
go fmt
2020-11-02 14:21:07 -08:00
Chris Lu
a9662d44ac
avoid closing chunkSaveErrChan too early
2020-11-02 14:20:38 -08:00
Chris Lu
e0fcab47c0
revert bytebufferpool, seems problematic
2020-11-02 13:49:38 -08:00
Chris Lu
e1423a83dd
mount: clear possible leftover entry data in memory
2020-11-01 23:28:56 -08:00
Chris Lu
9b10d1ac33
2.08
2020-11-01 12:45:20 -08:00
Chris Lu
8750cac090
move to util.RetryWaitTime
2020-11-01 02:36:43 -08:00
Chris Lu
ef908e166b
break import cycle
2020-11-01 02:28:11 -08:00
Chris Lu
4609644658
mount: remove files that may still are open
...
related to https://github.com/chrislusf/seaweedfs/issues/1581
2020-11-01 01:33:26 -08:00
Chris Lu
df8d976bb0
refactoring
2020-11-01 01:58:48 -07:00
Chris Lu
7192a378cc
add retry to master clients also
2020-11-01 01:40:16 -07:00
Chris Lu
004aa9ca5d
only retry transport related errors
2020-11-01 01:21:44 -07:00
Chris Lu
f7b21973ac
mount: add retry for all operations with filer
...
fix https://github.com/chrislusf/seaweedfs/issues/1589
2020-11-01 01:11:08 -07:00
Chris Lu
de86945aeb
go fmt
2020-10-31 16:45:38 -07:00
Chris Lu
306062b4e7
filer: add CORS support
2020-10-31 16:44:03 -07:00
Chris Lu
05acc33294
volume: add CORS support
2020-10-31 16:31:39 -07:00
Chris Lu
9708df47d1
ensure offset in the index file is positive
2020-10-31 15:50:01 -07:00
Chris Lu
10f9f1cc26
handle mknod requests
2020-10-30 23:51:32 -07:00
Chris Lu
8826601be1
mount: optional limit for the number of concurrent writers
2020-10-30 21:22:20 -07:00
Chris Lu
be95f68ca7
scaffold for sftpd
2020-10-30 20:57:08 -07:00
Konstantin Lebedev
d2a4ffdd60
glog
2020-10-31 03:21:54 +05:00
Konstantin Lebedev
46303c36bf
When the volume server is stopped, the master server immediately sees the deletion of volumes
2020-10-31 02:48:25 +05:00
Chris Lu
eb32af25d4
make space for the QR code
2020-10-30 10:55:31 -07:00
Chris Lu
7ecbb4b3c9
clean up
2020-10-30 10:46:31 -07:00
Kenny
9cc6575ab5
Update filer_copy.go
...
修改 weed filer.copy 上传目录时文件名变子目录的问题
2020-10-30 18:16:49 +08:00
Chris Lu
c057dd1f64
add back upload retries
2020-10-30 02:16:34 -07:00
Chris Lu
19098f2c2d
make a copy of the filer.entry for most read operations
2020-10-30 01:23:26 -07:00
Chris Lu
6135cbaa02
clear file.entry
2020-10-30 01:23:26 -07:00
Chris Lu
02dc51b1ed
properly clear out the file.entry
2020-10-29 23:31:06 -07:00
Chris Lu
6560ac6466
volume loading: trim out unreachable idx file content
...
fix https://github.com/chrislusf/seaweedfs/issues/1583
2020-10-29 22:25:23 -07:00
Chris Lu
0b68b68ec4
reload entry only when it is a hard link
...
fix https://github.com/chrislusf/seaweedfs/issues/1581
2020-10-29 16:32:45 -07:00
Chris Lu
47047516d6
Revert "increase default volume file size limit to 1024"
...
This reverts commit 0983060a
I am confused why this was changed to 1024 by myself. 1GB should be too large for most cases.
2020-10-29 15:46:29 -07:00
Chris Lu
6f8b426f4f
Merge pull request #1578 from taozix/master
...
S3 bucket list, response with uploaded storageclass.
2020-10-29 08:32:27 -07:00
ruitao.liu
22a9ea0512
adjust s3 header file.
2020-10-29 16:05:40 +08:00
Chris Lu
c6d9974fe7
server: Add option to enable/disable volume server
...
fix https://github.com/chrislusf/seaweedfs/issues/1572
2020-10-29 00:24:18 -07:00
ruitao.liu
b917be7955
S3 bucket list, response with uploaded storageclass.
2020-10-29 14:57:19 +08:00
Konstantin Lebedev
918546bbdb
add WaitForReady to grpc DialOption
2020-10-29 08:29:04 +05:00
Konstantin Lebedev
fab01f9d8d
add readonly
2020-10-28 23:17:03 +05:00
Konstantin Lebedev
884db215a1
add shell command volume mark writable
2020-10-28 22:47:09 +05:00
Konstantin Lebedev
1ad1b8c4f6
collection Volume ReadOnly Count with detailed status
2020-10-28 17:38:26 +05:00
Chris Lu
326fcdd86b
Merge pull request #1569 from taozix/master
...
save s3 metadata to filer.
2020-10-28 03:27:24 -07:00
ruitao.liu
2dcc178d0a
save s3 metadata to filer.
2020-10-28 18:16:05 +08:00
Chris Lu
744c5594da
volume: automatically trim out unreachable entries
2020-10-28 01:14:39 -07:00
Chris Lu
3e925faddd
trim out name extension
2020-10-28 01:14:05 -07:00
shibinbin
a67eb1afa1
fix: restart volumeserver sometime occurred out of memory error
2020-10-28 12:03:06 +08:00
Chris Lu
53c3aad875
volume: add a note file to avoid incomplete volume files
...
fix https://github.com/chrislusf/seaweedfs/issues/1567
2020-10-27 15:56:49 -07:00
Chris Lu
6da87720eb
shorter tag prefix
2020-10-27 15:33:26 -07:00
Chris Lu
24bf142596
copy large file first
2020-10-27 15:31:15 -07:00
Chris Lu
432c6f8db3
Merge pull request #1566 from taozix/master
...
return x-amz-tag-count header when GET object.
2020-10-27 13:16:58 -07:00
Chris Lu
06c15ab35c
volume: add special handling for .dat larger than 32GB
2020-10-27 13:11:56 -07:00
ruitao.liu
72f6b77dea
fix tag prefix.
2020-10-27 18:01:37 +08:00
ruitao.liu
8766ca1b95
rename s3 tag prefix.
2020-10-27 17:33:24 +08:00
ruitao.liu
19026ae55d
return x-amz-tag-count header when GET object.
2020-10-27 16:49:31 +08:00
Chris Lu
44921220b0
2.07
2020-10-25 22:03:46 -07:00
Chris Lu
e71463a9eb
mount: invalide file cache when metadata is changed
2020-10-25 19:24:15 -07:00
Chris Lu
e219c57849
passing full path when assign volume locations
2020-10-25 15:46:29 -07:00
Chris Lu
f375b93aef
renaming
2020-10-25 15:32:43 -07:00
Chris Lu
63b0fb54f7
minor
2020-10-25 14:15:53 -07:00
Chris Lu
27ba0a2be3
2.06
2020-10-25 12:36:47 -07:00
Chris Lu
147d4d23b6
filer.copy: automatically use the right bucket
2020-10-25 11:21:33 -07:00
Chris Lu
91fd311f7a
fix logic to read entry or not
2020-10-25 10:33:51 -07:00
Chris Lu
10a4a628e9
refresh cached file entry from sync metadata updates
2020-10-25 09:42:50 -07:00
Konstantin Lebedev
2fb1fce8a0
avoid old values when missing read only flags
2020-10-25 16:21:40 +05:00
Chris Lu
da36abf033
go fmt
2020-10-24 20:12:04 -07:00
Chris Lu
16fae84414
fix format error
2020-10-24 20:11:31 -07:00
Chris Lu
b7b1eb51b1
default to 2 for better performance in common computer hardware
2020-10-24 20:08:26 -07:00
Chris Lu
79690b9cd8
revert changes causing wrong data
2020-10-24 20:07:19 -07:00
Chris Lu
9104cfa744
reduce locks
2020-10-24 19:40:35 -07:00
Chris Lu
e9d40b80b2
less verbose logs
2020-10-24 09:42:54 -07:00
Chris Lu
6c4fb243a7
remove limits to concurrent writers
2020-10-24 02:05:05 -07:00
Chris Lu
e0002f8dd7
check existing volumes for writable status
2020-10-24 01:34:31 -07:00
Chris Lu
19772d70d7
print for debugging
2020-10-24 00:12:02 -07:00
Chris Lu
5b92dfe302
adjust log level
2020-10-23 23:58:54 -07:00
Chris Lu
fb124f2d4a
conditionally display volume server public url link
2020-10-23 23:58:46 -07:00
Chris Lu
1b17f71939
adjust election timeout to 10 seconds
2020-10-23 23:06:44 -07:00
Chris Lu
ef9c32ea0d
adjust compression threshold
2020-10-23 23:05:55 -07:00
Chris Lu
0cea84e7ab
make it easy with setting up multiple masters in a clcuster
2020-10-23 18:18:46 -07:00
Chris Lu
fc689319ae
add volume public url
2020-10-23 18:03:23 -07:00
Chris Lu
71624858af
avoid reusing cached []byte
...
this logic is just to ensure 100% correct.
fuse message, data+message hearder > 512 will not be cached.
2020-10-23 11:31:57 -07:00
Chris Lu
29c9fa2ef2
byte buffer for uploading
2020-10-23 11:29:51 -07:00
Chris Lu
2579edbc60
fix bug found by tests
2020-10-23 10:05:40 -07:00
Chris Lu
e0c8507d93
memory pool to reduce memory allocation
2020-10-23 00:41:40 -07:00
Chris Lu
b81359823f
postgres: support empty user
2020-10-22 14:27:47 -07:00
Chris Lu
11716fbf6f
make reading error more obvious
2020-10-22 13:33:45 -07:00
Chris Lu
d75bc62196
to read files that are quickly changing
2020-10-22 12:59:15 -07:00
Chris Lu
575d7952a1
add available resource stats
...
fix https://github.com/chrislusf/seaweedfs/issues/1555
2020-10-22 09:13:47 -07:00
Chris Lu
5179e559f7
skip empty logs
2020-10-22 00:35:48 -07:00
Chris Lu
de044e466e
Merge pull request #1553 from kmlebedev/multi_domainname
...
Multi domainname
2020-10-21 23:56:03 -07:00
Chris Lu
e73d6c9526
able to print partial file id
...
address https://github.com/chrislusf/seaweedfs/issues/1552
2020-10-21 23:48:07 -07:00
Konstantin Lebedev
a9a7005687
set desc of option
2020-10-22 11:23:00 +05:00
Chris Lu
720b1d9b88
adding locking to avoid nil VolumeLocationList
...
fix panic: runtime error: invalid memory address or nil pointer dereference
Oct 22 00:53:44 bedb-master1 weed[8055]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x17658da]
Oct 22 00:53:44 bedb-master1 weed[8055]: goroutine 310 [running]:
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/topology.(*VolumeLocationList).Length(...)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/topology/volume_location_list.go:35
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/topology.(*VolumeLayout).enoughCopies(...)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/topology/volume_layout.go:376
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/topology.(*VolumeLayout).ensureCorrectWritables(0xc000111d50, 0xc000b55438)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/topology/volume_layout.go:202 +0x5a
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/topology.(*Topology).SyncDataNodeRegistration(0xc00042ac60, 0xc001454d30, 0x1, 0x1, 0xc0005fc000, 0xc00135de40, 0x4, 0xc00135de50, 0x10, 0x10d, ...)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/topology/topology.go:224 +0x616
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/server.(*MasterServer).SendHeartbeat(0xc000162700, 0x23b97c0, 0xc000ae2c90, 0x0, 0x0)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/server/master_grpc_server.go:106 +0x325
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/pb/master_pb._Seaweed_SendHeartbeat_Handler(0x1f8e7c0, 0xc000162700, 0x23b0a60, 0xc00024b440, 0x3172c38, 0xc000ab7100)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/pb/master_pb/master.pb.go:4250 +0xad
Oct 22 00:53:44 bedb-master1 weed[8055]: google.golang.org/grpc.(*Server).processStreamingRPC(0xc0001f31e0, 0x23bb800, 0xc000ac5500, 0xc000ab7100, 0xc0001fea80, 0x311fec0, 0x0, 0x0, 0x0)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:1329 +0xcd8
Oct 22 00:53:44 bedb-master1 weed[8055]: google.golang.org/grpc.(*Server).handleStream(0xc0001f31e0, 0x23bb800, 0xc000ac5500, 0xc000ab7100, 0x0)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:1409 +0xc5c
Oct 22 00:53:44 bedb-master1 weed[8055]: google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0001ce8b0, 0xc0001f31e0, 0x23bb800, 0xc000ac5500, 0xc000ab7100)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:746 +0xa5
Oct 22 00:53:44 bedb-master1 weed[8055]: created by google.golang.org/grpc.(*Server).serveStreams.func1
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:744 +0xa5
Oct 22 00:53:44 bedb-master1 systemd[1]: weedmaster.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Oct 22 00:53:44 bedb-master1 systemd[1]: weedmaster.service: Failed with result 'exit-code'.
2020-10-21 23:15:48 -07:00
Chris Lu
2bc05cfbc8
fix typo
2020-10-21 22:26:41 -07:00
Chris Lu
4ff2ceee33
UI fix on rendering EC volumes
...
addressing UI problem with https://github.com/chrislusf/seaweedfs/issues/1551
2020-10-21 22:05:58 -07:00
Chris Lu
f5ee19e0db
shortcut for appending operations
2020-10-21 20:44:01 -07:00
Chris Lu
b5917bf846
clean up
2020-10-21 19:31:26 -07:00
Chris Lu
9884dfc369
use bytebuffer for interval list
2020-10-21 19:29:51 -07:00
Chris Lu
bd103c143a
add lock for vidCache
2020-10-21 19:28:59 -07:00
Konstantin Lebedev
05b5f12f2e
multiplate DomainNames through comma
2020-10-21 20:48:51 +05:00
Chris Lu
81cf8d04df
minor
2020-10-21 02:42:23 -07:00
Chris Lu
d90e7ee3cc
adjust log level
2020-10-21 02:17:40 -07:00
Chris Lu
3bf0116de1
mount: less channel waiting
2020-10-21 02:16:21 -07:00
Chris Lu
c31b254248
mount: shortcut when there is only one chunk
2020-10-20 23:48:29 -07:00
Chris Lu
f64252023e
Revert "a better byte buffer pool"
...
This reverts commit 59c6422777
.
2020-10-20 23:21:54 -07:00
Chris Lu
ad652ffb97
avoid close closed channel
2020-10-20 22:54:34 -07:00
Chris Lu
20442dc1bc
avoid extra data copying
2020-10-20 22:54:21 -07:00
Chris Lu
59c6422777
a better byte buffer pool
2020-10-20 22:53:58 -07:00
Chris Lu
6c92fa349d
filer: support default rack
...
fix https://github.com/chrislusf/seaweedfs/issues/1546
2020-10-20 17:41:39 -07:00
Chris Lu
b3aa2fab9a
s3: report error to s3 when updating an object but it is already a directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1545
2020-10-20 10:25:16 -07:00
Chris Lu
410b818aa7
master: avoid timer leakage
2020-10-19 14:24:57 -07:00
Chris Lu
69f0da5d73
mount: pass along replication and collection parameters
2020-10-19 00:12:42 -07:00
Chris Lu
c0ab458671
report nil instead of EOF for empty files
...
related to https://github.com/chrislusf/seaweedfs/issues/1541
2020-10-17 11:03:46 -07:00
Chris Lu
898db14729
2.05
2020-10-17 02:42:33 -07:00
Chris Lu
09bab17aff
mount: avoid "send on closed channel"
2020-10-16 12:54:23 -07:00
Chris Lu
9d80a3428c
add debug_webdav
2020-10-16 12:53:56 -07:00
Chris Lu
37f165d743
webdav: return io.EOF when at end of a file
...
fix https://github.com/chrislusf/seaweedfs/issues/1344
2020-10-16 12:53:37 -07:00
Chris Lu
68d39c86f1
mysql, postgres, cassandra: change kv key to base64 encoding
...
The exisitng key-value operation for stores using mysql, postgres, and maybe cassandra are already broken.
The kv is used to store hardlink, filer store signature and replication progress.
So users using hardlink and also uses mysql, postgres, or cassandra will have broken hard links.
Users using filer.sync will need to re-sync the files.
2020-10-16 11:10:12 -07:00
Chris Lu
ee1fc6558a
refactor
2020-10-16 11:02:44 -07:00
Chris Lu
06e6341097
set initial write time to avoid race condition with delayed chunks
2020-10-15 11:08:45 -07:00
Chris Lu
d598c47136
log errors
2020-10-15 11:01:43 -07:00
Chris Lu
5039aff310
release the goroutine to read errors
2020-10-15 10:59:18 -07:00
Chris Lu
ace0ea3d28
s3: avoid duplicated bucket
2020-10-15 10:52:20 -07:00
Konstantin Lebedev
dc2e13092d
add number of read only volumes metric
2020-10-15 15:48:40 +05:00
Chris Lu
e714c28a02
mount: async write file chunk
2020-10-14 23:28:03 -07:00
Chris Lu
c95d3d9dff
adjust tests
2020-10-14 12:27:52 -07:00
Chris Lu
93bcf56514
file read report EOF
...
fix https://github.com/chrislusf/seaweedfs/issues/1344
2020-10-14 12:18:24 -07:00
Chris Lu
1069b325dd
shell: volumeServer.evacuate adds printout for ec volumes
2020-10-13 20:26:03 -07:00
Chris Lu
0542911e29
go fmt
2020-10-13 19:50:46 -07:00
Chris Lu
58fa506491
minor
2020-10-13 19:50:22 -07:00
Chris Lu
28d4e1a51b
mount: retry for directory listing with filer
...
related to https://github.com/chrislusf/seaweedfs/issues/1530
2020-10-13 19:49:52 -07:00
Chris Lu
c127da1219
filer: linearize timeout for large chunk of data
2020-10-13 14:04:46 -07:00
Chris Lu
aac4cb1f0c
adds errror on read and write
2020-10-13 13:53:34 -07:00
Chris Lu
9b4f7fed14
mount: report filer IO error
...
related to https://github.com/chrislusf/seaweedfs/issues/1530
2020-10-13 11:21:13 -07:00
Chris Lu
3f7d1d1bf1
Only wait on retryable requests
2020-10-13 00:29:46 -07:00
Chris Lu
b18f21cce1
mount: fix bound tree with filer.path
...
fix https://github.com/chrislusf/seaweedfs/issues/1528
2020-10-12 21:58:37 -07:00
Chris Lu
f022aff289
add back http.StatusPartialContent
...
revert e7c04af1d0
2020-10-12 12:26:25 -07:00
Chris Lu
bbd0afd37e
2.04
2020-10-11 21:25:30 -07:00
Chris Lu
7704469d29
mount: outsideContainerClusterMode changed to use volume server publicUrl
2020-10-11 20:42:15 -07:00
Chris Lu
723ae11db4
refactoring in order to adjust volume server url later
2020-10-11 20:15:10 -07:00
Chris Lu
d155f907c2
mount: configurable read wait time
2020-10-10 20:09:43 -07:00
Chris Lu
9b0e8ef026
filer: added QR code to transfer files with mobile devices
2020-10-10 18:00:21 -07:00
Chris Lu
8a52379ecb
add retry if volume can not be found
2020-10-10 16:02:39 -07:00
Chris Lu
70af0ec24c
filer: fix hanging on read
2020-10-10 16:02:10 -07:00
Chris Lu
cff8bb6554
return proper error
2020-10-10 15:43:22 -07:00
Chris Lu
6da8eef54f
fix logic error
2020-10-09 00:01:47 -07:00
Chris Lu
5d01dd28a6
add logging
2020-10-08 23:38:00 -07:00
Chris Lu
8d34eb0050
mount:exponentially backoff if read error for about 10 minutes
2020-10-08 23:31:26 -07:00
Chris Lu
6e1f936efd
refactoring
2020-10-08 23:19:42 -07:00
Chris Lu
b2ee5873fb
fix error not being returned
2020-10-08 23:19:20 -07:00
Chris Lu
b067853162
mount: ignore recursion error
...
fix https://github.com/chrislusf/seaweedfs/issues/1514
the host OS should manage recursive deletion.
2020-10-08 21:55:55 -07:00
Chris Lu
e1a8e3f900
fix logging
2020-10-08 18:40:38 -07:00
Chris Lu
8676db0f6a
adjust logging
2020-10-08 18:33:37 -07:00
Chris Lu
bd8b9b0c2e
some changes to prepare fix after failed reads
2020-10-08 18:33:06 -07:00
Chris Lu
78859e35fc
s3: adjust config examples
2020-10-08 10:12:09 -07:00
Chris Lu
9e7a2772b1
s3: only admin can list all buckets
2020-10-08 10:12:09 -07:00
Chris Lu
eed492b73b
randomize file locations
2020-10-07 23:58:32 -07:00
Chris Lu
ec08a1670b
volume: avoid hanging when stopping a volume server
2020-10-07 23:48:24 -07:00
Chris Lu
62d3d3aea0
go fmt
2020-10-07 23:30:54 -07:00
Chris Lu
e91b9c85a2
s3: Added support for "List" action in weed s3 -config=... in the config file.
...
fix https://github.com/chrislusf/seaweedfs/issues/1511
2020-10-07 23:22:35 -07:00
Chris Lu
a8624c2e4f
read from alternative replica
...
related to https://github.com/chrislusf/seaweedfs/issues/1512
2020-10-07 22:49:04 -07:00
Chris Lu
da4edf3651
master: check peers for existing leader before starting a leader election
...
fix https://github.com/chrislusf/seaweedfs/issues/1509
2020-10-07 01:25:39 -07:00
Chris Lu
c543762e23
volume: auto expands -max setting
...
I have tried to run weed volume but got errors...
cmd:
weed volume -max=0 -mserver="127.0.0.1:9333" -port=8080 -dir="/var/www/d1/sfs,/var/www/d2/sfs,/var/www/d3/sfs"
log:
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d1/sfs Permission: -rwxr-xr-x
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d2/sfs Permission: -rwxr-xr-x
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d3/sfs Permission: -rwxr-xr-x
F1006 15:09:26 17040 volume.go:142] 3 directories by -dir, but only 1 max is set by -max
What am I doing wrong? Thx.
2020-10-06 09:05:30 -07:00
Chris Lu
36492c47ec
adjust
2020-10-05 14:06:18 -07:00
Chris Lu
8f8e9ddbde
adjust API
2020-10-05 09:47:07 -07:00
Konstantin Lebedev
e4f2d9eb4a
We return etag using the same algorithm as aws s3
...
https://teppen.io/2018/06/23/aws_s3_etags/
2020-10-05 14:43:32 +05:00
Chris Lu
4fc673341f
2.03
2020-10-04 21:58:22 -07:00
Chris Lu
6b591b02af
adjust log level
2020-10-04 16:21:43 -07:00
Устюжанин Антон Александрович
702b1cb876
fix: remove deleted peers if resumeState = true
2020-10-04 21:56:17 +05:00
Chris Lu
d522df0ecd
adjust log level
2020-10-04 01:35:32 -07:00
Chris Lu
666859f7cd
mount: read prefetching
2020-10-04 01:31:04 -07:00
Chris Lu
bc47835997
mount: avoid duplicated reads
2020-10-03 20:16:42 -07:00
Chris Lu
8d65ad1444
Revert "mount: adds read prefetching"
...
This reverts commit 692f0614d1
.
2020-10-03 19:40:56 -07:00
Chris Lu
692f0614d1
mount: adds read prefetching
2020-10-03 16:50:09 -07:00
Chris Lu
707936f482
re-enable caching larger than 16MB
...
revert 62ce85610e
2020-10-03 14:12:38 -07:00
Chris Lu
2c20ef72ae
fix starting problem
2020-10-03 13:56:38 -07:00
Chris Lu
d77e3c96e9
FUSE: use the mount directory permissions
2020-10-03 13:37:35 -07:00
Chris Lu
95ffed87c0
Merge pull request #1505 from ustuzhanin/remove_raft_state
...
Resume raft state
2020-10-03 10:38:47 -07:00
Устюжанин Антон Александрович
6fbfe73683
fix: resumeState
2020-10-03 22:16:47 +05:00
Устюжанин Антон Александрович
dc31b19469
fix: restore raft state
2020-10-03 14:03:41 +05:00
Chris Lu
f781cce500
s3: support object tagging
...
* GetObjectTagging
* PutObjectTagging
* DeleteObjectTagging
2020-10-02 22:21:51 -07:00
Устюжанин Антон Александрович
8c82fb7e5f
fix: restore raft state
2020-10-02 23:01:20 +05:00
Chris Lu
9ab98fa912
s3 metrics adjust the label
2020-10-01 07:17:47 -07:00
Chris Lu
f1e879fe7a
2.02
2020-10-01 07:15:49 -07:00
Chris Lu
a1c01d716b
volume: avoid deadlock when deleting volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/1501
2020-10-01 07:10:03 -07:00
Konstantin Lebedev
86329bbf2b
label name is statusCode
2020-10-01 01:22:38 +05:00
Konstantin Lebedev
68463e92c1
add status code in S3RequestCounter
2020-10-01 00:59:39 +05:00
Chris Lu
79ab10e300
adjust help message
2020-09-30 09:32:00 -07:00
Chris Lu
55cb68c09c
filer: replication follows master config if not specified
2020-09-30 09:15:55 -07:00
Chris Lu
b9887504e8
fix test
2020-09-27 23:19:50 -07:00
Chris Lu
f46eae284e
adjust for test
2020-09-27 23:08:11 -07:00
Chris Lu
75aca5e13d
2.01
2020-09-27 23:02:41 -07:00
Chris Lu
e6552b5e1e
filer: able to start s3 together
2020-09-27 23:00:43 -07:00
Chris Lu
474e2b6ac3
add a hard link marker to 16byte + maker, for future extensions
2020-09-27 22:38:30 -07:00
Chris Lu
c49e2bb9a3
adjust
2020-09-27 12:07:45 -07:00
Chris Lu
62ce85610e
skip caching too large chunks
2020-09-27 11:58:48 -07:00
Chris Lu
9ad2dcca2b
more tests
2020-09-27 11:42:51 -07:00
Chris Lu
e43d86c796
fix pre allocated volume size
2020-09-27 10:58:19 -07:00
Chris Lu
31fc7bb2e1
refactor
...
adjust for faster test
2020-09-27 10:41:29 -07:00
Chris Lu
a37535cd9f
avoid non utf-8 in filename
...
fix https://github.com/chrislusf/seaweedfs/issues/1493
2020-09-26 00:13:39 -07:00
Chris Lu
41be7a4c8f
filer: upload also set file size
...
ensure works same as fuse mount
2020-09-25 12:02:06 -07:00
Chris Lu
35f6518c36
weed upload: usePublicUrl was not being used on big files
...
fix https://github.com/chrislusf/seaweedfs/issues/1492
2020-09-25 02:41:38 -07:00
Chris Lu
22049dc2e6
Merge pull request #1491 from hilimd/master
...
s3: fix get object metadata problem
2020-09-25 02:35:00 -07:00
limd
402aef8f30
s3:
...
1.fix spark reading S3 directory wildcard problem
2.fix the problem of the spark history service writing S3 directory
reference git revsion number: b41b7ea4d0
2020-09-25 14:37:02 +08:00
Chris Lu
8516517c48
filer store: Cassandra supports username/password
2020-09-24 21:31:06 -07:00
limd
ed7816681a
Add cassandra authenticator mode
2020-09-25 11:22:47 +08:00
limd
370a98cf6b
code style
2020-09-25 11:19:42 +08:00
limd
aee3fd08b9
code style
2020-09-25 11:04:32 +08:00
Chris Lu
043b063136
Merge pull request #1482 from hilimd/master
...
Fix: s3 delete object
2020-09-24 18:21:34 -07:00
limd
48c578410f
mount: rollback default value
2020-09-25 09:18:52 +08:00
Chris Lu
dbf5327b98
s3: handle response-content-encoding
...
fix https://github.com/chrislusf/seaweedfs/issues/1487
2020-09-24 18:09:52 -07:00
Chris Lu
7726965a47
minor
2020-09-24 11:20:12 -07:00
Chris Lu
0790c6d605
fix empty hard link id
2020-09-24 11:16:43 -07:00
Chris Lu
1012df7bb5
switch hardlink id from int64 to bytes
2020-09-24 11:11:42 -07:00
Chris Lu
4856bce0ee
adjust for metrics port
2020-09-24 10:21:23 -07:00
Chris Lu
d2d3aec3e1
consolidate to one metricsPort in "weed server" mode
2020-09-24 09:55:02 -07:00
Chris Lu
2e9099369e
Merge pull request #1489 from kmlebedev/promhttp
...
Promhttp
2020-09-24 09:50:48 -07:00
Chris Lu
2e7c361a0d
hardlink deletion factors in hardlink counter
2020-09-24 09:43:52 -07:00
Chris Lu
1295347958
adjust hardlink update
...
simplify logic, pass entity content directly to hard link. The "weed mount" handles the logic to calculate hard link counter.
2020-09-24 09:43:00 -07:00
Konstantin Lebedev
98e9de6e11
fix style
2020-09-24 17:48:39 +05:00
Konstantin Lebedev
324e44d4b3
add start metrics server
2020-09-24 17:45:39 +05:00
limd
cbca14edc5
mount: fix k8s pvc and os mount directory permission bug
2020-09-24 18:07:16 +08:00
Chris Lu
5e239afdfc
hardlink works now
2020-09-24 03:06:48 -07:00
limd
59e91e9c7e
mount: fix k8s pvc mount directory permission
2020-09-24 10:25:45 +08:00
Chris Lu
c7d7b1a0f6
Merge pull request #1485 from LIBA-S/fix_oversized
...
Correct the oversized state of volume after compaction
2020-09-23 19:24:30 -07:00
Chris Lu
b80a2b3cc9
Merge pull request #1486 from LIBA-S/fix_race_condition
...
Fix a race condition when handle VolumeLocationList
2020-09-23 19:24:16 -07:00
LIBA-S
eecd6b5d35
Fix a race condition when handle VolumeLocationList
2020-09-23 20:56:51 +08:00
LIBA-S
0157798ebf
Correct the oversized state of volume after compaction
2020-09-23 20:27:42 +08:00
limd
9b21ec27e4
mount: auto created dir set corrct umask fix bug
2020-09-23 18:35:37 +08:00
hilimd
0db149fb5f
Merge pull request #18 from chrislusf/master
...
sync
2020-09-23 17:32:05 +08:00
Chris Lu
f7a0ccb595
mount: auto created dir set corrct umask
2020-09-23 02:31:19 -07:00
Chris Lu
f0e325b6fc
mount: auto created directory follow umask
2020-09-23 02:27:57 -07:00
limd
d506080c36
rollback
2020-09-23 14:29:53 +08:00
limd
8f9f29b773
fixed the problem of empty directory when S3 deleted the directory
2020-09-23 13:33:13 +08:00
limd
1892677b22
fixed the problem of empty directory when S3 deleted the directory
2020-09-23 13:15:06 +08:00
hilimd
85570f27a0
Merge pull request #17 from chrislusf/master
...
sync
2020-09-23 11:39:57 +08:00
limd
a99f63cb4d
1.add S3 copy directory function
...
2.fixed the problem of empty directory when S3 deleted the directory
2020-09-23 11:26:01 +08:00
Chris Lu
b61d33f251
expose only store wrapper to meta changes
2020-09-22 16:27:36 -07:00
Chris Lu
0adbb56cc1
rename
2020-09-22 16:24:13 -07:00
Chris Lu
3e52329cee
Revert "Merge pull request #1479 from LIBA-S/fix_oversized"
...
This reverts commit bd11f0b3e4
, reversing
changes made to ec5b9f1e91
.
2020-09-22 15:05:37 -07:00
Chris Lu
4a1fe4b8e2
add error logs
2020-09-22 09:16:10 -07:00
LIBA-S
8573ac82c6
Fix: remove the oversized state after compaction
2020-09-22 21:48:39 +08:00
James Hartig
658fc2e5b6
Allow option to enable volume pprof on server
2020-09-21 22:43:10 -04:00
James Hartig
91e4eca1e9
Fix deadlock with KeepConnected and SendHeartbeat
...
There's the potential where we're writing to a clientConn and it goes away
and we're stuck keeping a read lock on clientChansLock. This causes
KeepConnected to not be able to remove the client since it requires a write
lock on clientChansLock. This ends up backing up SendHeartbeat because it
can't get a read lock.
2020-09-21 22:41:38 -04:00
Chris Lu
9cdbfc1a49
refactor
2020-09-21 17:34:38 -07:00
Chris Lu
63373a9f9f
filer: add file extended properties
2020-09-21 11:08:34 -07:00
Chris Lu
b9c1f3e9de
s3: fixes for list multipart upload
2020-09-21 10:51:24 -07:00
Chris Lu
289e62a305
master: better locking of in memory volume data
...
related to https://github.com/chrislusf/seaweedfs/issues/1436#issuecomment-695880135
2020-09-20 23:07:55 -07:00
Chris Lu
9a3b564508
adjust printout
2020-09-20 16:01:56 -07:00
Chris Lu
62563a895a
refactoring
2020-09-20 16:00:01 -07:00
Chris Lu
c2faab23b6
refactor
2020-09-20 15:40:49 -07:00
Chris Lu
d013d09a9b
adjust logging
2020-09-20 15:38:59 -07:00
Chris Lu
f498c71199
shell: move volume operations to use flag parsing arguments
2020-09-20 09:27:34 -07:00
Chris Lu
fcbc520373
fix print out
2020-09-20 08:47:30 -07:00
Chris Lu
56094541c8
shell: adjust command option for collection.delete
2020-09-20 08:46:16 -07:00
Chris Lu
9cac9c4001
2.00
2020-09-20 00:47:04 -07:00
Chris Lu
dc4d2145ff
open up
2020-09-19 20:23:12 -07:00
Chris Lu
29abe980df
s3: add support for PostPolicy
...
fix https://github.com/chrislusf/seaweedfs/issues/1426
2020-09-19 20:14:19 -07:00
Chris Lu
41d508edfd
go fmt
2020-09-19 14:10:26 -07:00
Chris Lu
5b40a2690a
refactoring
2020-09-19 14:09:58 -07:00
Chris Lu
2c21eb1971
volume: get metrics configuration from master
...
fix https://github.com/chrislusf/seaweedfs/issues/1354
2020-09-19 00:03:00 -07:00
Chris Lu
2cbd1cf121
fix compilation
2020-09-18 00:15:54 -07:00
Chris Lu
23e9ede068
s3: collect metrics
2020-09-18 00:09:04 -07:00
Chris Lu
852e5f7cbc
filer: fix mongodb insert
...
fix https://github.com/chrislusf/seaweedfs/issues/1471
2020-09-17 21:50:52 -07:00
Chris Lu
6544e60bea
s3 add metrics
...
empty for now
2020-09-17 06:56:15 -07:00
Chris Lu
cb427d48fa
filer report metrics configuration
2020-09-17 06:46:51 -07:00
Chris Lu
e4e0234998
refactoring
2020-09-17 06:43:54 -07:00
Chris Lu
feca07bf96
Filer: ip bind
...
fix https://github.com/chrislusf/seaweedfs/issues/1470
2020-09-16 10:26:08 -07:00
Chris Lu
e861a6a3ab
simplify metrics configuration logic
2020-09-16 01:39:30 -07:00
Chris Lu
be54eeb364
passing value by checkWithMaster() instead
2020-09-16 01:33:45 -07:00
Chris Lu
4d21de63ee
go fmt
2020-09-16 01:27:24 -07:00
Chris Lu
913a16268d
volume: load configuration from master at the start
...
fix https://github.com/chrislusf/seaweedfs/issues/1469
2020-09-16 01:27:05 -07:00
Chris Lu
c9202c4b3d
add storage backend to GetMasterConfigurationResponse
2020-09-16 01:12:22 -07:00
Chris Lu
8a0710cb73
handle more than 2GB files
...
related to https://github.com/chrislusf/seaweedfs/issues/1468
2020-09-16 00:41:50 -07:00
Chris Lu
854007bc98
minor
2020-09-16 00:37:57 -07:00
Chris Lu
10f9081526
filer: adjust meta data events to received timestamp
...
if a client is already connected and start from t0. A message recieved at t+1 but with timestamp t-1 may not be processed by the client.
This commit changes to the event received time, so the replication can be ordered.
2020-09-15 01:18:33 -07:00
Chris Lu
684a875876
update help message
2020-09-15 00:40:38 -07:00
Chris Lu
2d3a904a82
shell: volumeServer.evacuate adds option to skip non moveable volumes
2020-09-15 00:33:49 -07:00
Chris Lu
a595916342
shell: add volumeServer.evacuate command
2020-09-14 23:47:11 -07:00
Chris Lu
103fafe00b
weed export: print out [start,stop) content range
2020-09-14 22:57:23 -07:00
Chris Lu
ada996fe55
filer: support createing empty folder
...
fix https://github.com/chrislusf/seaweedfs/issues/1161
2020-09-14 13:07:46 -07:00
Chris Lu
200fe5c83e
go fmt
2020-09-13 21:26:30 -07:00
Chris Lu
5d6753fb98
shell: add volumeServer.leave command
2020-09-13 21:25:51 -07:00
Chris Lu
1af95c5b76
refactoring
2020-09-13 12:41:26 -07:00
Chris Lu
f8fea19669
1.99
2020-09-12 13:46:33 -07:00
Chris Lu
b0c7de186d
filer: fix postgres prefixed directory listing problem
...
fix https://github.com/chrislusf/seaweedfs/issues/1465
2020-09-12 13:37:03 -07:00
Chris Lu
1a7afe7e6a
adjust default value
...
related to https://github.com/chrislusf/seaweedfs/issues/1453
2020-09-12 12:46:42 -07:00
Chris Lu
f2723c1bc8
do not idx file format
...
revert c9ab8d05fa
2020-09-12 12:42:36 -07:00
Chris Lu
ba984a4e29
1.98
2020-09-12 04:13:02 -07:00
Chris Lu
446e476a11
go fmt
2020-09-12 04:08:03 -07:00
Chris Lu
ea26a98753
volume: validate volume correctness if last entry is a deletion
2020-09-12 04:07:04 -07:00
Chris Lu
d15682b4a1
shell: volume.balance plan by ratio of fullness
2020-09-12 04:06:26 -07:00
Chris Lu
c0ee78d2fa
adjust make file
2020-09-12 04:05:42 -07:00
Chris Lu
cd9b89ba55
reduce default wait time to 10s before shutting down
2020-09-12 04:05:33 -07:00
Chris Lu
e2c741f76f
adjust replica placement after move
2020-09-12 01:01:19 -07:00
Chris Lu
2a0925590c
filer: etcd store avoid read with nil option
...
fix https://github.com/chrislusf/seaweedfs/issues/1463
2020-09-11 15:29:45 -07:00
Chris Lu
3984c3962f
add comment
2020-09-11 15:07:19 -07:00
Chris Lu
baa6bdf4d4
s3: listMultipartUploads fix output format
2020-09-11 15:04:01 -07:00
Chris Lu
3eda8d6dfc
s3: ListParts output xml format
...
fix https://github.com/chrislusf/seaweedfs/issues/1461
2020-09-11 14:53:50 -07:00
Chris Lu
ab201c2798
1.97
2020-09-11 13:47:50 -07:00
Chris Lu
9d4bdfcfdf
fix volume integrity checking
2020-09-11 11:34:10 -07:00
Chris Lu
1b8094ef75
weed export: export deleted files
2020-09-11 02:05:14 -07:00
Chris Lu
12a8f5294d
test for multi dc replication
2020-09-11 01:21:17 -07:00
Chris Lu
e60b2117c3
shell: volume balance follows replica placement
2020-09-11 00:29:25 -07:00
Chris Lu
89a62e8007
refactoring
2020-09-10 23:05:00 -07:00
Chris Lu
19537c9d21
1.96
2020-09-10 19:52:07 -07:00
Chris Lu
eaf9fdde99
avoid sharing context over separate goroutine
2020-09-10 19:51:03 -07:00
Chris Lu
cc5fe6f5ee
handle special characters in html link
2020-09-10 19:46:02 -07:00
Chris Lu
401ccf1509
fix test
2020-09-10 15:13:23 -07:00
Chris Lu
c9ab8d05fa
fixes for reading deleted fid
2020-09-10 14:42:52 -07:00
ruitao.liu
5b0676049a
change elastic initialize process similar as others.
2020-09-10 23:35:20 +08:00
Chris Lu
660d7c0edd
1.95
2020-09-10 01:32:05 -07:00
Chris Lu
0d9b858cfb
Merge pull request #1455 from taozix/master
...
add more basic elastic options.
2020-09-10 01:31:17 -07:00
ruitao.liu
9be4e97625
change logs print format.
2020-09-10 16:30:15 +08:00
ruitao.liu
719dc43af1
modify elastic urls from string to array.
2020-09-10 16:24:09 +08:00
ruitao.liu
72f9d7f047
use util to generate md5.
2020-09-10 16:11:18 +08:00
ruitao.liu
6a5b38c0d4
fix elastic kv ops.
2020-09-10 15:59:16 +08:00
ruitao.liu
3f7fbfddca
add more basic elastic options.
2020-09-10 14:22:07 +08:00
Chris Lu
b183ae54b5
master: changing 301 to 308 on redirect to volume servers
...
fix https://github.com/chrislusf/seaweedfs/issues/1454
2020-09-09 22:36:10 -07:00
Chris Lu
47b3f932e4
watch: adjust output format
2020-09-09 22:34:48 -07:00
Chris Lu
daf0a449f7
properly cancel context for streaming grpc
2020-09-09 12:07:15 -07:00
Chris Lu
7f69acd1f2
sync pprof
2020-09-09 11:33:52 -07:00
Chris Lu
387ab6796f
filer: cross cluster synchronization
2020-09-09 11:21:23 -07:00
Chris Lu
4fc0bd1a81
return http response directly
2020-09-09 03:53:09 -07:00
Chris Lu
5e13bc878c
adjust log level
2020-09-09 03:49:26 -07:00
Chris Lu
d1b816212f
return nil if no need to do anything
2020-09-07 16:03:05 -07:00
Chris Lu
64a621bcc8
shell: volume.fix.replication also purge over replicated volumes
2020-09-07 16:00:10 -07:00
Chris Lu
c18ea21f7a
1.94
2020-09-07 13:13:53 -07:00
Chris Lu
18b98fdb72
fix test
2020-09-07 12:57:38 -07:00
Chris Lu
44b3f2efc8
filer: Elastic Search return correct kv error
...
fix https://github.com/chrislusf/seaweedfs/issues/1452
2020-09-07 12:44:51 -07:00
Chris Lu
d80538a187
refactoring
2020-09-07 12:35:02 -07:00
Chris Lu
1a09bc43d1
refactor
2020-09-07 11:31:33 -07:00
Chris Lu
2b643f477d
typo
2020-09-06 12:47:55 -07:00
Chris Lu
05034aade5
printout over replicated locations
2020-09-06 12:44:02 -07:00
Chris Lu
432fe3d926
1.93
2020-09-06 12:12:09 -07:00
Chris Lu
d8af6f62df
adjust logging
2020-09-06 12:10:40 -07:00
Chris Lu
07f32feed0
print sync progress
2020-09-06 00:50:51 -07:00
Chris Lu
0af6252b2d
fix compilation
2020-09-06 00:50:38 -07:00
Chris Lu
ac494ff5e8
Update meta_aggregator.go
2020-09-06 00:29:16 -07:00
Chris Lu
c9f8f25ba5
read peer filer from start
2020-09-06 00:12:41 -07:00
Chris Lu
30dc365cbd
add self filer
2020-09-06 00:11:46 -07:00
Chris Lu
8acd714600
apply meta changes only if store is different
2020-09-05 23:25:03 -07:00
Chris Lu
d741ed66db
check whether shares the same filer store
2020-09-05 23:02:52 -07:00
Chris Lu
bba90ff3c8
read filer signature
2020-09-05 22:52:15 -07:00
Chris Lu
71b0e256c7
filer: setOrLoadFilerStoreSignature
2020-09-05 14:08:59 -07:00
ruitao.liu
450cf07505
skip the index that for kv usage.
2020-09-04 21:49:03 +08:00
ruitao.liu
1384ff9a2f
1.split kv in one file.
...
2.disable query for kv in es index.
2020-09-04 17:34:26 +08:00
ruitao.liu
83080b5e03
ES backended filer support kv ops.
2020-09-04 15:40:13 +08:00
Chris Lu
46f65a84a6
filer: elastic7 adjust default value
2020-09-03 20:12:38 -07:00
Chris Lu
f815fce81b
Merge pull request #1446 from taozix/master
...
new filer store option to es7.
2020-09-03 20:10:23 -07:00
Chris Lu
44b4ebf0ff
filer: Redis cleanly delete directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1448
2020-09-03 19:17:39 -07:00
Chris Lu
b8f32bcab9
filer: compress stored metadata
2020-09-03 11:00:20 -07:00
Chris Lu
f76a2b2c8a
printout meta data size
2020-09-03 09:51:21 -07:00
ruitao.liu
798280e980
change filer2 to filer.
2020-09-03 17:05:26 +08:00
ruitao.liu
a93d27d1e8
new filer option to es v7.
2020-09-03 16:34:58 +08:00
Chris Lu
1d56ea24ef
fix
2020-09-03 00:08:37 -07:00
Chris Lu
7e1aad0b54
mount: map uid/gid between local and filer
2020-09-03 00:07:22 -07:00
Chris Lu
68e878adb5
fix formatting
2020-09-02 21:42:12 -07:00
Chris Lu
0d99a5da3c
c*: support kv operations
2020-09-02 18:39:24 -07:00
Chris Lu
645a4af3db
mongodb: support kv operations
2020-09-02 17:19:14 -07:00
Chris Lu
ecaa30c408
better error message
2020-09-02 17:17:44 -07:00
Chris Lu
7c770b727c
Update abstract_sql_store_kv.go
2020-09-02 10:13:48 -07:00
Chris Lu
05a6e2dc7f
sql kv upsert
2020-09-02 10:12:44 -07:00
Chris Lu
ed62f52452
fix sql insert with duplicated primary key
2020-09-02 10:09:49 -07:00
Chris Lu
9ea290aa12
fix error message
2020-09-01 23:21:41 -07:00
Chris Lu
06a1188267
mysql/postgres: support kv operations
2020-09-01 22:48:36 -07:00
Chris Lu
aa40295f06
leveldb: report not found entry
2020-09-01 22:48:23 -07:00
Chris Lu
87d7312bf6
mysql/postgres: properly report entry not found
2020-09-01 22:47:57 -07:00
Chris Lu
72b0a5f1d1
mysql or postgres: log find error
2020-09-01 22:25:17 -07:00
Chris Lu
2815bbe6c0
go fmt
2020-09-01 21:59:26 -07:00
Chris Lu
37234bf3f8
filer store adds kv support
...
can compile now, need to implement those unimplemented
2020-09-01 21:59:00 -07:00
James Hartig
8e54e34576
volume: Don't unmount before deleting volume in copy
...
If we unmount first and then delete, the delete fails because the volume
was unmounted. Delete ends up doing the same thing as the unmount anyways.
2020-09-01 22:00:07 -04:00
Chris Lu
2b14ae5819
fix tests
2020-09-01 01:33:43 -07:00
Chris Lu
d91ec535b3
fix tests
2020-09-01 01:29:13 -07:00
Chris Lu
eb7929a971
rename filer2 to filer
2020-09-01 00:21:19 -07:00
Chris Lu
38e06d783d
volume: check disk space before compaction
...
fix https://github.com/chrislusf/seaweedfs/issues/1440
2020-08-31 18:10:53 -07:00
Chris Lu
dbb10e0f0b
testing with more than 30days
2020-08-31 17:59:29 -07:00
Chris Lu
dd176cdb8b
avoid tiny step pagination
2020-08-31 11:28:03 -07:00
Chris Lu
97733c3dff
typo
2020-08-31 10:49:17 -07:00
Chris Lu
c8398bdb09
adjust metrics
2020-08-31 10:41:05 -07:00
Chris Lu
8a1c8e41b3
simplify if else logic
2020-08-31 10:39:24 -07:00
Chris Lu
f0c89cfacd
go fmt
2020-08-31 10:23:31 -07:00
Chris Lu
edb9d65e05
Merge pull request #1431 from kmlebedev/wip-prefix-search
...
Wip prefix search
2020-08-31 10:22:14 -07:00
Konstantin Lebedev
9a195bebfd
accurate limit
2020-08-31 22:13:13 +05:00
Konstantin Lebedev
18b98f9747
not convert err to string
2020-08-31 21:55:18 +05:00
Konstantin Lebedev
22f32e75c4
rename
2020-08-31 21:52:05 +05:00
Konstantin Lebedev
a8de332beb
fix
2020-08-31 21:45:44 +05:00
Konstantin Lebedev
7f013d77d6
add var UnsupportedListDirectoryPrefixedErr
2020-08-31 21:43:32 +05:00
Konstantin Lebedev
60a86cfe04
add const
2020-08-31 21:35:16 +05:00
Konstantin Lebedev
82ea121d09
rm func
2020-08-31 18:50:06 +05:00
Konstantin Lebedev
2ea638f865
filer store wrapper can implement the logic to filter by prefi
2020-08-31 17:13:56 +05:00
Chris Lu
408e339c53
also delete the manifest chunk itself
2020-08-31 00:16:03 -07:00
Chris Lu
dd52037c27
1.92
2020-08-30 22:56:35 -07:00
Chris Lu
33d8c6c617
change log to warning
2020-08-30 21:01:44 -07:00
Chris Lu
e62d7f221f
a bit more logs
2020-08-30 21:00:24 -07:00
Chris Lu
22fe4ae573
mount: fix for UrBackup
2020-08-30 21:00:09 -07:00
Chris Lu
ae84a9ee2e
add logs
2020-08-30 20:59:04 -07:00
Chris Lu
a41588279a
change log level 5 to 4
2020-08-30 20:12:04 -07:00
Chris Lu
56244fb9a1
fix hard coded host address
2020-08-30 10:23:35 -07:00
Chris Lu
f2a8574448
filer and mount deletion resolves manifest chunks also
2020-08-30 02:07:14 -07:00
Chris Lu
99ecf63276
go fmt
2020-08-29 22:28:33 -07:00
Chris Lu
ef75ce8a34
use UTC for all time related folders
2020-08-29 21:01:14 -07:00
Chris Lu
b69cb74c03
read meta logs by timestamp
...
pass in event ts when moving logs
meta aggregator reads in memory logs only
2020-08-29 17:37:19 -07:00
Chris Lu
063c9ddac5
adjust logs
2020-08-29 11:56:22 -07:00
Chris Lu
ca658a97c5
add signatures to messages to avoid double processing
2020-08-28 23:48:48 -07:00
Chris Lu
63ad1abcce
watch: follow changes for a pattern
2020-08-28 19:43:04 -07:00
Chris Lu
b7a654a318
file mime avoid saving application/octet-stream
2020-08-28 19:42:40 -07:00
Chris Lu
707192f966
fix help message
2020-08-27 00:13:38 -07:00
Chris Lu
0be6863c87
rename
2020-08-26 22:40:15 -07:00
Chris Lu
50f7e2e89f
add log messages
2020-08-26 09:51:20 -07:00
Chris Lu
6bc618051c
log message
2020-08-26 09:32:56 -07:00
Chris Lu
9dae4e9d3e
fix logs
2020-08-26 09:31:11 -07:00
Chris Lu
4c1a3187bf
fix log error
2020-08-26 09:25:54 -07:00
Chris Lu
ab759f0ec2
erasure coding: fix EC error if multiple disks are configured in one volume server
2020-08-26 09:16:58 -07:00
Chris Lu
d1cf39f180
fix logging
2020-08-26 08:55:15 -07:00
Chris Lu
83012f543a
s: avoid possible completeMultipartUpload re-tries that can overwrite the uploaded file
2020-08-26 08:50:24 -07:00
Konstantin Lebedev
464d4c82ec
stop send heartbeat before stop volume server
2020-08-26 16:16:11 +05:00
Chris Lu
1901f15cd2
volume server: remove whitelist for status checking
...
Volume liveness/readiness not work if jwt enabled, I try use param
-whiteList
but jwt breaks
2020-08-25 02:27:47 -07:00
Chris Lu
81e5124faf
s3: list bucket permission change from admin to read
...
fix https://github.com/chrislusf/seaweedfs/issues/1430
2020-08-24 11:22:45 -07:00
Chris Lu
5f55a87101
close http response
2020-08-24 00:32:44 -07:00
Chris Lu
0b301b0b47
consume all response body in order to release requests
2020-08-23 21:32:29 -07:00
Chris Lu
2cdad77d82
fix checkin error
2020-08-23 18:30:11 -07:00
Chris Lu
81a8aa6581
1.91
2020-08-23 17:28:40 -07:00
Chris Lu
51346a5930
always keep the manifest list of chunks
2020-08-23 17:15:12 -07:00
Chris Lu
c1d1677a28
keep manifest chunks forever
2020-08-23 16:59:01 -07:00
Chris Lu
98175548c2
adjust deletion ordering
2020-08-23 16:27:12 -07:00
Chris Lu
f0e50e7b26
fix test
2020-08-23 16:12:34 -07:00
Chris Lu
aee27ccbe1
multiple fixes
...
* adjust isOpen count
* move ContinuousDirtyPages lock to filehandle
* fix problem with MergeIntoVisibles, avoid reusing slices
* let filer delete the garbage
2020-08-23 15:48:02 -07:00
Chris Lu
c19245886c
simpler test
2020-08-23 14:20:27 -07:00
Chris Lu
df816a58fe
add tests
2020-08-23 14:09:25 -07:00