Commit Graph

678 Commits

Author SHA1 Message Date
wusong
549354e324
Fix hardlink counting (#4042)
Signed-off-by: wusong <wangwusong@virtaitech.com>

Signed-off-by: wusong <wangwusong@virtaitech.com>
Co-authored-by: wusong <wangwusong@virtaitech.com>
2022-12-08 10:50:57 -08:00
aronneagu
77699855a7
Return ETag from remote when file doesn't exist on Filer (#4025) 2022-11-30 07:43:30 -08:00
chrislu
70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks()
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
chrislu
d90aa31d5f filer store: skip disabled location specific filer store initialization
fix https://github.com/seaweedfs/seaweedfs/issues/3971
2022-11-13 23:30:23 -08:00
Konstantin Lebedev
4a48332248
refactor error contains already deleted (#3932) 2022-10-31 11:34:45 -07:00
chrislu
ea2637734a refactor filer proto chunk variable from mtime to modified_ts_ns 2022-10-28 12:53:19 -07:00
chrislu
f5d4952d73 filer: redis store reduce from 2 redis operations to 1 for updates. 2022-10-12 23:50:09 -07:00
chrislu
cea73ac008 serialize sqlite operations
fix https://github.com/seaweedfs/seaweedfs/issues/3827
2022-10-12 00:03:27 -07:00
chrislu
dff85e9c71 fix error handling 2022-10-12 00:03:14 -07:00
LHHDZ
e00a12b099
associate Account and Identity by accountId (#3754) 2022-09-28 13:25:59 -07:00
chrislu
b2a171632b mount: fix 0 file size in linux 2022-09-18 21:42:13 -07:00
chrislu
9d230521a4 report write error 2022-09-18 16:49:48 -07:00
Ryan Russell
e0064ae097
docs: update fileIdsToDelete var (#3692)
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-15 02:09:06 -07:00
chrislu
21c0587900 go fmt 2022-09-14 23:06:44 -07:00
Ryan Russell
f2dde2b1da
refactor(ItemList): Incluseive -> Inclusive (#3673)
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-14 12:00:16 -07:00
Ryan Russell
72d8a9f9a8
refactor(exclusive_locker): Interval readability batch of updates (#3668)
* refactor(filechunk_manifest): `localProcesed` -> `localProcessed`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor: `saveChunkedFileIntevalToStorage` -> `saveChunkedFileIntervalToStorage`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor: `SafeRenewInteval` -> `SafeRenewInterval`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor: `InitLockInteval` -> `InitLockInterval`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor: `RenewInteval` -> `RenewInterval`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-14 10:29:55 -07:00
Ryan Russell
d54eb9966f
refactor: Directory readability (#3665) 2022-09-14 10:11:31 -07:00
gfx
48db56ddad
arangodb s3 bucket name compatibility (#3588)
* Update arangodb_store.go

* update readme, properly escape queries, add name patching

* use underscore

* use underscore

* better comment

* fix readme

Co-authored-by: a <a@a.a>
2022-09-09 09:43:42 -07:00
Konstantin Lebedev
3c3682fcce
more log detail for upload err and deleting (#3577) 2022-09-06 08:19:13 -07:00
famosss
131d389fc4
adjust log level (#3589) 2022-09-04 22:37:32 -07:00
gfx
bf844d8e46
Fix crash in arangodb filer when attempting to access a deleted bucket (#3587)
* Update helpers.go

* Update arangodb_store_bucket.go
2022-09-04 20:32:10 -07:00
gfx
3621987727
Correctly pass arangodb driver Transaction ID into the context (#3586)
Update arangodb_store.go
2022-09-04 19:29:52 -07:00
chrislu
a112cbe44c filer: rocksdb store report not found correctly
fix https://github.com/seaweedfs/seaweedfs/discussions/3555
2022-09-01 15:42:59 -07:00
chrislu
10efdc7aab align memory 2022-09-01 09:02:44 -07:00
Konstantin Lebedev
803ca3c958
avoid data race on doSubscribeToOneFiler/ma.filer.UniqueFilerEpoch (#3566)
https://github.com/seaweedfs/seaweedfs/issues/3565
2022-09-01 09:01:47 -07:00
chrislu
d81db3c703 s3: fix configuring IAM for the same user
hi, how can I add bucket permission to a user now?
Previously, if I needed to add permission to an existing credential, I simply repeated the s3.configure command with a different bucket name.
Now I am getting error:
duplicate accessKey[ХХХХ], already configured in user[YYYY]

s3.configure -access_key key -actions Read,Write,List -buckets bucket1 -secret_key secr -user user1
s3.configure -access_key key -actions Read,Write,List -buckets bucket2 -secret_key secr -user user1
2022-08-30 09:37:52 -07:00
chrislu
57a46f46a0 nano level precision 2022-08-26 16:55:15 -07:00
Patrick Schmidt
5df105b1f9
Fix a few data races when reading files in mount (#3527) 2022-08-26 16:41:37 -07:00
Konstantin Lebedev
e90ab4ac60
avoid race conditions for OnPeerUpdate (#3525)
https://github.com/seaweedfs/seaweedfs/issues/3524
2022-08-26 10:18:49 -07:00
Patrick Schmidt
2930263dfd
Fix race conditions during in-flight size checks (#3505) 2022-08-24 20:03:34 -07:00
chrislu
fef9c6a520 filer: do not always save files in "/etc" folder to filer store
fix https://github.com/seaweedfs/seaweedfs/issues/3476
2022-08-21 17:02:16 -07:00
Patrick Schmidt
f49a9297c2
Fix hanging reads in chunk cacher (#3473)
Sometimes when an unexpected error occurs the cacher would set an
error and return. However, it would not broadcast the condition
signal in that case, therefore leaving the goroutine that runs
readChunkAt stuck forever.
I figured that the condition is unnecessary because readChunkAt is
acquiring a lock that is still held by the cacher goroutine anyway.
Callees of startCaching have to wait for a WaitGroup which makes sure
that readChunkAt can't acquire the lock before startCaching.
This way readChunkAt can execute normally and check for the error.
2022-08-21 11:54:02 -07:00
chrislu
388f82f322 minor 2022-08-21 11:49:29 -07:00
chrislu
77e4b1376e refactoring 2022-08-21 11:35:54 -07:00
Patrick Schmidt
3f758820c1
Fix FUSE server buffer leaks in file gaps (#3472)
* Fix FUSE server buffer leaks in file gaps

This change zeros read buffers when encountering file gaps during
file/chunk reads in FUSE mounts.
It prevents leaking internal buffers of the FUSE server which could
otherwise reveal metadata, directory listings, file contents and
other data related to FUSE API calls.
The issue was that buffers are reused, but when a file gap was found
the buffer was not zeroed accordingly and the existing data of the
buffer was kept and returned.

* Move zero logic into its own method
2022-08-21 11:33:58 -07:00
chrislu
689b4ecdcc remove unused collection and replication from upload result 2022-08-20 18:14:57 -07:00
Guo Lei
3172c33fa6
fix redundant type from array (#3462) 2022-08-18 02:25:25 -07:00
chrislu
4573c99ae3 fix tests 2022-08-18 00:15:46 -07:00
chrislu
2b580a7566 also migrate jsonpb 2022-08-17 12:42:03 -07:00
chrislu
eaeb141b09 move proto package 2022-08-17 12:05:07 -07:00
chrislu
96caf21d09 less verbose log 2022-08-15 10:03:52 -07:00
chrislu
0aeec04c31 quicker to adapt to pattern change 2022-08-07 10:14:01 -07:00
chrislu
928d29af9e fix wrong logic about reader isRandomMode() 2022-08-07 10:13:04 -07:00
John W Higgins
3afda0c89c
Allow postgresql to use standard environment variables for connection (#3413) 2022-08-07 00:58:53 -07:00
LHHDZ
84ec68e11a
Add download speed limit support (#3408) 2022-08-05 01:16:42 -07:00
Konstantin Lebedev
4d08393b7c
filer prefer volume server in same data center (#3405)
* initial prefer same data center
https://github.com/seaweedfs/seaweedfs/issues/3404

* GetDataCenter

* prefer same data center for ReplicationSource

* GetDataCenterId

* remove glog
2022-08-04 17:35:00 -07:00
Chris Lu
b59bc607bf
Merge pull request #3338 from kmlebedev/issues/3083
rollback over onPeerUpdate implementation of automatic clean-up of failed servers in favor of synchronous ping
2022-08-01 08:23:10 -07:00
Konstantin Lebedev
a98f6d66a3 rollback over onPeerupdate implementation of automatic clean-up of failed servers in favor of synchronous ping 2022-08-01 12:51:41 +05:00
chrislu
036566629a filer.sync: fix synchronization logic in active-active mode
fix https://github.com/seaweedfs/seaweedfs/issues/3328
2022-08-01 00:06:18 -07:00
chrislu
303bd067b5 Revert "rename: delete source entry metadata only, skipping hard links"
This reverts commit 03466f955e.

fix https://github.com/seaweedfs/seaweedfs/issues/3386
2022-07-31 22:51:41 -07:00