Commit Graph

9214 Commits

Author SHA1 Message Date
chrislu
ef78631a7c just a bit safer 2022-08-31 00:10:17 -07:00
chrislu
9c944377dc minor 2022-08-31 00:09:32 -07:00
chrislu
a54f30b6c6 s3: close response body with copying object 2022-08-31 00:09:23 -07:00
chrislu
7eb121d5ae print process before do the work 2022-08-30 13:37:05 -07:00
Konstantin Lebedev
77098de69a
migrate from boltdb to bbolt (#3554) 2022-08-30 10:12:02 -07:00
chrislu
de5263364b master client: use GetMaster() for latest master address 2022-08-30 09:52:06 -07:00
chrislu
e583dbdd2a more detailed error 2022-08-30 09:43:17 -07:00
chrislu
d81db3c703 s3: fix configuring IAM for the same user
hi, how can I add bucket permission to a user now?
Previously, if I needed to add permission to an existing credential, I simply repeated the s3.configure command with a different bucket name.
Now I am getting error:
duplicate accessKey[ХХХХ], already configured in user[YYYY]

s3.configure -access_key key -actions Read,Write,List -buckets bucket1 -secret_key secr -user user1
s3.configure -access_key key -actions Read,Write,List -buckets bucket2 -secret_key secr -user user1
2022-08-30 09:37:52 -07:00
Konstantin Lebedev
ade94b0d0a
avoid race conditions access to SuperBlock.Version (#3539)
* avoid race conditions access to  SuperBlock.Version
https://github.com/seaweedfs/seaweedfs/issues/3515

* superBlockAccessLock replace to sync.Mutex
2022-08-30 00:08:00 -07:00
chrislu
ae6292f9f0 rename variable 2022-08-30 00:07:15 -07:00
Konstantin Lebedev
e16dda88e4
avoid race conditions access to MasterClient.currentMaster (#3538)
https://github.com/seaweedfs/seaweedfs/issues/3510
2022-08-30 00:05:38 -07:00
Konstantin Lebedev
105702ebe0
avoid data race access to uploadReaderToChunks.uploadErr (#3550)
avoid data race access to uploadErr
https://github.com/seaweedfs/seaweedfs/issues/3549
2022-08-30 00:03:01 -07:00
Konstantin Lebedev
903a06a160
avoid data race access to startDelay (#3548) 2022-08-30 00:02:09 -07:00
chrislu
570b488572 ensure memory alignment on different CPUs 2022-08-29 13:24:12 -07:00
Konstantin Lebedev
4966a3abc7
avoid race conditions access to growRequestCount (#3537)
https://github.com/seaweedfs/seaweedfs/issues/3511
2022-08-29 13:23:02 -07:00
dependabot[bot]
42b72e4006
Bump actions/setup-go from 3.2.1 to 3.3.0 (#3540) 2022-08-29 08:49:47 -07:00
dependabot[bot]
99b7f8fa4b
Bump cloud.google.com/go/pubsub from 1.24.0 to 1.25.1 (#3542) 2022-08-29 08:49:04 -07:00
dependabot[bot]
85fe01eefa
Bump github.com/linxGnu/grocksdb from 1.7.5 to 1.7.7 (#3543) 2022-08-29 08:48:51 -07:00
dependabot[bot]
75fe6e4ab2
Bump modernc.org/strutil from 1.1.2 to 1.1.3 (#3544) 2022-08-29 08:47:53 -07:00
dependabot[bot]
ac939ccb89
Bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.35.1 to 3.37.1 (#3545) 2022-08-29 08:47:16 -07:00
Patrick Schmidt
00041f6e08
Fix corrupted mount reads after changes on fh (#3536) 2022-08-29 08:27:13 -07:00
chrislu
b7a887fea1 3.24 2022-08-28 21:53:13 -07:00
Rain Li
089446f5b1
Fix reload page before all file uploaded bug (#3534) 2022-08-28 07:51:47 -07:00
chrislu
cc0c8c5f81 simplify 2022-08-27 00:21:57 -07:00
chrislu
3741779144 minor 2022-08-27 00:20:58 -07:00
chrislu
580a7fa685 adjust for data race
fix https://github.com/seaweedfs/seaweedfs/issues/3512
2022-08-27 00:19:13 -07:00
chrislu
87b70a6809 clean up 2022-08-27 00:09:04 -07:00
chrislu
27c7611ebf Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-26 23:47:39 -07:00
chrislu
9920d65bc0 gateway to remote object store: adjust upload concurrency 2022-08-26 23:47:37 -07:00
chrislu
c839ce1b19 s3 sink use s3 upload manager
fix https://github.com/seaweedfs/seaweedfs/issues/3531
2022-08-26 23:47:12 -07:00
famosss
dc4037925d
fix: Build DeletedVids before reset dn's children (#3530) 2022-08-26 22:52:08 -07:00
chrislu
3f3a1341d8 make CodeQL happy 2022-08-26 17:09:11 -07:00
chrislu
301b49b63f atomic operation 2022-08-26 17:04:11 -07:00
chrislu
57a46f46a0 nano level precision 2022-08-26 16:55:15 -07:00
chrislu
8dae81c5ed Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-26 16:49:51 -07:00
chrislu
e0f4366f4c more detailed error
related to https://github.com/seaweedfs/seaweedfs/issues/3528
2022-08-26 16:49:48 -07:00
Patrick Schmidt
5df105b1f9
Fix a few data races when reading files in mount (#3527) 2022-08-26 16:41:37 -07:00
chrislu
f5156cf3a8 rename varaible 2022-08-26 10:23:42 -07:00
Konstantin Lebedev
e90ab4ac60
avoid race conditions for OnPeerUpdate (#3525)
https://github.com/seaweedfs/seaweedfs/issues/3524
2022-08-26 10:18:49 -07:00
Konstantin Lebedev
4f7a1f67cd
avoid race conditions for diskLocation.MaxVolumeCount (#3526) 2022-08-26 08:41:42 -07:00
Konstantin Lebedev
913a5d0dd4
add target dev_race to MakeFile (#3522) 2022-08-26 01:28:03 -07:00
chrislu
6f483a4d36 atomic add
fix https://github.com/seaweedfs/seaweedfs/issues/3514
2022-08-25 22:20:34 -07:00
chrislu
7394f7feee minor: adjust ordering 2022-08-25 00:20:23 -07:00
chrislu
dbf0de4ce1 minor clean up 2022-08-25 00:19:08 -07:00
Patrick Schmidt
2930263dfd
Fix race conditions during in-flight size checks (#3505) 2022-08-24 20:03:34 -07:00
Patrick Schmidt
7b424a54dc
Add raft server access mutex to avoid races (#3503) 2022-08-24 09:49:05 -07:00
Chris Lu
f7e4359b59
Temporarily enable cgo
Temporarily enable cgo following findings from https://github.com/seaweedfs/seaweedfs/discussions/3152 by https://github.com/dyus

Resolve build issue:
```
# github.com/mattn/go-ieproxy
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:36:9: undefined: getConf
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:41:9: undefined: reloadConf
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:48:2: undefined: overrideEnvWithStaticProxy
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:53:13: psc.findProxyForURL undefined (type *ProxyScriptConf has no field or method findProxyForURL, but does have FindProxyForURL)
```
2022-08-24 02:24:21 -07:00
Konstantin Lebedev
f7aeb06544
s3: report metadata if the directory is explicitly created (#3498)
* replace mkdir to mkFile

* ContentLength must be zero

* revert mkDir

* Seaweedfs-Is-Directory-Key return metadata
2022-08-24 00:15:44 -07:00
Guo Lei
c57c79a0ab
optimiz commitig compact (#3388)
* optimiz vacuuming volume

* fix bugx

* rename parameters

* fix conflict

* change copyDataBasedOnIndexFile to an instance method

* close needlemap

* optimiz commiting Vacuum volume for  leveldb index

* fix bugs

* fix leveldb loading bugs

* refactor

* fix leveldb loading bug

* add leveldb recovery

* add test case for levelDB

* modify test case to cover all the new branches

* use one tmpNm instead of two instances

* refactor

* refactor

* move setWatermark to the end

* add test for watermark and updating leveldb

* fix error logic

* refactor, add test

* check nil before close needlemapeer
add test case
fix metric bug

* add tests, fix bugs

* adjust log level
remove wrong test case
refactor

* avoid duplicate  updating metric for leveldb index
2022-08-23 23:53:35 -07:00
chrislu
10414fd81c ping timeout at 15 seconds
this 72 minute timeout setting seems unreasonably long

15 seconds is around the time when a new raft leader should be elected.
2022-08-23 23:28:16 -07:00