Commit Graph

968 Commits

Author SHA1 Message Date
Chris Lu
0ef43a23a7 go fmt 2021-01-12 02:30:12 -08:00
Chris Lu
629c9962e7 filer: add leveldb3
support dedicated leveldb instance for each bucket
2021-01-12 02:29:44 -08:00
Chris Lu
394513f598 filer: ensure seamless meta data updates 2021-01-11 02:08:55 -08:00
Chris Lu
1efb51ba84 filer: change to saveToFilerLimit from cacheToFilerLimit
short circuit saving small files to volume server
2021-01-10 23:14:46 -08:00
henry
b42c5b71c1 Seaweed custom header are not visible to Vue or javascript 2021-01-05 13:01:29 +08:00
henry
1309dfe612 Seaweed custom header are not visible to Vue or javascript 2021-01-05 12:58:46 +08:00
henry
97a94eddab Merge branch 'master' of https://github.com/fuyouyshengwu/seaweedfs 2021-01-04 14:50:01 +08:00
Chris Lu
f4d87f8b9d works
performance seems about the same as leveldb for insert entries
2021-01-02 23:32:58 -08:00
Chris Lu
2c3c2c27d7 separate prefix from namePattern
fix https://github.com/chrislusf/seaweedfs/issues/1722
2021-01-01 20:23:23 -08:00
Peter Cai
cd2a9865ed filer: respect Content-Type on PUT
This is needed for the S3 gateway to conform to the S3 protocol -- the
server should always respect the Content-Type set by client.
2020-12-31 11:32:45 +08:00
henry
14ddd15508 1 Distinguish between folders and common files, folders are listed first
2 Add some custom headers that need to be exposed
2020-12-31 00:03:22 +08:00
Chris Lu
2e6cf541e7 filer: avoid deletion error when item is not found 2020-12-28 00:18:37 -08:00
henry
7458ff9523 Solve the Chinese name disorder of download file 2020-12-28 11:28:33 +08:00
Chris Lu
0a7c5f85a9 filer: add namePattern to search in current folder 2020-12-26 15:05:31 -08:00
Chris Lu
0a067944cc filer: add retries during volume moving
fix https://github.com/chrislusf/seaweedfs/issues/1704
2020-12-25 02:32:55 -08:00
Chris Lu
c3d1b3b5aa hook up 2020-12-23 21:49:01 -08:00
Chris Lu
1c7e1295dc Merge branch 'master' into support_ssd_volume 2020-12-22 17:44:52 -08:00
Chris Lu
a1009e8044 filer: add -defaultStoreDir so that filer.toml can be skipped
fix https://github.com/chrislusf/seaweedfs/issues/1659
2020-12-22 17:33:40 -08:00
Chris Lu
f696a2b2a7 assign volumes based on disk type 2020-12-17 13:25:05 -08:00
Chris Lu
1bf22c0b5b go fmt 2020-12-16 09:14:05 -08:00
Chris Lu
1d88865869 passing disk type along 2020-12-16 09:10:14 -08:00
Chris Lu
23014b6810 Merge branch 'master' into support_ssd_volume 2020-12-16 08:26:51 -08:00
Chris Lu
b789767cd1 filer: return http status 499 for client closed connection
fix https://github.com/chrislusf/seaweedfs/issues/1684
2020-12-16 08:18:00 -08:00
Chris Lu
2e8dba571b adjust volume server UI 2020-12-14 00:51:57 -08:00
Chris Lu
bf5873022d Merge branch 'master' into support_ssd_volume 2020-12-14 00:32:17 -08:00
Chris Lu
b7e3ca9172
Merge pull request #1650 from kmlebedev/masterServerEnableUI
security master use access.ui
2020-12-14 00:31:51 -08:00
Konstantin Lebedev
c4459249ef ui interface is always on 2020-12-14 13:24:41 +05:00
Chris Lu
94525aa0fd allocate volume by disk type 2020-12-13 23:08:21 -08:00
Chris Lu
f6a419c26c disk type configurable for each folder 2020-12-13 22:49:56 -08:00
Chris Lu
7c52a35942 refactor 2020-12-13 22:29:52 -08:00
Chris Lu
a9db24cd05 master allocate volumes if ssd type runs out 2020-12-13 19:44:57 -08:00
Chris Lu
8baba93fce rename parameter name to "disk" 2020-12-13 12:06:21 -08:00
Chris Lu
51eadaf2b6 rename parameter name to "disk" 2020-12-13 12:05:31 -08:00
Chris Lu
0d2ec832e2 rename from volumeType to diskType 2020-12-13 11:59:32 -08:00
Chris Lu
d156c74ec0 volume server set volume type and heartbeat to the master 2020-12-13 03:11:24 -08:00
Chris Lu
e9cd798bd3 adding volume type 2020-12-13 00:58:58 -08:00
Chris Lu
16cd6fb278 fix variable name 2020-12-12 21:19:40 -08:00
Chris Lu
3fedfec1e7 check cross device rename error 2020-12-10 23:50:32 -08:00
Chris Lu
83078ac6ce filer: change to /etc/seaweedfs folder on filer
fix https://github.com/chrislusf/seaweedfs/issues/1666
2020-12-10 11:11:02 -08:00
Chris Lu
765b3ef1be save /etc/iam/identity.json inside filer store 2020-12-10 00:15:22 -08:00
Chris Lu
606051cc79 filer: cache "/etc" filder small files to filer store 2020-12-07 00:11:35 -08:00
Chris Lu
ac22f1dd26 filer subscribe: handle rename subscription 2020-12-07 00:10:49 -08:00
Chris Lu
263eb29e9f filer: add option to cache small files to filer store 2020-12-04 22:39:43 -08:00
Konstantin Lebedev
eadbba5c95 security master use access.ui 2020-12-04 12:57:59 +05:00
Chris Lu
45dba088cd fix tests 2020-12-01 19:37:21 -08:00
Chris Lu
04062c56c7 webdav: improve webdav upload speed 2020-12-01 15:32:27 -08:00
Chris Lu
f4abd01adf filer: cache small file to filer store 2020-11-30 04:34:04 -08:00
Chris Lu
a9c6be5fc3 set creation time 2020-11-30 03:11:52 -08:00
Chris Lu
141ce67c09 close http request body 2020-11-30 02:45:00 -08:00
Chris Lu
965413c21b shell: add volume.vacuum command 2020-11-28 23:18:02 -08:00
Chris Lu
c57a7839ec adjust comments 2020-11-28 03:01:03 -08:00
Chris Lu
8438fb0cc3 fix ec shard spreading error 2020-11-28 02:21:16 -08:00
Chris Lu
85554bea38 filer: readonly handle static resources
fix https://github.com/chrislusf/seaweedfs/issues/1641
2020-11-27 13:42:14 -08:00
Chris Lu
6d30b21b10 volume: add "-dir.idx" option for separate index storage
fix https://github.com/chrislusf/seaweedfs/issues/1265
2020-11-27 03:17:10 -08:00
Chris Lu
b11449f955 pass in dir.idx parameter to DiskLocation 2020-11-26 14:59:03 -08:00
Chris Lu
0da7ecfd29 go fmt 2020-11-26 11:22:30 -08:00
Chris Lu
92f906b6fc remove zstd
fix https://github.com/chrislusf/seaweedfs/issues/1629
2020-11-21 13:06:45 -08:00
Chris Lu
dc304342b2 fs.configure: configurable volume growth 2020-11-17 01:00:02 -08:00
Chris Lu
6204dc6e85 add comment 2020-11-16 16:57:31 -08:00
Chris Lu
ef3e068c51 deprecating "filer.options.buckets_fsync" in filer.toml 2020-11-16 16:56:49 -08:00
Chris Lu
0ea5c087ce go fmt 2020-11-15 16:59:28 -08:00
Chris Lu
95c0de285d refactoring 2020-11-15 16:58:48 -08:00
Chris Lu
500bcab953 refactoring 2020-11-15 14:41:56 -08:00
Chris Lu
590f02179d filer: load filer conf when starting 2020-11-15 14:06:03 -08:00
Konstantin Lebedev
1eec5c8d5d gen pb 2020-11-12 04:10:06 +05:00
Konstantin Lebedev
fc7baef5bb fiil serverUrls sorted by data center 2020-11-12 02:13:33 +05:00
Konstantin Lebedev
dc26012a3b initial 2020-11-11 15:03:47 +05:00
Chris Lu
6856b0d57e filer: add API to add/modify/delete tagging 2020-11-09 01:00:07 -08:00
Chris Lu
0dafcf1f5a volume: detect max volume count changes based on disk usage
fix https://github.com/chrislusf/seaweedfs/issues/1594
2020-11-03 14:43:17 -08:00
Chris Lu
d29b787220 store together with "Seaweed-" prefix 2020-11-03 00:21:10 -08:00
Chris Lu
5448781dfc filer: Headers with "Seaweed-" prefix are stored as extended properties 2020-11-03 00:15:51 -08:00
Chris Lu
de86945aeb go fmt 2020-10-31 16:45:38 -07:00
Chris Lu
306062b4e7 filer: add CORS support 2020-10-31 16:44:03 -07:00
Chris Lu
05acc33294 volume: add CORS support 2020-10-31 16:31:39 -07:00
Konstantin Lebedev
d2a4ffdd60 glog 2020-10-31 03:21:54 +05:00
Konstantin Lebedev
46303c36bf When the volume server is stopped, the master server immediately sees the deletion of volumes 2020-10-31 02:48:25 +05:00
Chris Lu
eb32af25d4 make space for the QR code 2020-10-30 10:55:31 -07:00
ruitao.liu
22a9ea0512 adjust s3 header file. 2020-10-29 16:05:40 +08:00
ruitao.liu
b917be7955 S3 bucket list, response with uploaded storageclass. 2020-10-29 14:57:19 +08:00
ruitao.liu
2dcc178d0a save s3 metadata to filer. 2020-10-28 18:16:05 +08:00
Chris Lu
53c3aad875 volume: add a note file to avoid incomplete volume files
fix https://github.com/chrislusf/seaweedfs/issues/1567
2020-10-27 15:56:49 -07:00
Chris Lu
24bf142596 copy large file first 2020-10-27 15:31:15 -07:00
ruitao.liu
72f6b77dea fix tag prefix. 2020-10-27 18:01:37 +08:00
ruitao.liu
8766ca1b95 rename s3 tag prefix. 2020-10-27 17:33:24 +08:00
ruitao.liu
19026ae55d return x-amz-tag-count header when GET object. 2020-10-27 16:49:31 +08:00
Chris Lu
e219c57849 passing full path when assign volume locations 2020-10-25 15:46:29 -07:00
Chris Lu
f375b93aef renaming 2020-10-25 15:32:43 -07:00
Chris Lu
e9d40b80b2 less verbose logs 2020-10-24 09:42:54 -07:00
Chris Lu
fb124f2d4a conditionally display volume server public url link 2020-10-23 23:58:46 -07:00
Chris Lu
1b17f71939 adjust election timeout to 10 seconds 2020-10-23 23:06:44 -07:00
Chris Lu
fc689319ae add volume public url 2020-10-23 18:03:23 -07:00
Chris Lu
6c92fa349d filer: support default rack
fix https://github.com/chrislusf/seaweedfs/issues/1546
2020-10-20 17:41:39 -07:00
Chris Lu
69f0da5d73 mount: pass along replication and collection parameters 2020-10-19 00:12:42 -07:00
Chris Lu
37f165d743 webdav: return io.EOF when at end of a file
fix https://github.com/chrislusf/seaweedfs/issues/1344
2020-10-16 12:53:37 -07:00
Chris Lu
ace0ea3d28 s3: avoid duplicated bucket 2020-10-15 10:52:20 -07:00
Chris Lu
f022aff289 add back http.StatusPartialContent
revert e7c04af1d0
2020-10-12 12:26:25 -07:00
Chris Lu
723ae11db4 refactoring in order to adjust volume server url later 2020-10-11 20:15:10 -07:00
Chris Lu
9b0e8ef026 filer: added QR code to transfer files with mobile devices 2020-10-10 18:00:21 -07:00
Chris Lu
e1a8e3f900 fix logging 2020-10-08 18:40:38 -07:00
Chris Lu
bd8b9b0c2e some changes to prepare fix after failed reads 2020-10-08 18:33:06 -07:00
Chris Lu
ec08a1670b volume: avoid hanging when stopping a volume server 2020-10-07 23:48:24 -07:00
Chris Lu
62d3d3aea0 go fmt 2020-10-07 23:30:54 -07:00
Chris Lu
a8624c2e4f read from alternative replica
related to https://github.com/chrislusf/seaweedfs/issues/1512
2020-10-07 22:49:04 -07:00
Chris Lu
da4edf3651 master: check peers for existing leader before starting a leader election
fix https://github.com/chrislusf/seaweedfs/issues/1509
2020-10-07 01:25:39 -07:00
Konstantin Lebedev
e4f2d9eb4a We return etag using the same algorithm as aws s3
https://teppen.io/2018/06/23/aws_s3_etags/
2020-10-05 14:43:32 +05:00
Устюжанин Антон Александрович
702b1cb876 fix: remove deleted peers if resumeState = true 2020-10-04 21:56:17 +05:00
Устюжанин Антон Александрович
dc31b19469 fix: restore raft state 2020-10-03 14:03:41 +05:00
Устюжанин Антон Александрович
8c82fb7e5f fix: restore raft state 2020-10-02 23:01:20 +05:00
Chris Lu
55cb68c09c filer: replication follows master config if not specified 2020-09-30 09:15:55 -07:00
Chris Lu
31fc7bb2e1 refactor
adjust for faster test
2020-09-27 10:41:29 -07:00
Chris Lu
41be7a4c8f filer: upload also set file size
ensure works same as fuse mount
2020-09-25 12:02:06 -07:00
Chris Lu
1012df7bb5 switch hardlink id from int64 to bytes 2020-09-24 11:11:42 -07:00
Chris Lu
4856bce0ee adjust for metrics port 2020-09-24 10:21:23 -07:00
Chris Lu
2e9099369e
Merge pull request #1489 from kmlebedev/promhttp
Promhttp
2020-09-24 09:50:48 -07:00
Konstantin Lebedev
324e44d4b3 add start metrics server 2020-09-24 17:45:39 +05:00
Chris Lu
5e239afdfc hardlink works now 2020-09-24 03:06:48 -07:00
James Hartig
91e4eca1e9 Fix deadlock with KeepConnected and SendHeartbeat
There's the potential where we're writing to a clientConn and it goes away
and we're stuck keeping a read lock on clientChansLock. This causes
KeepConnected to not be able to remove the client since it requires a write
lock on clientChansLock. This ends up backing up SendHeartbeat because it
can't get a read lock.
2020-09-21 22:41:38 -04:00
Chris Lu
63373a9f9f filer: add file extended properties 2020-09-21 11:08:34 -07:00
Chris Lu
9a3b564508 adjust printout 2020-09-20 16:01:56 -07:00
Chris Lu
62563a895a refactoring 2020-09-20 16:00:01 -07:00
Chris Lu
d013d09a9b adjust logging 2020-09-20 15:38:59 -07:00
Chris Lu
41d508edfd go fmt 2020-09-19 14:10:26 -07:00
Chris Lu
2c21eb1971 volume: get metrics configuration from master
fix https://github.com/chrislusf/seaweedfs/issues/1354
2020-09-19 00:03:00 -07:00
Chris Lu
cb427d48fa filer report metrics configuration 2020-09-17 06:46:51 -07:00
Chris Lu
e4e0234998 refactoring 2020-09-17 06:43:54 -07:00
Chris Lu
e861a6a3ab simplify metrics configuration logic 2020-09-16 01:39:30 -07:00
Chris Lu
be54eeb364 passing value by checkWithMaster() instead 2020-09-16 01:33:45 -07:00
Chris Lu
4d21de63ee go fmt 2020-09-16 01:27:24 -07:00
Chris Lu
913a16268d volume: load configuration from master at the start
fix https://github.com/chrislusf/seaweedfs/issues/1469
2020-09-16 01:27:05 -07:00
Chris Lu
c9202c4b3d add storage backend to GetMasterConfigurationResponse 2020-09-16 01:12:22 -07:00
Chris Lu
8a0710cb73 handle more than 2GB files
related to https://github.com/chrislusf/seaweedfs/issues/1468
2020-09-16 00:41:50 -07:00
Chris Lu
854007bc98 minor 2020-09-16 00:37:57 -07:00
Chris Lu
ada996fe55 filer: support createing empty folder
fix https://github.com/chrislusf/seaweedfs/issues/1161
2020-09-14 13:07:46 -07:00
Chris Lu
5d6753fb98 shell: add volumeServer.leave command 2020-09-13 21:25:51 -07:00
Chris Lu
cc5fe6f5ee handle special characters in html link 2020-09-10 19:46:02 -07:00
Chris Lu
b183ae54b5 master: changing 301 to 308 on redirect to volume servers
fix https://github.com/chrislusf/seaweedfs/issues/1454
2020-09-09 22:36:10 -07:00
Chris Lu
daf0a449f7 properly cancel context for streaming grpc 2020-09-09 12:07:15 -07:00
Chris Lu
387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu
5e13bc878c adjust log level 2020-09-09 03:49:26 -07:00
Chris Lu
8acd714600 apply meta changes only if store is different 2020-09-05 23:25:03 -07:00
Chris Lu
bba90ff3c8 read filer signature 2020-09-05 22:52:15 -07:00
ruitao.liu
798280e980 change filer2 to filer. 2020-09-03 17:05:26 +08:00
ruitao.liu
a93d27d1e8 new filer option to es v7. 2020-09-03 16:34:58 +08:00
James Hartig
8e54e34576 volume: Don't unmount before deleting volume in copy
If we unmount first and then delete, the delete fails because the volume
was unmounted. Delete ends up doing the same thing as the unmount anyways.
2020-09-01 22:00:07 -04:00
Chris Lu
eb7929a971 rename filer2 to filer 2020-09-01 00:21:19 -07:00
Chris Lu
97733c3dff typo 2020-08-31 10:49:17 -07:00
Chris Lu
f0c89cfacd go fmt 2020-08-31 10:23:31 -07:00
Chris Lu
edb9d65e05
Merge pull request #1431 from kmlebedev/wip-prefix-search
Wip prefix search
2020-08-31 10:22:14 -07:00
Chris Lu
a41588279a change log level 5 to 4 2020-08-30 20:12:04 -07:00
Chris Lu
56244fb9a1 fix hard coded host address 2020-08-30 10:23:35 -07:00