Commit Graph

3007 Commits

Author SHA1 Message Date
Chris Lu
f498c71199 shell: move volume operations to use flag parsing arguments 2020-09-20 09:27:34 -07:00
Chris Lu
fcbc520373 fix print out 2020-09-20 08:47:30 -07:00
Chris Lu
56094541c8 shell: adjust command option for collection.delete 2020-09-20 08:46:16 -07:00
Chris Lu
9cac9c4001 2.00 2020-09-20 00:47:04 -07:00
Chris Lu
dc4d2145ff open up 2020-09-19 20:23:12 -07:00
Chris Lu
29abe980df s3: add support for PostPolicy
fix https://github.com/chrislusf/seaweedfs/issues/1426
2020-09-19 20:14:19 -07:00
Chris Lu
41d508edfd go fmt 2020-09-19 14:10:26 -07:00
Chris Lu
5b40a2690a refactoring 2020-09-19 14:09:58 -07:00
Chris Lu
2c21eb1971 volume: get metrics configuration from master
fix https://github.com/chrislusf/seaweedfs/issues/1354
2020-09-19 00:03:00 -07:00
Chris Lu
2cbd1cf121 fix compilation 2020-09-18 00:15:54 -07:00
Chris Lu
23e9ede068 s3: collect metrics 2020-09-18 00:09:04 -07:00
Chris Lu
852e5f7cbc filer: fix mongodb insert
fix https://github.com/chrislusf/seaweedfs/issues/1471
2020-09-17 21:50:52 -07:00
Chris Lu
6544e60bea s3 add metrics
empty for now
2020-09-17 06:56:15 -07:00
Chris Lu
cb427d48fa filer report metrics configuration 2020-09-17 06:46:51 -07:00
Chris Lu
e4e0234998 refactoring 2020-09-17 06:43:54 -07:00
Chris Lu
feca07bf96 Filer: ip bind
fix https://github.com/chrislusf/seaweedfs/issues/1470
2020-09-16 10:26:08 -07:00
Chris Lu
e861a6a3ab simplify metrics configuration logic 2020-09-16 01:39:30 -07:00
Chris Lu
be54eeb364 passing value by checkWithMaster() instead 2020-09-16 01:33:45 -07:00
Chris Lu
4d21de63ee go fmt 2020-09-16 01:27:24 -07:00
Chris Lu
913a16268d volume: load configuration from master at the start
fix https://github.com/chrislusf/seaweedfs/issues/1469
2020-09-16 01:27:05 -07:00
Chris Lu
c9202c4b3d add storage backend to GetMasterConfigurationResponse 2020-09-16 01:12:22 -07:00
Chris Lu
8a0710cb73 handle more than 2GB files
related to https://github.com/chrislusf/seaweedfs/issues/1468
2020-09-16 00:41:50 -07:00
Chris Lu
854007bc98 minor 2020-09-16 00:37:57 -07:00
Chris Lu
10f9081526 filer: adjust meta data events to received timestamp
if a client is already connected and start from t0. A message recieved at t+1 but with timestamp t-1 may not be processed by the client.

This commit changes to the event received time, so the replication can be ordered.
2020-09-15 01:18:33 -07:00
Chris Lu
684a875876 update help message 2020-09-15 00:40:38 -07:00
Chris Lu
2d3a904a82 shell: volumeServer.evacuate adds option to skip non moveable volumes 2020-09-15 00:33:49 -07:00
Chris Lu
a595916342 shell: add volumeServer.evacuate command 2020-09-14 23:47:11 -07:00
Chris Lu
103fafe00b weed export: print out [start,stop) content range 2020-09-14 22:57:23 -07:00
Chris Lu
ada996fe55 filer: support createing empty folder
fix https://github.com/chrislusf/seaweedfs/issues/1161
2020-09-14 13:07:46 -07:00
Chris Lu
200fe5c83e go fmt 2020-09-13 21:26:30 -07:00
Chris Lu
5d6753fb98 shell: add volumeServer.leave command 2020-09-13 21:25:51 -07:00
Chris Lu
1af95c5b76 refactoring 2020-09-13 12:41:26 -07:00
Chris Lu
f8fea19669 1.99 2020-09-12 13:46:33 -07:00
Chris Lu
b0c7de186d filer: fix postgres prefixed directory listing problem
fix https://github.com/chrislusf/seaweedfs/issues/1465
2020-09-12 13:37:03 -07:00
Chris Lu
1a7afe7e6a adjust default value
related to https://github.com/chrislusf/seaweedfs/issues/1453
2020-09-12 12:46:42 -07:00
Chris Lu
f2723c1bc8 do not idx file format
revert c9ab8d05fa
2020-09-12 12:42:36 -07:00
Chris Lu
ba984a4e29 1.98 2020-09-12 04:13:02 -07:00
Chris Lu
446e476a11 go fmt 2020-09-12 04:08:03 -07:00
Chris Lu
ea26a98753 volume: validate volume correctness if last entry is a deletion 2020-09-12 04:07:04 -07:00
Chris Lu
d15682b4a1 shell: volume.balance plan by ratio of fullness 2020-09-12 04:06:26 -07:00
Chris Lu
c0ee78d2fa adjust make file 2020-09-12 04:05:42 -07:00
Chris Lu
cd9b89ba55 reduce default wait time to 10s before shutting down 2020-09-12 04:05:33 -07:00
Chris Lu
e2c741f76f adjust replica placement after move 2020-09-12 01:01:19 -07:00
Chris Lu
2a0925590c filer: etcd store avoid read with nil option
fix https://github.com/chrislusf/seaweedfs/issues/1463
2020-09-11 15:29:45 -07:00
Chris Lu
3984c3962f add comment 2020-09-11 15:07:19 -07:00
Chris Lu
baa6bdf4d4 s3: listMultipartUploads fix output format 2020-09-11 15:04:01 -07:00
Chris Lu
3eda8d6dfc s3: ListParts output xml format
fix https://github.com/chrislusf/seaweedfs/issues/1461
2020-09-11 14:53:50 -07:00
Chris Lu
ab201c2798 1.97 2020-09-11 13:47:50 -07:00
Chris Lu
9d4bdfcfdf fix volume integrity checking 2020-09-11 11:34:10 -07:00
Chris Lu
1b8094ef75 weed export: export deleted files 2020-09-11 02:05:14 -07:00
Chris Lu
12a8f5294d test for multi dc replication 2020-09-11 01:21:17 -07:00
Chris Lu
e60b2117c3 shell: volume balance follows replica placement 2020-09-11 00:29:25 -07:00
Chris Lu
89a62e8007 refactoring 2020-09-10 23:05:00 -07:00
Chris Lu
19537c9d21 1.96 2020-09-10 19:52:07 -07:00
Chris Lu
eaf9fdde99 avoid sharing context over separate goroutine 2020-09-10 19:51:03 -07:00
Chris Lu
cc5fe6f5ee handle special characters in html link 2020-09-10 19:46:02 -07:00
Chris Lu
401ccf1509 fix test 2020-09-10 15:13:23 -07:00
Chris Lu
c9ab8d05fa fixes for reading deleted fid 2020-09-10 14:42:52 -07:00
ruitao.liu
5b0676049a change elastic initialize process similar as others. 2020-09-10 23:35:20 +08:00
Chris Lu
660d7c0edd 1.95 2020-09-10 01:32:05 -07:00
Chris Lu
0d9b858cfb
Merge pull request #1455 from taozix/master
add more basic elastic options.
2020-09-10 01:31:17 -07:00
ruitao.liu
9be4e97625 change logs print format. 2020-09-10 16:30:15 +08:00
ruitao.liu
719dc43af1 modify elastic urls from string to array. 2020-09-10 16:24:09 +08:00
ruitao.liu
72f9d7f047 use util to generate md5. 2020-09-10 16:11:18 +08:00
ruitao.liu
6a5b38c0d4 fix elastic kv ops. 2020-09-10 15:59:16 +08:00
ruitao.liu
3f7fbfddca add more basic elastic options. 2020-09-10 14:22:07 +08:00
Chris Lu
b183ae54b5 master: changing 301 to 308 on redirect to volume servers
fix https://github.com/chrislusf/seaweedfs/issues/1454
2020-09-09 22:36:10 -07:00
Chris Lu
47b3f932e4 watch: adjust output format 2020-09-09 22:34:48 -07:00
Chris Lu
daf0a449f7 properly cancel context for streaming grpc 2020-09-09 12:07:15 -07:00
Chris Lu
7f69acd1f2 sync pprof 2020-09-09 11:33:52 -07:00
Chris Lu
387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu
4fc0bd1a81 return http response directly 2020-09-09 03:53:09 -07:00
Chris Lu
5e13bc878c adjust log level 2020-09-09 03:49:26 -07:00
Chris Lu
d1b816212f return nil if no need to do anything 2020-09-07 16:03:05 -07:00
Chris Lu
64a621bcc8 shell: volume.fix.replication also purge over replicated volumes 2020-09-07 16:00:10 -07:00
Chris Lu
c18ea21f7a 1.94 2020-09-07 13:13:53 -07:00
Chris Lu
18b98fdb72 fix test 2020-09-07 12:57:38 -07:00
Chris Lu
44b3f2efc8 filer: Elastic Search return correct kv error
fix https://github.com/chrislusf/seaweedfs/issues/1452
2020-09-07 12:44:51 -07:00
Chris Lu
d80538a187 refactoring 2020-09-07 12:35:02 -07:00
Chris Lu
1a09bc43d1 refactor 2020-09-07 11:31:33 -07:00
Chris Lu
2b643f477d typo 2020-09-06 12:47:55 -07:00
Chris Lu
05034aade5 printout over replicated locations 2020-09-06 12:44:02 -07:00
Chris Lu
432fe3d926 1.93 2020-09-06 12:12:09 -07:00
Chris Lu
d8af6f62df adjust logging 2020-09-06 12:10:40 -07:00
Chris Lu
07f32feed0 print sync progress 2020-09-06 00:50:51 -07:00
Chris Lu
0af6252b2d fix compilation 2020-09-06 00:50:38 -07:00
Chris Lu
ac494ff5e8 Update meta_aggregator.go 2020-09-06 00:29:16 -07:00
Chris Lu
c9f8f25ba5 read peer filer from start 2020-09-06 00:12:41 -07:00
Chris Lu
30dc365cbd add self filer 2020-09-06 00:11:46 -07:00
Chris Lu
8acd714600 apply meta changes only if store is different 2020-09-05 23:25:03 -07:00
Chris Lu
d741ed66db check whether shares the same filer store 2020-09-05 23:02:52 -07:00
Chris Lu
bba90ff3c8 read filer signature 2020-09-05 22:52:15 -07:00
Chris Lu
71b0e256c7 filer: setOrLoadFilerStoreSignature 2020-09-05 14:08:59 -07:00
ruitao.liu
450cf07505 skip the index that for kv usage. 2020-09-04 21:49:03 +08:00
ruitao.liu
1384ff9a2f 1.split kv in one file.
2.disable query for kv in es index.
2020-09-04 17:34:26 +08:00
ruitao.liu
83080b5e03 ES backended filer support kv ops. 2020-09-04 15:40:13 +08:00
Chris Lu
46f65a84a6 filer: elastic7 adjust default value 2020-09-03 20:12:38 -07:00
Chris Lu
f815fce81b
Merge pull request #1446 from taozix/master
new filer store option to es7.
2020-09-03 20:10:23 -07:00
Chris Lu
44b4ebf0ff filer: Redis cleanly delete directory
fix https://github.com/chrislusf/seaweedfs/issues/1448
2020-09-03 19:17:39 -07:00
Chris Lu
b8f32bcab9 filer: compress stored metadata 2020-09-03 11:00:20 -07:00
Chris Lu
f76a2b2c8a printout meta data size 2020-09-03 09:51:21 -07:00
ruitao.liu
798280e980 change filer2 to filer. 2020-09-03 17:05:26 +08:00
ruitao.liu
a93d27d1e8 new filer option to es v7. 2020-09-03 16:34:58 +08:00
Chris Lu
1d56ea24ef fix 2020-09-03 00:08:37 -07:00
Chris Lu
7e1aad0b54 mount: map uid/gid between local and filer 2020-09-03 00:07:22 -07:00
Chris Lu
68e878adb5 fix formatting 2020-09-02 21:42:12 -07:00
Chris Lu
0d99a5da3c c*: support kv operations 2020-09-02 18:39:24 -07:00
Chris Lu
645a4af3db mongodb: support kv operations 2020-09-02 17:19:14 -07:00
Chris Lu
ecaa30c408 better error message 2020-09-02 17:17:44 -07:00
Chris Lu
7c770b727c Update abstract_sql_store_kv.go 2020-09-02 10:13:48 -07:00
Chris Lu
05a6e2dc7f sql kv upsert 2020-09-02 10:12:44 -07:00
Chris Lu
ed62f52452 fix sql insert with duplicated primary key 2020-09-02 10:09:49 -07:00
Chris Lu
9ea290aa12 fix error message 2020-09-01 23:21:41 -07:00
Chris Lu
06a1188267 mysql/postgres: support kv operations 2020-09-01 22:48:36 -07:00
Chris Lu
aa40295f06 leveldb: report not found entry 2020-09-01 22:48:23 -07:00
Chris Lu
87d7312bf6 mysql/postgres: properly report entry not found 2020-09-01 22:47:57 -07:00
Chris Lu
72b0a5f1d1 mysql or postgres: log find error 2020-09-01 22:25:17 -07:00
Chris Lu
2815bbe6c0 go fmt 2020-09-01 21:59:26 -07:00
Chris Lu
37234bf3f8 filer store adds kv support
can compile now, need to implement those unimplemented
2020-09-01 21:59:00 -07:00
James Hartig
8e54e34576 volume: Don't unmount before deleting volume in copy
If we unmount first and then delete, the delete fails because the volume
was unmounted. Delete ends up doing the same thing as the unmount anyways.
2020-09-01 22:00:07 -04:00
Chris Lu
2b14ae5819 fix tests 2020-09-01 01:33:43 -07:00
Chris Lu
d91ec535b3 fix tests 2020-09-01 01:29:13 -07:00
Chris Lu
eb7929a971 rename filer2 to filer 2020-09-01 00:21:19 -07:00
Chris Lu
38e06d783d volume: check disk space before compaction
fix https://github.com/chrislusf/seaweedfs/issues/1440
2020-08-31 18:10:53 -07:00
Chris Lu
dbb10e0f0b testing with more than 30days 2020-08-31 17:59:29 -07:00
Chris Lu
dd176cdb8b avoid tiny step pagination 2020-08-31 11:28:03 -07:00
Chris Lu
97733c3dff typo 2020-08-31 10:49:17 -07:00
Chris Lu
c8398bdb09 adjust metrics 2020-08-31 10:41:05 -07:00
Chris Lu
8a1c8e41b3 simplify if else logic 2020-08-31 10:39:24 -07:00
Chris Lu
f0c89cfacd go fmt 2020-08-31 10:23:31 -07:00
Chris Lu
edb9d65e05
Merge pull request #1431 from kmlebedev/wip-prefix-search
Wip prefix search
2020-08-31 10:22:14 -07:00
Konstantin Lebedev
9a195bebfd accurate limit 2020-08-31 22:13:13 +05:00
Konstantin Lebedev
18b98f9747 not convert err to string 2020-08-31 21:55:18 +05:00
Konstantin Lebedev
22f32e75c4 rename 2020-08-31 21:52:05 +05:00
Konstantin Lebedev
a8de332beb fix 2020-08-31 21:45:44 +05:00
Konstantin Lebedev
7f013d77d6 add var UnsupportedListDirectoryPrefixedErr 2020-08-31 21:43:32 +05:00
Konstantin Lebedev
60a86cfe04 add const 2020-08-31 21:35:16 +05:00
Konstantin Lebedev
82ea121d09 rm func 2020-08-31 18:50:06 +05:00
Konstantin Lebedev
2ea638f865 filer store wrapper can implement the logic to filter by prefi 2020-08-31 17:13:56 +05:00
Chris Lu
408e339c53 also delete the manifest chunk itself 2020-08-31 00:16:03 -07:00
Chris Lu
dd52037c27 1.92 2020-08-30 22:56:35 -07:00
Chris Lu
33d8c6c617 change log to warning 2020-08-30 21:01:44 -07:00
Chris Lu
e62d7f221f a bit more logs 2020-08-30 21:00:24 -07:00
Chris Lu
22fe4ae573 mount: fix for UrBackup 2020-08-30 21:00:09 -07:00
Chris Lu
ae84a9ee2e add logs 2020-08-30 20:59:04 -07:00
Chris Lu
a41588279a change log level 5 to 4 2020-08-30 20:12:04 -07:00
Chris Lu
56244fb9a1 fix hard coded host address 2020-08-30 10:23:35 -07:00
Chris Lu
f2a8574448 filer and mount deletion resolves manifest chunks also 2020-08-30 02:07:14 -07:00
Chris Lu
99ecf63276 go fmt 2020-08-29 22:28:33 -07:00
Chris Lu
ef75ce8a34 use UTC for all time related folders 2020-08-29 21:01:14 -07:00
Chris Lu
b69cb74c03 read meta logs by timestamp
pass in event ts when moving logs
meta aggregator reads in memory logs only
2020-08-29 17:37:19 -07:00
Chris Lu
063c9ddac5 adjust logs 2020-08-29 11:56:22 -07:00
Chris Lu
ca658a97c5 add signatures to messages to avoid double processing 2020-08-28 23:48:48 -07:00
Chris Lu
63ad1abcce watch: follow changes for a pattern 2020-08-28 19:43:04 -07:00
Chris Lu
b7a654a318 file mime avoid saving application/octet-stream 2020-08-28 19:42:40 -07:00
Chris Lu
707192f966 fix help message 2020-08-27 00:13:38 -07:00
Chris Lu
0be6863c87 rename 2020-08-26 22:40:15 -07:00
Chris Lu
50f7e2e89f add log messages 2020-08-26 09:51:20 -07:00
Chris Lu
6bc618051c log message 2020-08-26 09:32:56 -07:00
Chris Lu
9dae4e9d3e fix logs 2020-08-26 09:31:11 -07:00
Chris Lu
4c1a3187bf fix log error 2020-08-26 09:25:54 -07:00
Chris Lu
ab759f0ec2 erasure coding: fix EC error if multiple disks are configured in one volume server 2020-08-26 09:16:58 -07:00
Chris Lu
d1cf39f180 fix logging 2020-08-26 08:55:15 -07:00
Chris Lu
83012f543a s: avoid possible completeMultipartUpload re-tries that can overwrite the uploaded file 2020-08-26 08:50:24 -07:00
Konstantin Lebedev
464d4c82ec stop send heartbeat before stop volume server 2020-08-26 16:16:11 +05:00
Chris Lu
1901f15cd2 volume server: remove whitelist for status checking
Volume liveness/readiness not work if jwt enabled, I try use param
-whiteList
but jwt breaks
2020-08-25 02:27:47 -07:00
Chris Lu
81e5124faf s3: list bucket permission change from admin to read
fix https://github.com/chrislusf/seaweedfs/issues/1430
2020-08-24 11:22:45 -07:00
Chris Lu
5f55a87101 close http response 2020-08-24 00:32:44 -07:00
Chris Lu
0b301b0b47 consume all response body in order to release requests 2020-08-23 21:32:29 -07:00
Chris Lu
2cdad77d82 fix checkin error 2020-08-23 18:30:11 -07:00
Chris Lu
81a8aa6581 1.91 2020-08-23 17:28:40 -07:00
Chris Lu
51346a5930 always keep the manifest list of chunks 2020-08-23 17:15:12 -07:00
Chris Lu
c1d1677a28 keep manifest chunks forever 2020-08-23 16:59:01 -07:00
Chris Lu
98175548c2 adjust deletion ordering 2020-08-23 16:27:12 -07:00
Chris Lu
f0e50e7b26 fix test 2020-08-23 16:12:34 -07:00
Chris Lu
aee27ccbe1 multiple fixes
* adjust isOpen count
* move ContinuousDirtyPages lock to filehandle
* fix problem with MergeIntoVisibles, avoid reusing slices
* let filer delete the garbage
2020-08-23 15:48:02 -07:00
Chris Lu
c19245886c simpler test 2020-08-23 14:20:27 -07:00
Chris Lu
df816a58fe add tests 2020-08-23 14:09:25 -07:00
Chris Lu
77393d3d30 add file handle locking when changing file entry 2020-08-23 00:35:50 -07:00
Chris Lu
d60bcbf08a sorting chunks 2020-08-23 00:00:36 -07:00
Chris Lu
5e6b714836 add random test 2020-08-22 16:33:00 -07:00
Chris Lu
42ddbcc3a2 SQL insert falls back to update
fix https://github.com/chrislusf/seaweedfs/issues/1429
2020-08-20 07:52:46 -07:00
Chris Lu
f48567c5c6 remove unused function 2020-08-19 22:53:49 -07:00
Chris Lu
c45ba5d7d4 fix listObjectsV2 response format
fix https://github.com/chrislusf/seaweedfs/issues/1426 issue 1
2020-08-19 12:08:04 -07:00
James Hartig
3ccfa4c6ad Added VolumeMarkWritable and VolumeStatus grpc methods
This is necessary for copy to mark as read-only and then restore the
original state afterwards.
2020-08-19 11:42:56 -04:00
Chris Lu
3b4b1d4a77 fix tests 2020-08-19 01:37:56 -07:00
Chris Lu
839634097f also do flush on release 2020-08-19 01:27:10 -07:00
Chris Lu
c27e18aa6a read possible old deleted chunks 2020-08-19 00:43:07 -07:00
Chris Lu
a78772d5ea avoid shutdown in the middle of running 2020-08-19 00:42:02 -07:00
Chris Lu
ae9bc4a508 logs 2020-08-18 23:42:26 -07:00
Chris Lu
ed4b43b419 adjust logs 2020-08-18 23:42:09 -07:00
Chris Lu
9f1e0aeef5 delete chunks in the last step 2020-08-18 23:40:53 -07:00
Chris Lu
99d05f758c adjust logs 2020-08-18 23:39:18 -07:00
Chris Lu
fe01191b5b support read option readDeleted=true 2020-08-18 19:22:16 -07:00
Chris Lu
6ccd7f0a4d refactoring 2020-08-18 18:01:37 -07:00
Chris Lu
c026eb0592 refactoring 2020-08-18 17:39:29 -07:00
Chris Lu
7e91ae592c pass in option to read deleted entries
not working yet
2020-08-18 17:37:26 -07:00
Chris Lu
51ecb49db3 for debugging 2020-08-18 17:35:39 -07:00
Chris Lu
ee11d98650 refactoring 2020-08-18 17:35:19 -07:00
Chris Lu
6a92f0bc7a refactoring to typed Size
Go is amazing with refactoring!
2020-08-18 17:04:28 -07:00
Chris Lu
618b2f6829 release resources only when needed to 2020-08-18 12:53:08 -07:00
Chris Lu
208849702d logs 2020-08-18 12:52:54 -07:00
Chris Lu
1fcd083db3 printout data size 2020-08-18 09:09:29 -07:00
Chris Lu
3e5339337a minor 2020-08-18 08:50:14 -07:00
Chris Lu
85001cbec7 properly report io.EOF 2020-08-18 08:18:54 -07:00
Chris Lu
0ca45a5cbc adjust logs 2020-08-18 00:34:15 -07:00
Chris Lu
cd43738245 fix reading when filling zeros 2020-08-18 00:32:01 -07:00
Chris Lu
ecb3ce46be adjust error logs 2020-08-18 00:31:42 -07:00
Chris Lu
30fe424469 add one more test case 2020-08-17 22:47:27 -07:00
Chris Lu
1b68ba953b fix for out of range reads 2020-08-17 22:46:32 -07:00
Chris Lu
56fbd2c211 fix reading 2020-08-17 21:17:32 -07:00
Chris Lu
09e126bae5 refactoring: use interface 2020-08-17 20:20:08 -07:00
Chris Lu
be4d42b8e2 rename 2020-08-17 20:15:53 -07:00
Chris Lu
0625e63648 count 0 as part of the reads 2020-08-17 20:14:40 -07:00
Chris Lu
0be4b6e7f0 logs 2020-08-17 16:05:40 -07:00
Chris Lu
97e54a80d4 rename variables 2020-08-17 16:05:13 -07:00
Chris Lu
abdaf9958d possibly read more 2020-08-17 16:04:56 -07:00
Chris Lu
9d46c7bc78 rename 2020-08-17 11:12:10 -07:00
Chris Lu
4ccfdaeb4d prevent nil 2020-08-17 10:07:34 -07:00
Chris Lu
24c8e6bcb4 minor optimization 2020-08-17 10:03:34 -07:00
Chris Lu
f5837b7000 report error first 2020-08-16 23:49:10 -07:00
Chris Lu
2ac27616bc fix possible out of range bytes
avoid buff out of range resp.Data = buff[:totalRead]
2020-08-16 23:47:34 -07:00
Chris Lu
ff200398bb 1.90 2020-08-16 21:18:50 -07:00
Chris Lu
4ceeba9e70 streaming reads 2020-08-16 21:07:46 -07:00
Chris Lu
e72953dff7 logs 2020-08-16 21:06:03 -07:00
Chris Lu
627b081b67 adjust logs 2020-08-16 16:32:22 -07:00
Chris Lu
22e5132b3a adjust log level 2020-08-16 16:25:11 -07:00
Chris Lu
ee0f92a6be reduce memory allocation 2020-08-16 16:24:40 -07:00
Chris Lu
2ba817afac read randomly written data 2020-08-16 15:16:46 -07:00
Chris Lu
6111b265e7 fix compilation 2020-08-16 01:38:16 -07:00
Chris Lu
4a77f0820a clean up logs 2020-08-16 01:37:50 -07:00
Chris Lu
8c9e6eaacd fix tests 2020-08-16 01:35:52 -07:00
Chris Lu
b71df82292 1.89 2020-08-16 00:57:28 -07:00
Chris Lu
1d9ea30b72 fix ViewFromVisibleIntervals 2020-08-16 00:49:26 -07:00
Chris Lu
aec7f32b02 fix reader_at 2020-08-16 00:49:08 -07:00
Chris Lu
5d80fc2ec7 adjust logs 2020-08-15 21:09:31 -07:00
Chris Lu
e0bfd3161a update metadata only if changed 2020-08-15 19:56:24 -07:00
Chris Lu
003d48da21 adjust logs 2020-08-15 19:55:28 -07:00
Chris Lu
6ee8d952d2 adjust log level 2020-08-15 18:24:35 -07:00
Chris Lu
3e1395b767 adjust log message 2020-08-15 17:06:16 -07:00
Chris Lu
a22ee30596 fix nil 2020-08-15 17:01:42 -07:00
Chris Lu
0d60e67816 ensure meta data changes are updated 2020-08-15 14:15:07 -07:00
Chris Lu
5b43bddf20 proper deletion ordering
delete central file store first, then delete local cache
2020-08-15 09:33:41 -07:00
Chris Lu
c647deace1 file size support set file length
use Attr.FileSize and TotalChunkSize to determine file size
2020-08-15 09:32:47 -07:00
Chris Lu
c03bb180eb fix error reporting 2020-08-14 00:44:02 -07:00
Chris Lu
eb493283dd deletion error report 2020-08-14 00:23:01 -07:00
Chris Lu
a7f669044e rename also applies to open file handle 2020-08-14 00:22:49 -07:00
Chris Lu
edfa73782f adjust log level 2020-08-14 00:22:21 -07:00
Chris Lu
0983060a90 increase default volume file size limit to 1024
avoid possible large chunk size set on mount or filer
2020-08-13 09:07:22 -07:00
Chris Lu
090612492c fix compilation 2020-08-13 00:07:56 -07:00
Chris Lu
d43129d27f add back handles lock 2020-08-12 23:52:13 -07:00
Chris Lu
f735d579d3 adjust comment 2020-08-12 13:22:19 -07:00
Chris Lu
8824a9755c remove directory cache 2020-08-12 13:11:04 -07:00
Chris Lu
83cad3da79 add retry file upload 2020-08-11 20:30:11 -07:00
Chris Lu
152a6cbc2b minor adjustments 2020-08-10 20:42:27 -07:00
Chris Lu
f1b40567e3
Merge pull request #1419 from robin1900/master
fix bug:  two same volumeId in different collections
2020-08-10 20:38:55 -07:00
cheng.li01
25fbff5d52 fix bug: two same volumeId in different collections
1, there will be two leader when master server startup in a few seconds
2, raft server will get a leader even there is only one master, so there is no need to do hard code to set the server to be leader
2020-08-10 16:37:47 +08:00
Chris Lu
e74dc4e4bc add back fs node cache for renaming 2020-08-09 21:56:09 -07:00
Chris Lu
f86c7d911a remove println 2020-08-09 14:42:25 -07:00
Chris Lu
9ecc1170a3 istObjects和listObjectsV2不能查询子目录 #1418
fix https://github.com/chrislusf/seaweedfs/issues/1418
2020-08-09 14:35:53 -07:00
Chris Lu
3f4aff5dde s3: fix delimiter in list response 2020-08-09 09:09:35 -07:00
Chris Lu
b056707770 1.88 2020-08-08 22:55:12 -07:00
Chris Lu
9832653e1d FUSE mount: proper error with deleting non empty folder 2020-08-08 21:37:36 -07:00
Chris Lu
3b1a95ac26 filer refactoring: same auto chunking logic for POST and PUT, no size limit 2020-08-08 12:02:06 -07:00
Chris Lu
67348e7b15 less noisy heartbeat logs 2020-08-08 10:53:35 -07:00
Chris Lu
bee0d7e5eb lower log priority for noisy heartbeat 2020-08-08 10:52:13 -07:00
Chris Lu
ae00cce4bd support POST and PUT auto chunking 2020-08-08 10:45:37 -07:00
Chris Lu
bd8bfdae07 refactoring 2020-08-08 10:18:43 -07:00
Chris Lu
ab6e5c0dc4 adjust error message 2020-08-08 10:18:32 -07:00
Chris Lu
828a5ae429 check signature only when auth is enabled 2020-08-08 09:11:40 -07:00
Chris Lu
20e2ac1add filer: store md5 metadata for files uploaded by filer
fix https://github.com/chrislusf/seaweedfs/issues/1412
2020-08-06 10:04:17 -07:00
Chris Lu
93ea0801ea volume: the variable for the master node may be stale?
related to https://github.com/chrislusf/seaweedfs/issues/1414
2020-08-06 09:48:54 -07:00
Chris Lu
4ecfa9879d volume: report Content-MD5 in response header 2020-08-06 05:22:53 -07:00
Chris Lu
41007ced77 remove logging 2020-08-06 04:26:29 -07:00
Chris Lu
2b74abf766 S3: configurable access for anonymous user
fix https://github.com/chrislusf/seaweedfs/issues/1413
2020-08-06 03:41:34 -07:00
Устюжанин Антон Александрович
9ca011e3ca ListDirectoryPrefixedEntries 2020-08-06 00:38:59 +05:00
Устюжанин Антон Александрович
b231f7bdab ListDirectoryPrefixedEntries 2020-08-06 00:37:42 +05:00
Устюжанин Антон Александрович
a457c308ad test ListDirectoryPrefixedEntries 2020-08-06 00:24:31 +05:00
Устюжанин Антон Александрович
2d3b355fb6 test ListDirectoryPrefixedEntries 2020-08-05 23:56:06 +05:00
Устюжанин Антон Александрович
67f134ebd7 test ListDirectoryPrefixedEntries 2020-08-05 23:44:48 +05:00
Устюжанин Антон Александрович
dc9fc01254 test ListDirectoryPrefixedEntries 2020-08-05 23:38:00 +05:00
Устюжанин Антон Александрович
33a9e5e2d1 test ListDirectoryPrefixedEntries 2020-08-05 22:19:16 +05:00
Chris Lu
cbd80253e3 better needle id format 2020-08-04 22:09:07 -07:00
Chris Lu
1b3a80dd3d non-fatal error 2020-08-01 13:46:52 -07:00
Chris Lu
bbbea8159c http request use gzip if possible 2020-08-01 13:20:52 -07:00
Chris Lu
2f03481cb2 in case when content is not compressed 2020-08-01 13:08:10 -07:00
Chris Lu
49199c69d4 volume: avoid special logic for .gz files 2020-08-01 11:19:57 -07:00
Chris Lu
f89186a3cf better logging 2020-08-01 11:19:57 -07:00
Chris Lu
49d6dcfbc6 dynamically change the log level for reading the disk status 2020-08-01 11:19:57 -07:00
limd
b41b7ea4d0 fix spark read s3 bug (sc.binaryFiles) 2020-08-01 01:08:30 +08:00
Chris Lu
a6b59d50f7 1.87 2020-07-30 13:52:04 -07:00
Chris Lu
eed525b717 FUSE mount: remove DirListCacheLimit
outdated parameter
2020-07-28 09:24:39 -07:00
Chris Lu
a566bfc6e1 s3: use bucket in the domain
fix https://github.com/chrislusf/seaweedfs/issues/1405
2020-07-28 08:47:27 -07:00
limd
92c32f9d46 PutObject: fix create folder bug 2020-07-28 00:58:42 +08:00
Chris Lu
54cd582653 refactor 2020-07-26 12:58:58 -07:00
Chris Lu
c50ec1599c auth for presigned put object part 2020-07-26 12:09:50 -07:00
Chris Lu
ea0180de4e auth for presigned put operation 2020-07-26 12:06:45 -07:00
Chris Lu
3faa0aa448 rename 2020-07-25 20:09:19 -07:00
Chris Lu
023a1efdf2 set filename in Content-Disposition header 2020-07-25 20:06:40 -07:00
popstk
a09fcfcf20 fix s3api auth bug 2020-07-25 17:14:49 +08:00
popstk
feb52400be fix s3api copy object handler SerializationError 2020-07-25 15:52:31 +08:00
Chris Lu
d40de39e75 FUSE: do not change crtime, uid, gid on save 2020-07-24 10:13:45 -07:00
Chris Lu
8dfeba8023 FUSE: skip changing to empty uid and gid during flush
Skip uid and gid changes if request uid and gid are zero

mongodb lock file, and many interim files, has empty uid and gid

I0724 10:05:51 93643 filehandle.go:173] /db/diagnostic.data/metrics.interim fh 1333342842031408359 flush Flush [ID=0x3 Node=0x50 Uid=0 Gid=0 Pid=178] 0x10 fl=0x0 lk=0x0
2020-07-24 10:06:43 -07:00
Chris Lu
dd29b8c81d fix compilation 2020-07-24 09:36:05 -07:00
Chris Lu
5788f1f6bf remove fsnode cache
fix https://github.com/chrislusf/seaweedfs/issues/1400
2020-07-24 09:25:54 -07:00
Chris Lu
6962ca48d3 fix compilation 2020-07-23 21:31:23 -07:00
Chris Lu
df7eee523b FUSE: implement dir fsync
fix https://github.com/chrislusf/seaweedfs/issues/1398
2020-07-23 21:09:40 -07:00
Chris Lu
3bbac0820b FUSE: faster rename implementation 2020-07-23 21:08:42 -07:00
Chris Lu
943fa4ea0c fix compilation error 2020-07-22 16:11:17 -07:00
Chris Lu
49446af516 simplify 2020-07-22 13:43:54 -07:00
James Hartig
229f11c660 Added VolumeNeedleStatus volume server grpc method
This is needed for the diffing tool to get the cookie for a needle
2020-07-22 15:02:21 -04:00
James Hartig
0be3bb4a87 idx: Switch WalkIndexFile to accept generic io.ReaderAt
This is necessary to use it with in-memory idx files.
2020-07-22 15:01:16 -04:00
Chris Lu
7a7e70b6cc 1.86 2020-07-20 23:39:58 -07:00
Chris Lu
885c624bce volume.fsck: follow manifest chunks 2020-07-20 22:02:05 -07:00
Chris Lu
44057a4de1 clean up chunks in manifest 2020-07-20 22:01:39 -07:00
Chris Lu
0b2e06268b use merge factor 1000 2020-07-20 18:30:09 -07:00
Chris Lu
d02c0fe0c0 refactoring 2020-07-20 03:34:06 -07:00
Chris Lu
60d14a9800 mount: fix difference with storage format in local cache 2020-07-20 00:17:50 -07:00
Chris Lu
97d97f3528 go code can read and write chunk manifest 2020-07-19 17:59:43 -07:00
李远军
4162766c46 Fix the bug of "http://xx/xx,xxx.PNG?width=100"
If you want to do a resize a UPPER ext pic file, there is a bug caused by commit 308688c8d0 (also see https://github.com/chrislusf/seaweedfs/blob/master/weed/images/resizing.go#L43)
2020-07-17 19:57:35 +08:00
Chris Lu
f43146b237 resolve directories if containing home directory 2020-07-16 22:50:14 -07:00
Chris Lu
befb396892 mount: resolve home directory
fix https://github.com/chrislusf/seaweedfs/issues/1391
2020-07-16 22:38:55 -07:00
Chris Lu
2b2ca7d1dc stop if the directory can not be found
related to https://github.com/chrislusf/seaweedfs/issues/1391
2020-07-16 13:24:53 -07:00
Chris Lu
2286d27730 rename 2020-07-15 14:19:15 -07:00
Chris Lu
0db4204c81 add is_file_chunks to filer.proto 2020-07-14 22:42:52 -07:00
Chris Lu
bbc8ae110f report errors 2020-07-14 11:27:00 -07:00
Chris Lu
17f2d05cde filer: ensure logs are persisted 2020-07-14 11:25:50 -07:00
Chris Lu
ddec7b2bb9 go fmt 2020-07-14 07:34:16 -07:00
Chris Lu
4f6096c7f0 add reading from persisted logs for local filer store 2020-07-13 22:55:28 -07:00
Chris Lu
87b5031714 change variable visibility 2020-07-13 08:19:48 -07:00
Chris Lu
7597831cac filer: leveldb2 supports peers also 2020-07-13 00:05:40 -07:00
Chris Lu
5ba894bb72 leveldb2: support subscribing from peers 2020-07-13 00:05:40 -07:00
Chris Lu
5a4f258a8c 1.85 2020-07-12 17:34:09 -07:00
Chris Lu
4eecc6abc6 filer: emit metadata notification correctly for batch deletion 2020-07-12 17:32:19 -07:00
Chris Lu
1dd2c76532 refactoring 2020-07-12 17:31:24 -07:00
Chris Lu
224103e13b aggregated logs has empty flushFn 2020-07-11 09:12:03 -07:00
Chris Lu
bfabb4159b refactoring 2020-07-11 09:11:15 -07:00
Chris Lu
f2e8ad6241 refactoring 2020-07-11 06:35:54 -07:00
Chris Lu
2d1df97933 mount: fatal error if can not process meta data 2020-07-11 06:16:48 -07:00
Chris Lu
e3e00d36c5 reset cache for each new release 2020-07-11 06:16:17 -07:00
Chris Lu
7ad0bce2f5 remove old code 2020-07-10 23:03:22 -07:00
Chris Lu
e4ef3814e7 fix compilation 2020-07-10 10:02:59 -07:00
Chris Lu
e7c04af1d0 skip http.StatusPartialContent in case of error and superfluous response.WriteHeader 2020-07-10 08:56:23 -07:00
李远军
8c318470dd Revert "remove fixJpgOrientation"
This reverts commit de5ca9b2

minimise the revert changes
2020-07-10 10:28:28 +08:00
李远军
6608cb5f43 Revert "remove fixJpgOrientation"
This reverts commit de5ca9b2
2020-07-10 10:08:36 +08:00
cheng.li01
dad1161c70 fix dn.volumes Iterate when write issue 2020-07-08 19:57:19 +08:00
Chris Lu
54d6b3c30f filer: support running multiple mount to different filers avoiding duplicated cache
fix https://github.com/chrislusf/seaweedfs/issues/1382
2020-07-07 23:18:44 -07:00
Chris Lu
4bed1663e2 adjust log level 2020-07-07 23:17:31 -07:00
Chris Lu
482d13468b remove println 2020-07-07 23:17:17 -07:00
Chris Lu
a2eb680f34 filer: default filer store directory
1. set default filer store directory
2. set peers, avoiding empty string counted as 1.
2020-07-07 23:06:48 -07:00
Chris Lu
3c269da37f fix compilation 2020-07-06 09:46:24 -07:00
Chris Lu
d198e8c6d6 filer: support cross filer meta data sync if sharing the same store 2020-07-06 09:24:54 -07:00
Chris Lu
0b8cc9b626 aggregate multiple filer metadata chagne events 2020-07-05 23:05:02 -07:00
Chris Lu
49929e0869 rename 2020-07-05 15:52:36 -07:00
Chris Lu
70d8a3a1d3 add SubscribeLocalMetadata without checking persisted meta logs 2020-07-05 15:50:07 -07:00
Chris Lu
55e40b08fc refactoring 2020-07-05 15:43:06 -07:00
Chris Lu
881e0fde2e rename 2020-07-05 08:55:09 -07:00
Chris Lu
353bea8ddb refactoring 2020-07-03 22:25:35 -07:00
Chris Lu
ce3630e7a4 refactoring 2020-07-03 22:16:59 -07:00
Chris Lu
59152334c7 default to 1% 2020-07-03 16:42:53 -07:00
Chris Lu
acf4a9ea99 volume: min free space default to 1% 2020-07-03 16:42:02 -07:00
Chris Lu
4d1484628a refactoring 2020-07-03 16:41:30 -07:00
Chris Lu
faa5c2e89a refactoring 2020-07-03 16:34:31 -07:00
Chris Lu
37d5b3ba12 replication: pass isFromOtherCluster also to EventNotification
EventNotification is consistent with message queue and metadata logs.
2020-07-01 08:06:20 -07:00
Chris Lu
31e23e9783 filer: support active<=>active filer replication 2020-06-30 22:53:57 -07:00
Chris Lu
df75b5c98d passing ctx for metadata logging 2020-06-28 14:34:51 -07:00
Chris Lu
12f6b683fc default to 8 volumes
in case all the volumes are allocated for a collection
2020-06-28 14:34:21 -07:00
Chris Lu
42a338d7b0 FUSE mount: clean up file handles during renaming
resolve dir rename when file is still open. Need to clean the file handles as soon as possible.

These can happen out of order:
file rename, then file release
file release, then file rename
2020-06-28 13:41:00 -07:00
Chris Lu
95a1860d65 remove empty calls for list cache 2020-06-28 10:25:54 -07:00
Chris Lu
6498d0c86b remove list cache 2020-06-28 10:21:52 -07:00
Chris Lu
9033a7d369 removed async option 2020-06-28 10:18:32 -07:00
Chris Lu
1bb8cae65d reverting and working 2020-06-28 10:14:17 -07:00
Chris Lu
b813fac4a3 1.84 2020-06-28 07:28:21 -07:00
Chris Lu
20d9cfae86 volume: following md5 base64 encoding
fix https://github.com/chrislusf/seaweedfs/issues/1374
2020-06-28 07:25:50 -07:00
Chris Lu
f0f884edd5 1.83 2020-06-27 15:46:59 -07:00
Chris Lu
c2797cc6de add logs 2020-06-27 15:46:53 -07:00
Chris Lu
62260e526b add logging 2020-06-27 12:54:37 -07:00
Chris Lu
bef356ce4c since we already know the chunk size, no need to iterate 2020-06-27 12:51:04 -07:00
Chris Lu
a808b3b5df incase the memory data is too small 2020-06-27 11:59:15 -07:00
Chris Lu
3dbd51c3c2 a little bit more efficient 2020-06-26 10:02:37 -07:00
Chris Lu
212b6e7d42 error logging 2020-06-26 10:01:55 -07:00
Chris Lu
48b23f2fdd FUSE mount: prevent concurrent modification 2020-06-26 10:00:48 -07:00
Chris Lu
3cec4b3c49 FUSE mount: fix file id written twice?
fix https://github.com/chrislusf/seaweedfs/issues/1373
2020-06-25 20:11:49 -07:00
Chris Lu
212c64241b checking ecx file 2020-06-25 16:36:06 -07:00
Chris Lu
a4dc25cd22 volume: generate ec shard first, before generating the ec index file.
possible fix https://github.com/chrislusf/seaweedfs/issues/1364

the ec index file is used to determine whether the ec files are ready or not
2020-06-25 11:06:58 -07:00
Chris Lu
9fd7cdadf1 fix 2020-06-25 10:45:34 -07:00
Chris Lu
3b638d3994 add more ec encoding logging 2020-06-25 09:43:38 -07:00
Chris Lu
e8e0d629af ensure correct file mode 2020-06-24 23:26:54 -07:00
Chris Lu
32a1f22730 more logs 2020-06-24 22:07:53 -07:00
Chris Lu
5be12eea37 zstd fix 2020-06-24 11:39:12 -07:00
Chris Lu
2ff37ccdbd testing parsing the upload
The mime type is always the value passed in.

Compress or not depends on the content detection, file name extension, and compression ratio.

If the content is already compressed, need to know the content size.
2020-06-24 11:39:12 -07:00
Chris Lu
c21f4ebfee
Merge pull request #1371 from Kimbsen/content_md5_validation
Optional md5 validation of uploads
2020-06-24 11:33:44 -07:00
Kimbsen
4eeab2a379 remove unused imports 2020-06-24 13:37:00 +02:00
Kimbsen
ffddecebef do md5 validation AFTER decompression 2020-06-24 13:35:13 +02:00
Chris Lu
fe60db404a prepare for zstd 2020-06-23 09:12:02 -07:00
Kimbsen
6b1e93ba0b check request header not part header for content-encoding 2020-06-23 17:04:24 +02:00
Kimbsen
38626cb584 don't forget the last few bytes 2020-06-23 16:53:49 +02:00
Kimbsen
8d1f6cd567 added optional md5 verification of uploaded data 2020-06-23 16:24:11 +02:00
Chris Lu
a4e3cffe0a S3: DeleteObject API also support deleting a folder recursively 2020-06-22 10:01:00 -07:00
Chris Lu
98493e0152 volume: add default minFreeSpacePercent
fix https://github.com/chrislusf/seaweedfs/issues/1366
2020-06-21 08:44:06 -07:00
Chris Lu
e7c36f740a refactoring 2020-06-21 08:38:00 -07:00
Chris Lu
8b4a32e782 adjust help message 2020-06-21 08:34:24 -07:00
Chris Lu
b4ec04d464 fix the help message 2020-06-21 08:31:20 -07:00
Chris Lu
de5ca9b258 remove fixJpgOrientation 2020-06-20 12:50:40 -07:00
Chris Lu
3080c197e3 rename UnCompressData to DecompressData 2020-06-20 08:16:16 -07:00
Chris Lu
97239ce6f1 rename filechunk is_gzipped to is_compressed 2020-06-20 08:15:49 -07:00
Chris Lu
a735fd767c adjust uncompress data 2020-06-20 08:01:00 -07:00
Chris Lu
ca3516ac6d adjust protoc 2020-06-20 08:00:25 -07:00
Chris Lu
e912fd15e3 renaming 2020-06-19 22:45:27 -07:00
Chris Lu
16fe132a20 detect mime type before replicating to other volume servers 2020-06-19 22:11:36 -07:00
Chris Lu
044841c885 master: always clear previous master meta data directory 2020-06-19 20:42:16 -07:00
Chris Lu
2eeb7551f9 less noisy disk size checking 2020-06-19 15:06:33 -07:00
Chris Lu
c48b407053 1.82 2020-06-19 13:47:27 -07:00
Chris Lu
b78a2dc59b adjust help message 2020-06-19 12:55:09 -07:00
Chris Lu
f7a45d448f FUSE mount: lazy loading meta cache 2020-06-19 09:45:42 -07:00
Chris Lu
c0283eee1a more test cases 2020-06-19 09:45:42 -07:00
Chris Lu
55b6efb755 fix checking visited nodes 2020-06-19 09:45:42 -07:00
Chris Lu
0e7c1a300b
Merge pull request #1363 from DXist/fix_utc
Return last modified in UTC
2020-06-19 08:49:00 -07:00
Rinat Shigapov
320d70d305 return last modified in UTC 2020-06-19 10:26:40 +03:00
Chris Lu
546709c1de check has visited by the boundary 2020-06-18 22:28:17 -07:00
Chris Lu
0e3b970c0c added bounded tree to track exploration boundary 2020-06-18 21:09:58 -07:00
Chris Lu
ae1994cbc1 erasure coding: fix cases where there are no .ecj files 2020-06-18 09:52:35 -07:00
Chris Lu
b74eced7a3 mount: a fix to prevent possible repeated calls
related to https://github.com/bazil/fuse/issues/130
2020-06-17 13:55:16 -07:00
Chris Lu
7ccc31853c hard limit volume file size 2020-06-16 00:27:48 -07:00
Chris Lu
48880edc2c volume: fix typo in minFreeSpacePercent option 2020-06-14 20:37:05 -07:00
Chris Lu
a0464fc267 fix help
fix https://github.com/chrislusf/seaweedfs/issues/1359
2020-06-12 22:06:21 -07:00
Chris Lu
0d1d83a5a5 mount: add option to auto create dir to mount 2020-06-12 11:46:58 -07:00
Chris Lu
00f96d86a1 fix typo 2020-06-11 10:53:25 -07:00
Chris Lu
63a1db64c5
Merge pull request #1358 from DXist/s3fixes
Return NotSuchKey error code for bucket S3 DeleteObject method
2020-06-11 10:37:19 -07:00
Rinat Shigapov
283b749ff1 keep 204 response 2020-06-11 19:00:34 +03:00
Rinat Shigapov
fafc41a27f return xml encoded NotFound status code for s3 delete 2020-06-11 17:53:15 +03:00
Rinat Shigapov
2d2c5dfa39 proxy status code of filer delete response 2020-06-11 15:00:47 +03:00
Chris Lu
628b27ef3b purge old cache implementation 2020-06-11 01:50:00 -07:00
Chris Lu
b9365de47b fix volume server pprof 2020-06-10 13:10:10 -07:00
Chris Lu
718052656a
Merge pull request #1355 from Kimbsen/add_http_pprof
Add --pprof to volume servers to enable http pprof handlers
2020-06-10 13:05:04 -07:00
Chris Lu
95a9686eaf 1.81 2020-06-10 13:00:42 -07:00
Chris Lu
c66f357715 report error correctly 2020-06-10 10:59:25 -07:00
Kimbsen
03e8f976d9 added --pprof to volume servers 2020-06-10 11:52:24 +02:00
Chris Lu
c38066be89 also add it for webdav 2020-06-09 18:07:03 -07:00
Chris Lu
4598473251 fix possible EOF problem 2020-06-09 18:04:40 -07:00
Chris Lu
bb3e86fbd5 auto create non-existing cache directory
fix https://github.com/chrislusf/seaweedfs/issues/1350
2020-06-08 08:49:57 -07:00
Chris Lu
bdbe5ddbe9 volume server: add a fix for dead lock
https://github.com/chrislusf/seaweedfs/issues/1353
2020-06-08 08:12:59 -07:00
Chris Lu
b8e346cc86 1.80 2020-06-05 15:29:25 -07:00
Chris Lu
45a9d852d3 go fmt 2020-06-05 15:27:10 -07:00
Chris Lu
03c2f03509
Merge pull request #1351 from ekozlov-search/master
Set volumes ReadOnly if low free disk space
2020-06-05 09:56:46 -07:00
Chris Lu
ebd1759741 fix ReadAt
fix https://github.com/chrislusf/seaweedfs/issues/1344
2020-06-05 09:54:14 -07:00
Evgenii Kozlov
2410ffbd79 Name fix 2020-06-05 19:00:38 +03:00
Evgenii Kozlov
0e0db70f55 Set volumes ReadOnly if low free disk space 2020-06-05 18:18:15 +03:00
Chris Lu
393ab07c7c disable pulse seconds
there are a few other hard coded timing settings, which will cause disconnected volume servers.
2020-06-04 10:52:01 -07:00
Evgenii Kozlov
48f9ff52cf Discs statistics on Windows platform. 2020-06-02 11:52:16 +03:00
Chris Lu
bc2ec6774d inject git version into build 2020-06-02 00:10:38 -07:00
bingoohuang
1f8782a1ed try showing the first 100 volume ids and an extra ... 2020-05-29 16:15:33 +08:00
bingoohuang
1a642b9876 add Volume Ids column only for max 100 volumes for convenience in the master ui. 2020-05-29 15:37:58 +08:00
bingoohuang
accb4964b7 util.BytesToHumanReadable 2020-05-29 10:00:07 +08:00
bingoohuang
21d0a013d5 rename 'needle' which collides with imported package name 2020-05-28 17:00:00 +08:00
bingoohuang
41cab5d69f log JSON response if httpStatus >= 400 2020-05-28 08:56:55 +08:00
Chris Lu
ef2b3a0801 leveldb recover error handling 2020-05-26 00:03:44 -07:00
Chris Lu
5d3ec22975 refactoring 2020-05-26 00:03:31 -07:00
Chris Lu
d4235afe4d support multiple locks 2020-05-25 18:39:53 -07:00
bingoohuang
ecdeef8c66 simplify func(w http.ResponseWriter, r *http.Request) to http.HandlerFunc 2020-05-25 21:00:12 +08:00
Chris Lu
e5a0787653 filer: leveldb store add auto recovery, report correct error
fix https://github.com/chrislusf/seaweedfs/issues/1326
2020-05-22 10:54:42 -07:00
Chris Lu
ed9e4dc749 1.79
fix compilation problem
2020-05-19 08:21:51 -07:00
Chris Lu
7ff2b0121d specify constant data type to uint64 instead of default int
fix https://github.com/chrislusf/seaweedfs/issues/1324
2020-05-19 07:55:23 -07:00
Chris Lu
11b33c0760 fix compilation problem 2020-05-18 10:53:31 -07:00
Chris Lu
ac48c899e8 add printouts for synchronizing metadata which can be slow for large folders 2020-05-18 10:31:12 -07:00
Chris Lu
0c856e8387 avoid possible wrong arguments
fix https://github.com/chrislusf/seaweedfs/issues/1320
2020-05-18 10:07:12 -07:00
Chris Lu
2bfd810912 combine msgBroker into weed server command 2020-05-17 22:44:20 -07:00
Chris Lu
85aeead3d2 1.78 2020-05-17 20:28:04 -07:00
Chris Lu
8005786d9a logs 2020-05-17 20:27:10 -07:00
Chris Lu
75179544c3 logging 2020-05-17 20:20:21 -07:00
Chris Lu
91b91d6cb7 add error to avoid copying not found volume
fix https://github.com/chrislusf/seaweedfs/issues/1317
2020-05-17 20:20:12 -07:00
Chris Lu
d3925d086c add delete channel 2020-05-17 17:42:42 -07:00
Chris Lu
e0e31e67a8
Merge pull request #1318 from chrislusf/msg_channel
Add messaging, add channel
2020-05-17 17:39:16 -07:00
Chris Lu
081ee6fe34 reduc logs 2020-05-17 17:38:31 -07:00
Chris Lu
bff09fb74f fix looping 2020-05-17 17:38:21 -07:00
Chris Lu
f5684839a2 add DeleteTopic 2020-05-17 17:34:10 -07:00
Chris Lu
3a57aef7a9 sync subscribe() 2020-05-17 17:33:53 -07:00
Chris Lu
95ca9dd8a2 subscribe support cancel 2020-05-17 11:10:45 -07:00
Chris Lu
f11233cd49 simplify disconnected stream detection 2020-05-17 08:57:47 -07:00
Chris Lu
1ea9bc66d9 avoid leaking grpc.NewClientStream 2020-05-17 08:56:33 -07:00
Chris Lu
85b53ac510 detect disconnected subscribers 2020-05-16 18:53:54 -07:00
Chris Lu
759cda0fe2 log 2020-05-16 18:47:41 -07:00
Chris Lu
b0de01ff3b able to detect disconnected subscribers 2020-05-16 08:57:29 -07:00
Chris Lu
e02a8c67da revert to one subscriber one thread 2020-05-15 21:38:42 -07:00
Chris Lu
b4e02ec525 fix prometheus problem
metrics.go:123] could not push metrics to prometheus push gateway 192.168.xx:9091: unexpected status code 200 while pushing to http://192.168.xx:9091/metrics/job/filer/instance/m111p112%3A8888:
2020-05-14 21:08:34 -07:00
Chris Lu
3fbf635308 Merge branch 'master' into msg_channel 2020-05-13 03:54:15 -07:00
Chris Lu
3a5e4769cf prevent empty locations returned
fix https://github.com/chrislusf/seaweedfs/issues/1313
2020-05-13 03:46:38 -07:00
Chris Lu
ca4017dd87 rename 2020-05-12 22:55:55 -07:00
Chris Lu
25257acd51 rename 2020-05-12 21:26:49 -07:00
Chris Lu
a7959c1c48 multiple subscriber with same subscriberId shares the topic manager
rename topicControl to topicCursor
2020-05-12 21:26:02 -07:00
Chris Lu
2f243f5b0b refactor 2020-05-12 08:48:00 -07:00
Chris Lu
4b7fa31468 ensure montonically increasing tsNs 2020-05-11 01:53:54 -07:00
Chris Lu
d5a8297a1c able to connect publisher 2020-05-10 23:45:48 -07:00
Chris Lu
d58371c5cd Merge branch 'master' into msg_channel 2020-05-10 23:44:59 -07:00
Chris Lu
015dd3a147 batch file id deletion 2020-05-10 23:25:39 -07:00
Chris Lu
39e72fb23c go fmt 2020-05-10 03:50:30 -07:00
Chris Lu
6bf3eb69cb async chan write read, no write for closed chan 2020-05-10 03:48:35 -07:00
Chris Lu
78afb8bf46 Merge branch 'master' into msg_channel 2020-05-09 00:43:58 -07:00
Chris Lu
d693e77418 add pub sub md5 2020-05-09 00:43:53 -07:00
Chris Lu
07d7abe428 add deleteTopic, refactoring 2020-05-09 00:31:34 -07:00