mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-02 16:39:16 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,987 Commits 32 Branches 296 Tags 149 MiB
1a3f859c58
Commit Graph

252 Commits

Author SHA1 Message Date
Chris Lu
bdd166c031 adjust readme 2021-01-03 00:47:58 -08:00
Chris Lu
f4d87f8b9d works 
performance seems about the same as leveldb for insert entries
 2021-01-02 23:32:58 -08:00
Chris Lu
2c3c2c27d7 separate prefix from namePattern 
fix https://github.com/chrislusf/seaweedfs/issues/1722
 2021-01-01 20:23:23 -08:00
Chris Lu
da7e5aaa65 filer: optimize for less number of directory lookup 
bottom up directory lookup
 2020-12-27 20:46:17 -08:00
Chris Lu
da87f6b265 remove unused code 2020-12-26 15:21:12 -08:00
Chris Lu
0a7c5f85a9 filer: add namePattern to search in current folder 2020-12-26 15:05:31 -08:00
Chris Lu
0e016bc7bd hbase add ttl 2020-12-24 12:10:35 -08:00
Chris Lu
9bf6c10505 fix prefix 2020-12-24 00:18:59 -08:00
Chris Lu
2fea8cfa0f fix compilation 2020-12-23 23:54:15 -08:00
Chris Lu
53bc1ea25b fix compilation 2020-12-23 23:53:46 -08:00
Chris Lu
b5e2be635a adjust for directory listing 2020-12-23 23:49:22 -08:00
Chris Lu
c4a202ec41 fix wrong column family 2020-12-23 23:23:05 -08:00
Chris Lu
94e3757c08 fix 2020-12-23 21:45:16 -08:00
Chris Lu
64c48c9724 seems compiling 2020-12-23 21:44:53 -08:00
Chris Lu
a0990b929d minor 2020-12-22 17:33:40 -08:00
Chris Lu
97e3432dfe avoid wrong error 
fix https://github.com/chrislusf/seaweedfs/issues/1691
 2020-12-22 02:46:24 -08:00
Chris Lu
4f31c1bb94 go fmt 2020-12-22 02:34:08 -08:00
Chris Lu
6c4f32d173 remove unused code 2020-12-22 02:33:31 -08:00
Chris Lu
90d785a15f filer: redis, redis cluster, cassandra support super large directory 2020-12-22 02:26:05 -08:00
Chris Lu
fe46411cd4 handle directory listing edge cases 2020-12-21 23:37:43 -08:00
Chris Lu
c74bede730 handle listing directories 2020-12-21 23:19:05 -08:00
Chris Lu
adf8cb4000 filer: path-specific stores trim out common prefixes 2020-12-21 22:57:13 -08:00
Chris Lu
0823bde8d8 refactor: split into two files 2020-12-21 22:05:15 -08:00
Chris Lu
8ab70ccf76 refactor 2020-12-21 21:46:32 -08:00
Chris Lu
65cc6dc636 refactor to dedicated function 2020-12-21 21:18:34 -08:00
Chris Lu
488c2680e8 change default filer store root 2020-12-21 21:15:22 -08:00
Chris Lu
f5a19a87be filer: report error if the path specific store is not found 
related https://github.com/chrislusf/seaweedfs/issues/1691
 2020-12-21 12:43:52 -08:00
Chris Lu
41c0f3ad24 filer: support path-specific filer store 2020-12-19 01:27:09 -08:00
Chris Lu
0d5683fb0e todo: load path-specific store from filer.toml 2020-12-18 03:05:42 -08:00
Chris Lu
3269fd7eaf filer: use store by path 2020-12-18 02:57:49 -08:00
Chris Lu
e605f1e001 refactor, add getActualStore() by path 2020-12-18 02:55:00 -08:00
Chris Lu
23903aa95b refactoring 2020-12-18 02:35:45 -08:00
Chris Lu
4e6b316913 assert FilerStoreWrapper is VirtualFilerStore 2020-12-18 02:17:06 -08:00
Chris Lu
38fc89d041 filer store: sql update if any insert error happens 
fix https://github.com/chrislusf/seaweedfs/issues/1673
 2020-12-13 20:49:44 -08:00
Chris Lu
23280257df sql put kv: avoid unnecessary update 2020-12-13 19:45:47 -08:00
Chris Lu
14910d035c the pagination size was too big for recursive deletion 2020-12-12 13:26:10 -08:00
Chris Lu
03637d6f57 s3: move "delete-directory-if-empty" to read time 
move "delete-directory-if-empty" to read time instead of entry deletion time

the listing speed for a s3 bucket folder will slow down if it has many sub folders

related to 0d345ac97d

fix https://github.com/chrislusf/seaweedfs/issues/1647

fix https://github.com/chrislusf/seaweedfs/issues/1670
 2020-12-12 03:38:34 -08:00
Chris Lu
37075a414d adjust logs 2020-12-12 03:33:57 -08:00
Chris Lu
3fedfec1e7 check cross device rename error 2020-12-10 23:50:32 -08:00
Chris Lu
c2f18a10cb minor 2020-12-10 22:23:22 -08:00
Chris Lu
4b0c2a846b add logs 2020-12-10 19:55:28 -08:00
Chris Lu
83078ac6ce filer: change to /etc/seaweedfs folder on filer 
fix https://github.com/chrislusf/seaweedfs/issues/1666
 2020-12-10 11:11:02 -08:00
Chris Lu
765b3ef1be save /etc/iam/identity.json inside filer store 2020-12-10 00:15:22 -08:00
Chris Lu
a9c619c270 added error handling for saving 2020-12-09 01:34:24 -08:00
Chris Lu
8e78187a97 add back last read chunk cache to reader and properly close the reader 2020-12-08 22:26:46 -08:00
Chris Lu
900d22c6ec mount: avoid memory leaking read buffer 
fix https://github.com/chrislusf/seaweedfs/issues/1654

the reader goes together with the file handle, which may stay for a long time.
 2020-12-08 02:38:53 -08:00
Chris Lu
fce8803087 break import cycle 2020-12-07 00:29:17 -08:00
Chris Lu
01e2da5782 refactoring 2020-12-06 23:16:20 -08:00
Chris Lu
9acda432fe fix import cycle 2020-12-06 20:12:52 -08:00
Chris Lu
5931a2f53f filer conf: support json conf only 2020-12-06 19:47:06 -08:00
Chris Lu
f4abd01adf filer: cache small file to filer store 2020-11-30 04:34:04 -08:00
Chris Lu
a9c6be5fc3 set creation time 2020-11-30 03:11:52 -08:00
Chris Lu
0d345ac97d s3: remove empty parent folder on delete 
fix https://github.com/chrislusf/seaweedfs/issues/1637
 2020-11-26 12:21:58 -08:00
Chris Lu
cc2bd97ad9 refactor 2020-11-26 11:25:56 -08:00
Chris Lu
0da7ecfd29 go fmt 2020-11-26 11:22:30 -08:00
Chris Lu
effa00ed08 refactor 2020-11-26 11:21:58 -08:00
Chris Lu
513bcd6e0d filer: avoid duplicated FindEntry for deletion 2020-11-26 11:14:56 -08:00
Chris Lu
eab53ea80d filer leveldb store: a bit more efficient directory listing with prefix 2020-11-22 21:10:41 -08:00
Chris Lu
290b5e2cd0 directly delete file chunks 
keeping current async deletions for now
 2020-11-17 17:20:21 -08:00
Chris Lu
dc304342b2 fs.configure: configurable volume growth 2020-11-17 01:00:02 -08:00
Chris Lu
5f19e81dab filer confi: support hierachical configuration 2020-11-16 16:50:12 -08:00
Chris Lu
c0d279c54e filere.conf: prefer to use json format 2020-11-15 21:48:17 -08:00
Chris Lu
ee2fa14dbe filer conf: delete location specific configuration 2020-11-15 20:15:47 -08:00
Chris Lu
71056dae07 fs.configure: read and local add filer configuration 2020-11-15 18:09:35 -08:00
Chris Lu
0ea5c087ce go fmt 2020-11-15 16:59:28 -08:00
Chris Lu
95c0de285d refactoring 2020-11-15 16:58:48 -08:00
Chris Lu
500bcab953 refactoring 2020-11-15 14:41:56 -08:00
Chris Lu
590f02179d filer: load filer conf when starting 2020-11-15 14:06:03 -08:00
Chris Lu
0a406f652e load filer conf and match by prefix 2020-11-15 00:26:05 -08:00
Chris Lu
0fc9ffc603 able to read filer.conf if updated 2020-11-14 21:21:20 -08:00
Chris Lu
442e092995 filer watch "/etc" folder for configuration changes 2020-11-14 14:26:08 -08:00
Konstantin Lebedev
aa30604cec fix tests 2020-11-12 08:49:38 +05:00
Konstantin Lebedev
1eec5c8d5d gen pb 2020-11-12 04:10:06 +05:00
Konstantin Lebedev
fc7baef5bb fiil serverUrls sorted by data center 2020-11-12 02:13:33 +05:00
Chris Lu
8750cac090 move to util.RetryWaitTime 2020-11-01 02:36:43 -08:00
Chris Lu
df8d976bb0 refactoring 2020-11-01 01:58:48 -07:00
Chris Lu
16fae84414 fix format error 2020-10-24 20:11:31 -07:00
Chris Lu
e9d40b80b2 less verbose logs 2020-10-24 09:42:54 -07:00
Chris Lu
b81359823f postgres: support empty user 2020-10-22 14:27:47 -07:00
Chris Lu
d75bc62196 to read files that are quickly changing 2020-10-22 12:59:15 -07:00
Chris Lu
5179e559f7 skip empty logs 2020-10-22 00:35:48 -07:00
Chris Lu
bd103c143a add lock for vidCache 2020-10-21 19:28:59 -07:00
Chris Lu
68d39c86f1 mysql, postgres, cassandra: change kv key to base64 encoding 
The exisitng key-value operation for stores using mysql, postgres, and maybe cassandra are already broken.

The kv is used to store hardlink, filer store signature and replication progress.

So users using hardlink and also uses mysql, postgres, or cassandra will have broken hard links.

Users using filer.sync will need to re-sync the files.
 2020-10-16 11:10:12 -07:00
Chris Lu
c95d3d9dff adjust tests 2020-10-14 12:27:52 -07:00
Chris Lu
93bcf56514 file read report EOF 
fix https://github.com/chrislusf/seaweedfs/issues/1344
 2020-10-14 12:18:24 -07:00
Chris Lu
0542911e29 go fmt 2020-10-13 19:50:46 -07:00
Chris Lu
58fa506491 minor 2020-10-13 19:50:22 -07:00
Chris Lu
aac4cb1f0c adds errror on read and write 2020-10-13 13:53:34 -07:00
Chris Lu
3f7d1d1bf1 Only wait on retryable requests 2020-10-13 00:29:46 -07:00
Chris Lu
723ae11db4 refactoring in order to adjust volume server url later 2020-10-11 20:15:10 -07:00
Chris Lu
d155f907c2 mount: configurable read wait time 2020-10-10 20:09:43 -07:00
Chris Lu
8a52379ecb add retry if volume can not be found 2020-10-10 16:02:39 -07:00
Chris Lu
70af0ec24c filer: fix hanging on read 2020-10-10 16:02:10 -07:00
Chris Lu
cff8bb6554 return proper error 2020-10-10 15:43:22 -07:00
Chris Lu
6da8eef54f fix logic error 2020-10-09 00:01:47 -07:00
Chris Lu
5d01dd28a6 add logging 2020-10-08 23:38:00 -07:00
Chris Lu
8d34eb0050 mount:exponentially backoff if read error for about 10 minutes 2020-10-08 23:31:26 -07:00
Chris Lu
6e1f936efd refactoring 2020-10-08 23:19:42 -07:00
Chris Lu
b2ee5873fb fix error not being returned 2020-10-08 23:19:20 -07:00
Chris Lu
8676db0f6a adjust logging 2020-10-08 18:33:37 -07:00