mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-02 08:36:57 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,768 Commits 32 Branches 296 Tags 149 MiB
47f5ec0cde
Commit Graph

219 Commits

Author SHA1 Message Date
Chris Lu
9449100757 adjust throwing exception 2020-10-10 01:06:54 -07:00
Chris Lu
d3d3609a03 adjust logging 2020-10-10 01:04:31 -07:00
Chris Lu
6fc272f913 adjust throwing exception 2020-10-10 01:03:04 -07:00
Chris Lu
00a75d7c99 Hadoop: fix reading file tail 2020-10-10 00:57:57 -07:00
Chris Lu
e1a3ffcdbf Hadoop: add exponential back off for failed reads 2020-10-10 00:57:57 -07:00
hilimd
411e49f964
Merge pull request #23 from chrislusf/master 
sync
 2020-10-10 13:07:57 +08:00
limd
ac162fc857 hdfs: Hadoop on SeaweedFS: create empty file 2020-10-10 13:07:27 +08:00
Chris Lu
a34bad2cee moving grafana dashboard here 2020-09-30 13:30:21 -07:00
Chris Lu
436ab7d811 HCFS: 1.4.8 2020-09-27 23:20:08 -07:00
Chris Lu
4929d0634e Hadoop on SeaweedFS: create empty file 
fix https://github.com/chrislusf/seaweedfs/issues/1494
 2020-09-25 10:57:43 -07:00
limd
95bfec4931 hadoop: filesystem cannot create file 
issues: https://github.com/chrislusf/seaweedfs/issues/1494
 2020-09-26 01:38:48 +08:00
Chris Lu
1012df7bb5 switch hardlink id from int64 to bytes 2020-09-24 11:11:42 -07:00
Chris Lu
5e239afdfc hardlink works now 2020-09-24 03:06:48 -07:00
Chris Lu
cb427d48fa filer report metrics configuration 2020-09-17 06:46:51 -07:00
Chris Lu
5eee4983f3 1.4.7 hdfs configurable fs.seaweed.buffer.size 2020-09-16 17:18:18 -07:00
Chris Lu
387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu
bba90ff3c8 read filer signature 2020-09-05 22:52:15 -07:00
Chris Lu
ca658a97c5 add signatures to messages to avoid double processing 2020-08-28 23:48:48 -07:00
Chris Lu
48ed7f8a32 generated files 2020-08-16 21:25:24 -07:00
Chris Lu
f5d6b8b777 Hadoop HCFS 1.4.6 random access file 2020-08-16 21:17:03 -07:00
Chris Lu
13bfe5deef same logic for reading random access files from Go 2020-08-16 21:14:39 -07:00
Chris Lu
20e0bae5d1 add for testing 2020-08-16 09:03:18 -07:00
Chris Lu
24bfd26719 make it easy to test randome access file 2020-08-15 14:37:07 -07:00
Chris Lu
4f195a54ca 1.4.5 
change the scope for hadoop depencies to provided
 2020-08-09 16:34:47 -07:00
Chris Lu
15dc0a704d Revert "add read ahead input stream" 
This reverts commit b3089dcc8e.
 2020-08-03 09:11:24 -07:00
Chris Lu
b3089dcc8e add read ahead input stream 2020-08-03 09:06:09 -07:00
Chris Lu
53190a9972 fix compilation 2020-08-03 09:05:49 -07:00
Chris Lu
d6073f6386 ensure GC 2020-08-03 00:40:23 -07:00
Chris Lu
9981748498 only try to cache chunk manifest chunks 2020-08-02 23:50:44 -07:00
Chris Lu
0ed1f43d29 decompress after decrypt if necessary 
skip any decompress error
 2020-08-02 23:50:23 -07:00
Chris Lu
e1f070a9a1 Hadoop: 1.4.4 2020-08-01 12:42:41 -07:00
Chris Lu
4d9da157bc HDFS: read gzip content 2020-08-01 11:36:29 -07:00
Chris Lu
8cc35e2c13 fix wrong decompression logic 2020-08-01 11:19:57 -07:00
Chris Lu
6b41c5250b Hadoop file system: 1.4.3 
added buffered fs input stream
 2020-07-29 21:33:10 -07:00
Chris Lu
703057bff9 mirror changes from hdfs2 2020-07-29 20:15:43 -07:00
Chris Lu
a6d0f962e8 add buffered seaweed input stream 2020-07-29 20:10:04 -07:00
Chris Lu
5080bc1d69 debug 2020-07-29 18:26:26 -07:00
Chris Lu
daf8c0c8ce cache vid locations 2020-07-29 18:25:52 -07:00
Chris Lu
b684c312d2 minor 2020-07-29 18:25:35 -07:00
Chris Lu
735912e0f5 enable read cache 2020-07-29 17:39:44 -07:00
Chris Lu
6f058b3791 HCFS: 1.4.1 2020-07-22 22:56:53 -07:00
Chris Lu
648ef566c4 HCFS: avoid lock bottleneck 2020-07-22 22:54:06 -07:00
Chris Lu
6839f96c0c simplify 2020-07-22 22:52:49 -07:00
Chris Lu
54f9f9138e HCFS: 1.4.0 2020-07-20 23:25:08 -07:00
Chris Lu
64926d2345 disable ChunkCache 2020-07-20 21:18:58 -07:00
Chris Lu
ae3e6d8244 remove changing buffer size 2020-07-20 20:27:19 -07:00
Chris Lu
b380b9e5d7 shared http client 2020-07-20 18:27:11 -07:00
Chris Lu
0e341a2a9a error logging 2020-07-20 18:26:59 -07:00
Chris Lu
cacc601cc8 ensure changing buffer size requirements 2020-07-20 18:26:48 -07:00
Chris Lu
aee6d89350 before writing and after reading file chunks 2020-07-20 18:26:17 -07:00
Chris Lu
6debe3c3ec use 1000 as merge factor 2020-07-20 18:25:26 -07:00
Chris Lu
1d724ab237 hdfs: support read write chunk manifest 2020-07-20 03:29:17 -07:00
Chris Lu
f90d2c93c9 1.3.9 remove logs 2020-07-17 16:11:57 -07:00
Chris Lu
3abd74b1d7 1.3.8 2020-07-17 15:46:20 -07:00
Chris Lu
0b9d73fb7c async copy 2020-07-17 15:46:20 -07:00
Chris Lu
54445b207d Hadoop: 1.3.6 2020-07-16 16:20:01 -07:00
Chris Lu
d6d95765f9 reduce chunk cache size 2020-07-16 16:19:01 -07:00
Chris Lu
2629da2cb9 simplify inputstream 2020-07-16 10:54:29 -07:00
Chris Lu
c9b50e8a22 copy changes to hadoop2 2020-07-16 09:05:34 -07:00
李明达
74456b3d5e 1、Add SeaweedFS implementation of Hadoop AbstractFileSystem. The implementation delegates to the existing SeaweedFS FileSystem and is only necessary for use with Hadoop 2.x/3.x. Configuration example in Hadoop core-site.xml file: 
<property>
    <name>fs.AbstractFileSystem.seaweedfs.impl</name>
    <value>seaweed.hdfs.SeaweedAbstractFileSystem</value>
 </property>

2、Fix hiveserver2 startup NullPointerException
 2020-07-16 16:47:09 +08:00
Chris Lu
64d7d5fe89 increase inbound message size 2020-07-16 00:33:54 -07:00
Chris Lu
7bca72deed reuse bytebuffer 2020-07-15 23:33:31 -07:00
Chris Lu
2286d27730 rename 2020-07-15 14:19:15 -07:00
Chris Lu
86c8137546 1.3.4 2020-07-15 13:49:06 -07:00
Chris Lu
22f8aae028 fix compilation 2020-07-15 13:27:10 -07:00
Chris Lu
bc3be0bb37 Hadoop: 1.3.3 
improve memory efficiency
 2020-07-15 13:25:44 -07:00
Chris Lu
316e853e0e option to disable chunk cache 2020-07-15 09:51:51 -07:00
Chris Lu
0db4204c81 add is_file_chunks to filer.proto 2020-07-14 22:42:52 -07:00
Chris Lu
fc9af962ec 1.3.2 2020-07-12 14:51:24 -07:00
Chris Lu
4bd8f3281e 1.3.0 
* add option to include last entry
*
 2020-07-11 22:24:02 -07:00
Chris Lu
578f316173 refactoring 2020-07-11 15:33:31 -07:00
Chris Lu
70d8a3a1d3 add SubscribeLocalMetadata without checking persisted meta logs 2020-07-05 15:50:07 -07:00
Chris Lu
37d5b3ba12 replication: pass isFromOtherCluster also to EventNotification 
EventNotification is consistent with message queue and metadata logs.
 2020-07-01 08:06:20 -07:00
Chris Lu
31e23e9783 filer: support active<=>active filer replication 2020-06-30 22:53:57 -07:00
Chris Lu
97239ce6f1 rename filechunk is_gzipped to is_compressed 2020-06-20 08:15:49 -07:00
Chris Lu
ca3516ac6d adjust protoc 2020-06-20 08:00:25 -07:00
Chris Lu
ce79ec0bd4 HCFS: 1.2.9 fix concurrent modification 2020-06-18 07:52:32 -07:00
Chris Lu
222f93e816 possibly fix concurrent access to entry object in SeaweedOutputStream 2020-06-17 14:59:43 -07:00
Chris Lu
8dfaaeabfd HCFS: 1.2.8 fix hbase related bugs 
1. SeaweedFileSystem.listStatus need to work with file also
2. SeaweedRead readChunkView has wrong len
 2020-05-24 17:07:34 -07:00
Chris Lu
dfccc3c263 able to read chan and write chan 2020-05-08 02:47:22 -07:00
Chris Lu
1e3e4b3072 add broker connects to filer 2020-05-05 02:05:28 -07:00
Chris Lu
972f437225 HCFS 1.2.7 2020-04-26 05:44:56 -07:00
Chris Lu
b52b8ec685 Hadoop: fix entry not found for HCFS 
also fix cipher related changes.
 2020-04-26 05:21:54 -07:00
Chris Lu
e24b25de78 async meta caching: can stream updates now 2020-04-21 21:16:13 -07:00
Chris Lu
bda82f61bc filer: able to append to a file 2020-04-17 02:28:09 -07:00
Chris Lu
7764e0465c refactoring 2020-04-12 21:00:55 -07:00
Chris Lu
6f948e4887 remove configurable topics folder location 2020-04-12 13:07:59 -07:00
Chris Lu
ec2eb8bc48 add If-None-Match and If-Modified-Since 
fix https://github.com/chrislusf/seaweedfs/issues/1269
 2020-04-08 08:12:00 -07:00
Chris Lu
2d43f85577 watch entries with common path prefix 2020-04-05 12:51:21 -07:00
Chris Lu
bf270d9e8c filer: able to tail meta data changes 2020-04-05 00:51:16 -07:00
Chris Lu
50a5018b7f writing meta logs is working 2020-03-30 01:19:33 -07:00
Chris Lu
5ebf232e79 HDFS: 1.2.6 2020-03-29 01:40:04 -07:00
Chris Lu
b656e05aaf HDFS: support chunk cache 2020-03-29 00:55:40 -07:00
Chris Lu
ae2309dc58 Update SeaweedRead.java 2020-03-28 23:58:59 -07:00
Chris Lu
3d3bab2447 add notes 2020-03-28 23:43:24 -07:00
Chris Lu
c4bea45099 S3 API: fix DeleteMultipleObjectsHandler 
fix https://github.com/chrislusf/seaweedfs/issues/1241
 2020-03-20 14:17:31 -07:00
Chris Lu
de1ba85346 HDFS support encrypted data storage 2020-03-14 00:27:57 -07:00
Chris Lu
2e3f6ad3a9 filer: remember content is gzipped or not 2020-03-08 21:39:33 -07:00
Chris Lu
13e215ee5c filer: option to encrypt data on volume server 2020-03-06 00:49:47 -08:00
Chris Lu
555413d9fc weed queue starts 2020-02-27 00:07:13 -08:00