Chris Lu
|
821c46edf1
|
Merge branch 'master' into support_ssd_volume
|
2021-02-09 11:37:07 -08:00 |
|
Chris Lu
|
5b1def9080
|
Java: 1.6.1 refacoring API
|
2021-02-08 02:42:01 -08:00 |
|
Chris Lu
|
ad36c7b0d7
|
refactoring: only expose FilerClient class
|
2021-02-08 02:28:45 -08:00 |
|
Chris Lu
|
5138d3954f
|
Java 1.6.0 fix filerProxy mode
|
2021-02-06 00:09:12 -08:00 |
|
Chris Lu
|
35ba277a97
|
Java: fix filerProxy mode
|
2021-02-05 22:43:56 -08:00 |
|
Chris Lu
|
9c1efdf11b
|
HCFS: 1.5.9
|
2021-02-05 11:47:56 -08:00 |
|
Chris Lu
|
8f3a51f2b8
|
Java: 1.5.8 additional fixes
|
2021-02-05 10:42:20 -08:00 |
|
Chris Lu
|
6a2a9b67e8
|
Java: 1.5.8
|
2021-02-04 22:22:15 -08:00 |
|
Chris Lu
|
694df89331
|
java: add configurable volume access mode
|
2021-02-04 22:21:55 -08:00 |
|
Chris Lu
|
8c3177d835
|
java: resolve parent directory if started with seaweedfs://
|
2021-02-04 21:41:19 -08:00 |
|
Chris Lu
|
42e5ef4b01
|
Java: adjust examples
|
2021-02-04 21:02:54 -08:00 |
|
Chris Lu
|
9fa7977714
|
Java: SeaweedOutputStream refactoring
|
2021-02-04 20:30:49 -08:00 |
|
Chris Lu
|
ce416d765f
|
Java: SeaweedOutputStream add replication option
|
2021-02-04 20:22:40 -08:00 |
|
Chris Lu
|
82c167aaca
|
Java: supportFlush set to true
|
2021-02-04 20:18:33 -08:00 |
|
Chris Lu
|
7f90d14f10
|
Java: add SeaweedOutputStream example
|
2021-02-04 20:16:08 -08:00 |
|
Chris Lu
|
502554887f
|
Java: add SeaweedInputStream example
|
2021-02-04 18:44:57 -08:00 |
|
Chris Lu
|
87d1bfa862
|
Hadoop Compatible FS: 1.5.7
|
2021-01-30 06:18:41 -08:00 |
|
Chris Lu
|
6f4aab51f9
|
refactoring SeaweedInputStream
|
2021-01-30 06:16:02 -08:00 |
|
Chris Lu
|
043c2d7960
|
refactoring SeaweedOutputStream
|
2021-01-30 05:39:09 -08:00 |
|
Chris Lu
|
94525aa0fd
|
allocate volume by disk type
|
2020-12-13 23:08:21 -08:00 |
|
Chris Lu
|
0d2ec832e2
|
rename from volumeType to diskType
|
2020-12-13 11:59:32 -08:00 |
|
Chris Lu
|
e9cd798bd3
|
adding volume type
|
2020-12-13 00:58:58 -08:00 |
|
Chris Lu
|
4d2855476c
|
Hadoop: add BufferedByteBufferReadableInputStream
fix https://github.com/chrislusf/seaweedfs/issues/1645
|
2020-12-03 00:08:05 -08:00 |
|
Chris Lu
|
3857f9c840
|
Hadoop: switch to ByteBuffer
fix https://github.com/chrislusf/seaweedfs/issues/1645
|
2020-12-02 23:45:39 -08:00 |
|
Chris Lu
|
a9efaa6385
|
HDFS: implement ByteBufferReadable
fix https://github.com/chrislusf/seaweedfs/issues/1645
|
2020-12-01 17:20:31 -08:00 |
|
Chris Lu
|
f4abd01adf
|
filer: cache small file to filer store
|
2020-11-30 04:34:04 -08:00 |
|
Chris Lu
|
da04bb3d1b
|
1.5.6
|
2020-11-18 01:46:38 -08:00 |
|
limd
|
4737df597d
|
HCFS:
1. add replication parameter
2. fix close sequence
|
2020-11-17 19:05:57 +08:00 |
|
hilimd
|
aec071187e
|
Merge pull request #38 from chrislusf/master
sync
|
2020-11-17 19:01:04 +08:00 |
|
Chris Lu
|
dc304342b2
|
fs.configure: configurable volume growth
|
2020-11-17 01:00:02 -08:00 |
|
Chris Lu
|
5f19e81dab
|
filer confi: support hierachical configuration
|
2020-11-16 16:50:12 -08:00 |
|
hilimd
|
218561c206
|
Merge pull request #37 from chrislusf/master
sync
|
2020-11-16 16:30:19 +08:00 |
|
Chris Lu
|
fe01059675
|
volume id cache: reduce cache time
|
2020-11-15 21:27:21 -08:00 |
|
Chris Lu
|
61d96fde01
|
protect against edge cases when locations expires
|
2020-11-15 21:26:04 -08:00 |
|
Chris Lu
|
95c0de285d
|
refactoring
|
2020-11-15 16:58:48 -08:00 |
|
Chris Lu
|
824e96ffcc
|
updates
|
2020-11-13 12:47:46 -08:00 |
|
Chris Lu
|
3e362451d2
|
add example of watch files
|
2020-11-13 12:10:55 -08:00 |
|
Chris Lu
|
0d5355c614
|
rename
|
2020-11-13 11:43:25 -08:00 |
|
Chris Lu
|
7d24980840
|
HCFS: 1.5.4
|
2020-11-13 11:37:53 -08:00 |
|
hilimd
|
e0d5207ed9
|
Merge pull request #36 from chrislusf/master
sync
|
2020-11-13 15:34:38 +08:00 |
|
Chris Lu
|
a2962604ad
|
add volume id cache
|
2020-11-12 23:25:11 -08:00 |
|
Chris Lu
|
559dfacdac
|
adding test example
|
2020-11-12 23:24:39 -08:00 |
|
Chris Lu
|
a4dfa00030
|
adjust filer conf
|
2020-11-12 00:39:59 -08:00 |
|
Chris Lu
|
c32b6f4d9d
|
prepare for path based configuration
|
2020-11-11 23:46:45 -08:00 |
|
hilimd
|
a38efe2e7b
|
Merge pull request #35 from chrislusf/master
sync
|
2020-11-11 10:16:39 +08:00 |
|
Chris Lu
|
4f676aa7d3
|
2.09
|
2020-11-08 22:06:06 -08:00 |
|
hilimd
|
546f1bcb90
|
Merge pull request #34 from chrislusf/master
sync
|
2020-11-05 12:02:47 +08:00 |
|
Chris Lu
|
c709059b69
|
HCFS: add close() to SeaweedFileSystem.java
|
2020-11-03 00:46:00 -08:00 |
|
hilimd
|
843865f2ca
|
Merge pull request #33 from chrislusf/master
sync
|
2020-10-26 22:01:50 +08:00 |
|
Chris Lu
|
f375b93aef
|
renaming
|
2020-10-25 15:32:43 -07:00 |
|
Chris Lu
|
596d476e3d
|
HCFS: 1.5.2
|
2020-10-25 14:44:18 -07:00 |
|
hilimd
|
ab1105c524
|
Merge pull request #31 from chrislusf/master
sync
|
2020-10-21 20:04:11 +08:00 |
|
Chris Lu
|
6c92fa349d
|
filer: support default rack
fix https://github.com/chrislusf/seaweedfs/issues/1546
|
2020-10-20 17:41:39 -07:00 |
|
hilimd
|
6e63e5eaf0
|
Merge pull request #29 from chrislusf/master
sync
|
2020-10-19 09:42:43 +08:00 |
|
Chris Lu
|
b7c224cf61
|
HCFS: 1.5.1
|
2020-10-18 14:30:01 -07:00 |
|
Chris Lu
|
459de70a77
|
Hadoop: more accurate block size
|
2020-10-18 14:13:46 -07:00 |
|
hilimd
|
62af2d961d
|
Merge pull request #28 from chrislusf/master
sync
|
2020-10-18 09:44:04 +08:00 |
|
Chris Lu
|
ace0ea3d28
|
s3: avoid duplicated bucket
|
2020-10-15 10:52:20 -07:00 |
|
hilimd
|
a911375798
|
Merge pull request #26 from chrislusf/master
sync
|
2020-10-14 16:39:45 +08:00 |
|
Chris Lu
|
cad7fd8775
|
Merge pull request #1524 from chrislusf/dependabot/maven/other/java/s3copier/junit-junit-4.13.1
Bump junit from 3.8.1 to 4.13.1 in /other/java/s3copier
|
2020-10-12 11:46:07 -07:00 |
|
Chris Lu
|
4e31ca9507
|
upgrade junit to 4.13.1
|
2020-10-12 11:37:15 -07:00 |
|
dependabot[bot]
|
b57d435abe
|
Bump junit from 3.8.1 to 4.13.1 in /other/java/s3copier
Bumps [junit](https://github.com/junit-team/junit4) from 3.8.1 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.13.1.md)
- [Commits](https://github.com/junit-team/junit4/commits/r4.13.1)
Signed-off-by: dependabot[bot] <support@github.com>
|
2020-10-12 18:14:12 +00:00 |
|
Chris Lu
|
253323ce6d
|
Hadoop 1.5.0
|
2020-10-12 11:13:38 -07:00 |
|
hilimd
|
2ff727a32d
|
Merge pull request #25 from chrislusf/master
sync
|
2020-10-12 11:01:13 +08:00 |
|
Chris Lu
|
d5fcb0f474
|
Hadoop: printout the entry creation error message
|
2020-10-11 15:11:16 -07:00 |
|
Chris Lu
|
ea13e0523b
|
fix logging
|
2020-10-11 14:59:48 -07:00 |
|
Chris Lu
|
8e2f3b510e
|
Hadoop client: 1.4.9
|
2020-10-10 15:43:04 -07:00 |
|
hilimd
|
b9a446839a
|
Merge pull request #24 from chrislusf/master
sync
|
2020-10-10 16:20:26 +08:00 |
|
Chris Lu
|
912ef2bc53
|
Hadoop: remove unused variable bufferSize
|
2020-10-10 01:14:17 -07:00 |
|
Chris Lu
|
9449100757
|
adjust throwing exception
|
2020-10-10 01:06:54 -07:00 |
|
Chris Lu
|
d3d3609a03
|
adjust logging
|
2020-10-10 01:04:31 -07:00 |
|
Chris Lu
|
6fc272f913
|
adjust throwing exception
|
2020-10-10 01:03:04 -07:00 |
|
Chris Lu
|
00a75d7c99
|
Hadoop: fix reading file tail
|
2020-10-10 00:57:57 -07:00 |
|
Chris Lu
|
e1a3ffcdbf
|
Hadoop: add exponential back off for failed reads
|
2020-10-10 00:57:57 -07:00 |
|
hilimd
|
411e49f964
|
Merge pull request #23 from chrislusf/master
sync
|
2020-10-10 13:07:57 +08:00 |
|
limd
|
ac162fc857
|
hdfs: Hadoop on SeaweedFS: create empty file
|
2020-10-10 13:07:27 +08:00 |
|
Chris Lu
|
436ab7d811
|
HCFS: 1.4.8
|
2020-09-27 23:20:08 -07:00 |
|
Chris Lu
|
4929d0634e
|
Hadoop on SeaweedFS: create empty file
fix https://github.com/chrislusf/seaweedfs/issues/1494
|
2020-09-25 10:57:43 -07:00 |
|
limd
|
95bfec4931
|
hadoop: filesystem cannot create file
issues: https://github.com/chrislusf/seaweedfs/issues/1494
|
2020-09-26 01:38:48 +08:00 |
|
Chris Lu
|
1012df7bb5
|
switch hardlink id from int64 to bytes
|
2020-09-24 11:11:42 -07:00 |
|
Chris Lu
|
5e239afdfc
|
hardlink works now
|
2020-09-24 03:06:48 -07:00 |
|
Chris Lu
|
cb427d48fa
|
filer report metrics configuration
|
2020-09-17 06:46:51 -07:00 |
|
Chris Lu
|
5eee4983f3
|
1.4.7 hdfs configurable fs.seaweed.buffer.size
|
2020-09-16 17:18:18 -07:00 |
|
Chris Lu
|
387ab6796f
|
filer: cross cluster synchronization
|
2020-09-09 11:21:23 -07:00 |
|
Chris Lu
|
bba90ff3c8
|
read filer signature
|
2020-09-05 22:52:15 -07:00 |
|
Chris Lu
|
ca658a97c5
|
add signatures to messages to avoid double processing
|
2020-08-28 23:48:48 -07:00 |
|
Chris Lu
|
48ed7f8a32
|
generated files
|
2020-08-16 21:25:24 -07:00 |
|
Chris Lu
|
f5d6b8b777
|
Hadoop HCFS 1.4.6 random access file
|
2020-08-16 21:17:03 -07:00 |
|
Chris Lu
|
13bfe5deef
|
same logic for reading random access files from Go
|
2020-08-16 21:14:39 -07:00 |
|
Chris Lu
|
20e0bae5d1
|
add for testing
|
2020-08-16 09:03:18 -07:00 |
|
Chris Lu
|
24bfd26719
|
make it easy to test randome access file
|
2020-08-15 14:37:07 -07:00 |
|
Chris Lu
|
4f195a54ca
|
1.4.5
change the scope for hadoop depencies to provided
|
2020-08-09 16:34:47 -07:00 |
|
Chris Lu
|
15dc0a704d
|
Revert "add read ahead input stream"
This reverts commit b3089dcc8e .
|
2020-08-03 09:11:24 -07:00 |
|
Chris Lu
|
b3089dcc8e
|
add read ahead input stream
|
2020-08-03 09:06:09 -07:00 |
|
Chris Lu
|
53190a9972
|
fix compilation
|
2020-08-03 09:05:49 -07:00 |
|
Chris Lu
|
d6073f6386
|
ensure GC
|
2020-08-03 00:40:23 -07:00 |
|
Chris Lu
|
9981748498
|
only try to cache chunk manifest chunks
|
2020-08-02 23:50:44 -07:00 |
|
Chris Lu
|
0ed1f43d29
|
decompress after decrypt if necessary
skip any decompress error
|
2020-08-02 23:50:23 -07:00 |
|
Chris Lu
|
e1f070a9a1
|
Hadoop: 1.4.4
|
2020-08-01 12:42:41 -07:00 |
|
Chris Lu
|
4d9da157bc
|
HDFS: read gzip content
|
2020-08-01 11:36:29 -07:00 |
|
Chris Lu
|
8cc35e2c13
|
fix wrong decompression logic
|
2020-08-01 11:19:57 -07:00 |
|
Chris Lu
|
6b41c5250b
|
Hadoop file system: 1.4.3
added buffered fs input stream
|
2020-07-29 21:33:10 -07:00 |
|
Chris Lu
|
703057bff9
|
mirror changes from hdfs2
|
2020-07-29 20:15:43 -07:00 |
|
Chris Lu
|
a6d0f962e8
|
add buffered seaweed input stream
|
2020-07-29 20:10:04 -07:00 |
|
Chris Lu
|
5080bc1d69
|
debug
|
2020-07-29 18:26:26 -07:00 |
|
Chris Lu
|
daf8c0c8ce
|
cache vid locations
|
2020-07-29 18:25:52 -07:00 |
|
Chris Lu
|
b684c312d2
|
minor
|
2020-07-29 18:25:35 -07:00 |
|
Chris Lu
|
735912e0f5
|
enable read cache
|
2020-07-29 17:39:44 -07:00 |
|
Chris Lu
|
6f058b3791
|
HCFS: 1.4.1
|
2020-07-22 22:56:53 -07:00 |
|
Chris Lu
|
648ef566c4
|
HCFS: avoid lock bottleneck
|
2020-07-22 22:54:06 -07:00 |
|
Chris Lu
|
6839f96c0c
|
simplify
|
2020-07-22 22:52:49 -07:00 |
|
Chris Lu
|
54f9f9138e
|
HCFS: 1.4.0
|
2020-07-20 23:25:08 -07:00 |
|
Chris Lu
|
64926d2345
|
disable ChunkCache
|
2020-07-20 21:18:58 -07:00 |
|
Chris Lu
|
ae3e6d8244
|
remove changing buffer size
|
2020-07-20 20:27:19 -07:00 |
|
Chris Lu
|
b380b9e5d7
|
shared http client
|
2020-07-20 18:27:11 -07:00 |
|
Chris Lu
|
0e341a2a9a
|
error logging
|
2020-07-20 18:26:59 -07:00 |
|
Chris Lu
|
cacc601cc8
|
ensure changing buffer size requirements
|
2020-07-20 18:26:48 -07:00 |
|
Chris Lu
|
aee6d89350
|
before writing and after reading file chunks
|
2020-07-20 18:26:17 -07:00 |
|
Chris Lu
|
6debe3c3ec
|
use 1000 as merge factor
|
2020-07-20 18:25:26 -07:00 |
|
Chris Lu
|
1d724ab237
|
hdfs: support read write chunk manifest
|
2020-07-20 03:29:17 -07:00 |
|
Chris Lu
|
f90d2c93c9
|
1.3.9 remove logs
|
2020-07-17 16:11:57 -07:00 |
|
Chris Lu
|
3abd74b1d7
|
1.3.8
|
2020-07-17 15:46:20 -07:00 |
|
Chris Lu
|
0b9d73fb7c
|
async copy
|
2020-07-17 15:46:20 -07:00 |
|
Chris Lu
|
54445b207d
|
Hadoop: 1.3.6
|
2020-07-16 16:20:01 -07:00 |
|
Chris Lu
|
d6d95765f9
|
reduce chunk cache size
|
2020-07-16 16:19:01 -07:00 |
|
Chris Lu
|
2629da2cb9
|
simplify inputstream
|
2020-07-16 10:54:29 -07:00 |
|
Chris Lu
|
c9b50e8a22
|
copy changes to hadoop2
|
2020-07-16 09:05:34 -07:00 |
|
李明达
|
74456b3d5e
|
1、Add SeaweedFS implementation of Hadoop AbstractFileSystem. The implementation delegates to the existing SeaweedFS FileSystem and is only necessary for use with Hadoop 2.x/3.x. Configuration example in Hadoop core-site.xml file:
<property>
<name>fs.AbstractFileSystem.seaweedfs.impl</name>
<value>seaweed.hdfs.SeaweedAbstractFileSystem</value>
</property>
2、Fix hiveserver2 startup NullPointerException
|
2020-07-16 16:47:09 +08:00 |
|
Chris Lu
|
64d7d5fe89
|
increase inbound message size
|
2020-07-16 00:33:54 -07:00 |
|
Chris Lu
|
7bca72deed
|
reuse bytebuffer
|
2020-07-15 23:33:31 -07:00 |
|
Chris Lu
|
2286d27730
|
rename
|
2020-07-15 14:19:15 -07:00 |
|
Chris Lu
|
86c8137546
|
1.3.4
|
2020-07-15 13:49:06 -07:00 |
|
Chris Lu
|
22f8aae028
|
fix compilation
|
2020-07-15 13:27:10 -07:00 |
|
Chris Lu
|
bc3be0bb37
|
Hadoop: 1.3.3
improve memory efficiency
|
2020-07-15 13:25:44 -07:00 |
|
Chris Lu
|
316e853e0e
|
option to disable chunk cache
|
2020-07-15 09:51:51 -07:00 |
|
Chris Lu
|
0db4204c81
|
add is_file_chunks to filer.proto
|
2020-07-14 22:42:52 -07:00 |
|
Chris Lu
|
fc9af962ec
|
1.3.2
|
2020-07-12 14:51:24 -07:00 |
|
Chris Lu
|
4bd8f3281e
|
1.3.0
* add option to include last entry
*
|
2020-07-11 22:24:02 -07:00 |
|
Chris Lu
|
578f316173
|
refactoring
|
2020-07-11 15:33:31 -07:00 |
|
Chris Lu
|
70d8a3a1d3
|
add SubscribeLocalMetadata without checking persisted meta logs
|
2020-07-05 15:50:07 -07:00 |
|
Chris Lu
|
37d5b3ba12
|
replication: pass isFromOtherCluster also to EventNotification
EventNotification is consistent with message queue and metadata logs.
|
2020-07-01 08:06:20 -07:00 |
|
Chris Lu
|
31e23e9783
|
filer: support active<=>active filer replication
|
2020-06-30 22:53:57 -07:00 |
|
Chris Lu
|
97239ce6f1
|
rename filechunk is_gzipped to is_compressed
|
2020-06-20 08:15:49 -07:00 |
|
Chris Lu
|
ca3516ac6d
|
adjust protoc
|
2020-06-20 08:00:25 -07:00 |
|
Chris Lu
|
ce79ec0bd4
|
HCFS: 1.2.9 fix concurrent modification
|
2020-06-18 07:52:32 -07:00 |
|
Chris Lu
|
222f93e816
|
possibly fix concurrent access to entry object in SeaweedOutputStream
|
2020-06-17 14:59:43 -07:00 |
|
Chris Lu
|
8dfaaeabfd
|
HCFS: 1.2.8 fix hbase related bugs
1. SeaweedFileSystem.listStatus need to work with file also
2. SeaweedRead readChunkView has wrong len
|
2020-05-24 17:07:34 -07:00 |
|
Chris Lu
|
dfccc3c263
|
able to read chan and write chan
|
2020-05-08 02:47:22 -07:00 |
|
Chris Lu
|
1e3e4b3072
|
add broker connects to filer
|
2020-05-05 02:05:28 -07:00 |
|
Chris Lu
|
972f437225
|
HCFS 1.2.7
|
2020-04-26 05:44:56 -07:00 |
|