Chris Lu
f9d8bd51ad
ec shard balancing
2019-06-10 21:32:56 -07:00
Chris Lu
9d9162ca35
ec.balance: collect dc rack info
2019-06-07 00:25:05 -07:00
Chris Lu
e66bddd84c
customizable sleep intervals
2019-06-06 00:39:08 -07:00
Chris Lu
50aa769554
jwt for read access control
2019-06-06 00:29:02 -07:00
Chris Lu
d344e0a035
fix ec related bugs
2019-06-05 23:20:26 -07:00
Chris Lu
450f4733ce
report empty volume or ec shards
2019-06-05 13:32:33 -07:00
Chris Lu
7f6fb9ffd0
every 17 minutes instead of seconds
...
TODO: make this configurable
2019-06-05 13:32:08 -07:00
Chris Lu
6b08db65b0
fix shard count reporting
2019-06-05 01:58:37 -07:00
Chris Lu
784141c5e6
adjust script parameter
2019-06-05 01:48:03 -07:00
Chris Lu
ede876cfdb
periodic scripts exeuction from leader master
2019-06-05 01:30:24 -07:00
Chris Lu
b9e138713c
ec.encode: add -fullPercent option
2019-06-05 00:13:13 -07:00
Chris Lu
0e49add2bc
adjust help message
2019-06-05 00:11:37 -07:00
Chris Lu
eaa76f11b7
free volume slots factor in ec shard counts
2019-06-04 23:41:56 -07:00
Chris Lu
ca8a2bb534
go fmt
2019-06-04 22:04:10 -07:00
Chris Lu
2215e81be7
ui add ec shard statuses
2019-06-04 21:52:37 -07:00
Chris Lu
0e52862586
avoid the test that depends on ordering in a different folder
2019-06-04 02:22:32 -07:00
Chris Lu
2f7710a75d
debug sporadic test error on travis
2019-06-04 02:09:06 -07:00
Chris Lu
06133ae98f
correct count for free ec slots, avoid removing generated shards
2019-06-04 01:49:51 -07:00
Chris Lu
6cea23d091
purge duplicated ec shards
2019-06-04 01:32:36 -07:00
Chris Lu
b05456fe07
able to purge extra ec shard copies
2019-06-03 20:25:02 -07:00
Chris Lu
11cffb3168
fix ec.rebuild bugs
2019-06-03 11:50:54 -07:00
Chris Lu
d85b41b904
fix ec.encode not finding the local ec shards
2019-06-03 10:38:21 -07:00
Chris Lu
7e80b2b882
fix multiple bugs
2019-06-03 02:26:31 -07:00
Chris Lu
55be09996d
fix volume balance bug
2019-06-03 00:13:31 -07:00
Chris Lu
9ce6b949bf
adjust help message
2019-06-01 10:37:08 -07:00
Chris Lu
d6e2c877fa
ec.balance: use specific collection names
2019-06-01 02:00:18 -07:00
Chris Lu
133b772fb5
destroy ec volume if it is empty
2019-06-01 01:51:28 -07:00
Chris Lu
ba18314aab
ec shard delete also check ec volumes, in addition to volumes
2019-06-01 01:41:22 -07:00
Chris Lu
f919d0235c
ec encode volumes quiet for a period of time
2019-05-31 23:41:17 -07:00
Chris Lu
a72cef3c42
encode by collection
2019-05-31 15:48:40 -07:00
Chris Lu
de1a35acdd
conditionally fresh the shard locations
2019-05-31 02:22:12 -07:00
Chris Lu
47f1901843
ask for the ec volume version
2019-05-31 00:58:51 -07:00
Chris Lu
689930f092
forget shards that are broken
2019-05-31 00:19:13 -07:00
Chris Lu
aaab2c25f5
resolve concurrent modification problem
...
n and err can be modified concurrently
2019-05-30 13:59:07 -07:00
Chris Lu
f53024d79d
delete ec collection
2019-05-30 10:40:22 -07:00
Chris Lu
40ca2f2903
add collection.delete
2019-05-30 09:47:54 -07:00
Chris Lu
1d111d6ce8
collection list normal and ec volumes
2019-05-30 09:27:23 -07:00
Chris Lu
e941d0c2f4
list ec collections also
2019-05-30 09:17:58 -07:00
Chris Lu
866197eee3
print out the ec balancing plan
2019-05-30 01:38:59 -07:00
Chris Lu
5c6c757619
clean up test files
2019-05-29 00:34:30 -07:00
Chris Lu
5dd67f9acf
reading by recover from other shards
2019-05-28 23:48:39 -07:00
Chris Lu
3f9ecee40f
working with reading remote intervals
2019-05-28 21:29:07 -07:00
Chris Lu
302d9fbc6d
adjust error message for file not found
2019-05-28 14:27:21 -07:00
Chris Lu
4f76342cbc
WIP
...
no errors, but not returning file content
* the interval needs to use actual file zie
* need to read the actual version instead of the current version
2019-05-28 00:51:01 -07:00
Chris Lu
2858a954b3
read ec volume shards locations from master
2019-05-28 00:13:13 -07:00
Chris Lu
713596e781
caching ec shard locations
2019-05-27 22:54:58 -07:00
Chris Lu
3a8c1055a2
refactoring ecx to ecVolume
2019-05-27 22:00:36 -07:00
Chris Lu
217cde0a3b
refactoring
2019-05-27 21:40:51 -07:00
Chris Lu
03b9291e5d
volum server passes grpc option and master location to store
2019-05-27 21:22:23 -07:00
Chris Lu
ac1ba3b667
remove comments
2019-05-27 21:21:52 -07:00
Chris Lu
b4b407e403
add grpc ec shard read
2019-05-27 11:59:03 -07:00
Chris Lu
a4f3d82c57
convert needle id to ec intervals to read from
2019-05-27 01:29:46 -07:00
Chris Lu
a463759edf
adjust error message
2019-05-26 01:14:42 -07:00
Chris Lu
92f9c7b1aa
adjust ec shard status on disconnect
2019-05-26 01:05:08 -07:00
Chris Lu
ac14cd5ad0
wdclient need to have ec shard info on initial dump
2019-05-26 01:01:04 -07:00
Chris Lu
b58e25e588
broadcast vid->location map to master clients for ec chards
2019-05-26 00:49:15 -07:00
Chris Lu
af67d99ca4
incrementally update master ec shards state
2019-05-26 00:21:17 -07:00
Chris Lu
db94a41f9e
mount/unmount ec shards
2019-05-25 23:23:19 -07:00
Chris Lu
41e8ae61f8
generate, balance, delete copied shards, delete old volume
2019-05-25 14:02:06 -07:00
Chris Lu
f0e6574d5e
allocate ec shards to volume servers
2019-05-25 02:02:44 -07:00
Chris Lu
6f4b09b6a4
pb shard info uses ShardBits instead one message for one shard
2019-05-24 13:28:44 -07:00
Chris Lu
228850d588
shard id starts from zero
2019-05-24 11:52:23 -07:00
Chris Lu
24b8ad9a14
push the ec shard info to master
2019-05-23 23:47:49 -07:00
Chris Lu
8838176d82
register ec shards to topology
2019-05-23 23:34:29 -07:00
Chris Lu
8a96445f40
register ec shards to each data node
2019-05-23 22:51:18 -07:00
Chris Lu
4659d80035
prepare to register ec shard info in master
2019-05-23 00:42:28 -07:00
Chris Lu
e913ee380a
add comments
2019-05-23 00:04:24 -07:00
Chris Lu
7180520889
ec shard info can be queried via VolumeList()
2019-05-22 22:44:28 -07:00
Chris Lu
17ac1290c0
volume: load ec shards during heartbeats to master
2019-05-21 22:41:20 -07:00
Chris Lu
54b835e1ae
filer: redis cluster add option for password
...
fix https://github.com/chrislusf/seaweedfs/issues/971
2019-05-20 09:00:30 -07:00
Chris Lu
fbbc74abb4
adds VolumeEcGenerateSlices, VolumeEcCopy
2019-05-20 00:53:17 -07:00
Chris Lu
ae499fd5aa
Merge branch 'master' into erasure_coding
2019-05-19 21:38:18 -07:00
Chris Lu
698dea779c
refactoring
2019-05-19 21:37:49 -07:00
Chris Lu
693d6c0f1b
clean up tests
2019-05-19 21:16:01 -07:00
Chris Lu
0094ca3f8a
mark the webdav as unstable
2019-05-19 21:08:26 -07:00
Chris Lu
6386a3174b
able to validate by randomly selected ec files
2019-05-19 14:24:33 -07:00
Chris Lu
7c2c60c376
add locating data inside the ec files
2019-05-19 03:01:58 -07:00
Chris Lu
87f63b9c08
generate ec01~ec14, generate ecx file with sorted needle values
2019-05-18 22:46:24 -07:00
Chris Lu
12dc6608f0
Merge branch 'master' into erasure_coding
2019-05-18 11:16:07 -07:00
Chris Lu
f2c4c888f6
1.34
2019-05-17 22:53:30 -07:00
Chris Lu
8123a488cd
filer: optimize leveldb parameters
2019-05-17 18:14:25 -07:00
Chris Lu
76cf57a0e1
volume: optimzation for leveldb
2019-05-17 17:34:03 -07:00
Chris Lu
0fe286a6ca
filer: optimization for leveldb, add leveldb2
...
large filer db may see high CPU and disk usage due to background compaction
2019-05-17 17:33:49 -07:00
Chris Lu
7e0c3bb732
filer: adjust recommended mysql meta data type to LONGBLOB
...
fix https://github.com/chrislusf/seaweedfs/issues/966
2019-05-17 09:11:08 -07:00
Chris Lu
bf9d490538
fix for tests
2019-05-17 02:28:20 -07:00
Chris Lu
82b0759493
filer: migrating filer store from persisting shorter structured file id instead of a string
2019-05-17 02:03:23 -07:00
Chris Lu
8156958ee9
move function to make travis happy
2019-05-15 10:02:44 -07:00
Chris Lu
0a36f628c6
testing RS coding
2019-05-15 01:02:00 -07:00
Chris Lu
ab22407763
filer ip only use bind ip
2019-05-14 23:43:40 -07:00
Chris Lu
017d0957c3
avoid switching master node if same ip and host name
...
fix https://github.com/chrislusf/seaweedfs/issues/955
2019-05-11 18:42:46 -07:00
Chris Lu
4dfcd2169e
weedfuse: adapt to fstab command line pattern
2019-05-11 01:17:55 -07:00
Chris Lu
9a4fb14ea0
weedfuse: daemonize
2019-05-10 17:08:15 -07:00
Chris Lu
1ca1ec906a
add weedfuse
2019-05-10 15:03:31 -07:00
Chris Lu
69b7dd398d
adjust help message
2019-05-07 14:02:01 -07:00
Chris Lu
d01807ba05
shell: volume.balance add option for all_collections, each_collection
2019-05-06 14:12:19 -07:00
Chris Lu
8ffdd5dcdb
writel throttling moving volumes
2019-05-06 13:58:42 -07:00
Chris Lu
cf58fc0e63
refactor: extract out the write throttler
2019-05-06 13:56:08 -07:00
Chris Lu
4e42e7b5e7
shell: volume.balance add dataCenter filter
2019-05-06 13:30:14 -07:00
Chris Lu
4c392ca762
use ip.bind to listen for grpc calls
...
help to avoid https://github.com/chrislusf/seaweedfs/issues/952
2019-05-06 08:10:18 -07:00
Chris Lu
8656102127
shell: volume.balance runs for each collection, and for ALL at the end
2019-05-05 22:28:14 -07:00
Chris Lu
f08bbb72de
refactoring
2019-05-05 22:21:28 -07:00
Chris Lu
0df6346611
add error checking
2019-05-05 21:58:46 -07:00
Chris Lu
3ff92037cc
shell: add volume.balance command
2019-05-05 21:17:23 -07:00
Chris Lu
3f2a30bd05
shell: volume.list output sorted voume list
2019-05-05 20:23:50 -07:00
Chris Lu
46a675ecc2
align memory to avoid nil with atomic operations
2019-05-04 21:33:05 -07:00
Chris Lu
177f3add81
adjust memory layout to avoid possible nil pointer
2019-05-04 08:47:11 -07:00
Chris Lu
25941e0500
master: add jwt expires_after_seconds
2019-05-04 08:42:25 -07:00
Chris Lu
bd8af92b54
remove unused code
2019-05-04 08:20:22 -07:00
Chris Lu
4b27f892e6
remove 128 bit needle id build option
...
fix https://github.com/chrislusf/seaweedfs/issues/949
2019-05-03 22:39:00 -07:00
Chris Lu
3fbe22b203
better error handling for nil raftServer
...
addressing https://github.com/chrislusf/seaweedfs/issues/717
2019-05-03 22:33:08 -07:00
Chris Lu
5681208f2d
refactor: remove logger
2019-05-03 22:21:06 -07:00
Chris Lu
8c982db25d
1.33
2019-05-03 17:27:50 -07:00
Chris Lu
b335f81a4f
volume: add option to limit compaction speed
2019-05-03 17:22:39 -07:00
Chris Lu
f0f981e7c8
webdav: fix directory renaming
2019-05-03 14:12:51 -07:00
Chris Lu
bfbecd7253
webdav: fix directory creation
2019-05-03 13:13:11 -07:00
Chris Lu
9d3c945b00
Merge pull request #950 from chrislusf/webdav
...
add WebDav
2019-05-03 01:25:10 -07:00
Chris Lu
7064b2ea48
webdav: can write now
2019-05-03 00:55:52 -07:00
Chris Lu
b30c14b631
webdav: can read now
2019-05-03 00:24:35 -07:00
Chris Lu
47f14775d7
scaffolding, but seems hard to satisfy mac, windows, and different os versions
...
https://www.reddit.com/r/golang/comments/3wrxo3/webdav_server_that_provide_filesystem_using/
2019-05-02 14:22:10 -07:00
Wine93
32f93fb09e
style: go fmt for all
2019-04-30 03:22:19 +00:00
Chris Lu
82e06a780b
mount: allow nonempty fuse mount to work with Samba
...
possibly fix https://github.com/chrislusf/seaweedfs/issues/936
2019-04-29 10:42:09 -07:00
Chris Lu
108d0fb08d
adjust log level
2019-04-26 09:32:37 -07:00
Chris Lu
575926cdec
minor
2019-04-26 09:32:37 -07:00
Chris Lu
630cf1e33e
fix
2019-04-24 12:31:52 -07:00
Chris Lu
ea4592161c
fix
2019-04-24 12:31:14 -07:00
Chris Lu
c5a931b22c
logging
2019-04-24 12:30:36 -07:00
Chris Lu
9beea63960
memory alignment
...
fix https://github.com/chrislusf/seaweedfs/issues/939
2019-04-24 09:23:19 -07:00
Chris Lu
9774ede3cf
adjust usage description
2019-04-24 00:25:20 -07:00
Chris Lu
b04d7e3ac0
weed server: also optionally start S3 gateway
2019-04-24 00:18:01 -07:00
Chris Lu
dabc9c9a89
filer: listen on specified ip address
2019-04-24 00:01:39 -07:00
Chris Lu
21fe5bde7c
filer: remove customizable grpc port, fixed to filer port + 10000
2019-04-23 23:58:52 -07:00
Chris Lu
067b935763
report unknown commands
2019-04-23 22:29:36 -07:00
Chris Lu
53e2e73de2
fix metrics
...
fix https://github.com/chrislusf/seaweedfs/issues/935
2019-04-22 12:32:10 -07:00
Chris Lu
0c96cfdb2e
minor
2019-04-21 15:50:51 -07:00
Chris Lu
00b6f653fa
shell: add fs.cat
2019-04-21 15:43:43 -07:00
Chris Lu
59d532a8c3
1.31
2019-04-21 14:09:50 -07:00
Chris Lu
5c411f3e5f
minor
2019-04-21 13:33:32 -07:00
Chris Lu
440111a349
volume: support http status 304 for the same file id
2019-04-21 13:33:23 -07:00
Chris Lu
ebe94be1a1
maintain layout correctness when changing volumes
2019-04-21 13:32:36 -07:00
Chris Lu
0302b9496c
volume: also check cookie for possible duplicates
2019-04-21 13:31:45 -07:00
Chris Lu
132921ad41
adjust log or error
2019-04-21 10:14:17 -07:00
Chris Lu
316bd27f75
add remove volumes with version info
2019-04-20 23:53:37 -07:00
Chris Lu
64a9a0e104
shell: fix volume.copy
2019-04-20 20:48:07 -07:00
Chris Lu
5f3d0e33a1
shell: add commands volume.copy volume.delete volume.mount volume.unmount
2019-04-20 20:31:35 -07:00
Chris Lu
d06d3c57d4
add notes
2019-04-20 20:12:07 -07:00
Chris Lu
2ffe98443b
refactoring
2019-04-20 12:05:28 -07:00
Chris Lu
6fc1f53018
shell: add command volume.move
2019-04-20 11:35:23 -07:00
stlpmo-jn
2200ea9cb9
add function ParseFileIdFromString
2019-04-20 18:39:06 +08:00
Chris Lu
3b3651dea3
volume: atomic copying file, adds version and stopOffset
2019-04-19 12:29:49 -07:00
Chris Lu
730a032137
filer mysql: adjust scaffolding instructions
2019-04-19 09:55:46 -07:00
Chris Lu
0be2d51c96
read volume lastAppendAtNs when loading a volume
2019-04-19 01:56:38 -07:00
Chris Lu
ac2727853f
fix needle map entry size
2019-04-19 00:39:34 -07:00
Chris Lu
e5506152c0
refactoring
2019-04-18 21:43:36 -07:00
Chris Lu
33c92b819a
refactoring
2019-04-18 21:17:43 -07:00
Chris Lu
a2d34d4802
filer.copy: add adjustable compression level
2019-04-18 20:21:28 -07:00
Chris Lu
fa176fe80f
volume tailing chunks large files
2019-04-18 19:22:13 -07:00
Chris Lu
3dce1016cb
add volume tailer
2019-04-18 11:05:02 -07:00
Chris Lu
b142f9f1d5
go fmt
2019-04-18 00:19:18 -07:00
Chris Lu
b09e8dbb37
add VolumeStreamFollow, but not used yet
2019-04-18 00:18:29 -07:00
Chris Lu
13ad5c1966
refactoring
2019-04-17 22:04:49 -07:00
Chris Lu
3b9446606a
adjust log level
2019-04-16 23:31:44 -07:00
Chris Lu
07091c44cd
lock btree writes
...
fix https://github.com/chrislusf/seaweedfs/issues/930
2019-04-16 19:48:07 -07:00
Chris Lu
4393b99332
add notes
2019-04-16 13:57:45 -07:00
Chris Lu
338e6d60a5
refactor: prepare for snapshotting
2019-04-16 09:55:37 -07:00
Chris Lu
af49aea0c6
weed shell: fs.meta.save adjusts meta data file name
2019-04-16 09:40:27 -07:00
Chris Lu
8ea1ee6dfa
weed shell: add fs.meta.notify, removing filer.export
2019-04-16 01:58:28 -07:00
Chris Lu
014906ec79
weed shell: add command fs.meta.load to restore meta data
2019-04-16 01:37:11 -07:00
Chris Lu
79c2cca9c1
better error message
2019-04-16 01:15:30 -07:00
Chris Lu
d35023c713
weed shell: add command fs.meta.save
2019-04-16 01:06:32 -07:00
Chris Lu
b3b42bc947
replicate need to include new entry path
2019-04-16 00:44:31 -07:00
Chris Lu
967e108b9a
increase max idle conn per hosts
2019-04-15 22:00:50 -07:00
Chris Lu
f567777d64
volume: fix needle checking for deleted needles
2019-04-15 21:58:43 -07:00
Chris Lu
bd76ad0ff1
Revert "Revert "volume: close request body if any parsing error""
...
This reverts commit 359b7c1650
.
2019-04-15 21:43:28 -07:00
Chris Lu
359b7c1650
Revert "volume: close request body if any parsing error"
...
This reverts commit 5996656141
.
2019-04-15 21:35:12 -07:00
Chris Lu
5996656141
volume: close request body if any parsing error
2019-04-15 21:11:45 -07:00
Chris Lu
6e116b3d67
volume: reset leader if error happens
2019-04-15 09:09:46 -07:00
Chris Lu
e85048bcdc
http exhaust and close response body
2019-04-14 23:28:24 -07:00
Chris Lu
3e8a3a8fec
fix race detector found problems
2019-04-14 23:00:37 -07:00
Chris Lu
2e03f3adef
filer: rename -port.public to -port.readonly
2019-04-11 20:42:55 -07:00
Chris Lu
59a9e15733
1.30
2019-04-11 19:40:36 -07:00
Chris Lu
e11e127d91
close grpc connections during errors
2019-04-11 09:27:08 -07:00
Wine93
361912224d
typo: remove blank
2019-04-11 09:18:53 +00:00
Chris Lu
0e33272f56
small refactoring
2019-04-10 23:39:53 -07:00
Chris Lu
cb2061f86a
Merge pull request #925 from stlpmo-jn/checkUnhealthReplication
...
the implement of repair the error replications of the volume
2019-04-10 23:37:12 -07:00
Chris Lu
c0d6cc0067
weed mount: mount to the same folder name
2019-04-10 23:25:41 -07:00
stlpmo-jn
c3fa50d3b3
remove the health chekcer, because it's the same as command_volume_fix_replication
2019-04-11 13:40:31 +08:00
Chris Lu
2f76681d62
refactor: adjust grpc API name
2019-04-10 21:41:17 -07:00
stlpmo-jn
f2031884f0
fix bug : CI build failed
2019-04-11 09:53:31 +08:00
Chris Lu
4df10e50b3
update glide.yaml
2019-04-10 14:27:28 -07:00
stlpmo-jn
c1a0403da2
repair the error replications of the volume
2019-04-10 19:41:55 +08:00
Chris Lu
3555628ad5
weed volume: fix bug, remove boltdb, btree
2019-04-09 10:08:59 -07:00
Chris Lu
8a5ce16e96
weed volume: remove boltdb, btree options, add options for leveldb medium, large
2019-04-09 09:42:06 -07:00
Chris Lu
9924fa3b1a
weed volume: disable block cache on leveldb
...
saving 8MB for each volume, but at the cost of slower lookup
2019-04-09 00:24:32 -07:00
Chris Lu
35aba35c07
Revert "weed volume: disable block cache on leveldb"
...
This reverts commit a55e708d36
.
2019-04-09 00:22:40 -07:00
Chris Lu
a55e708d36
weed volume: disable block cache on leveldb
...
saving 8MB for each volume, but at the cost of slower (offset,size) lookup
2019-04-09 00:18:32 -07:00
Chris Lu
6d4c44a85c
1.29
2019-04-08 22:41:59 -07:00
Chris Lu
88ba08cb9e
add large disk support
2019-04-08 22:32:42 -07:00
Chris Lu
784c5bb73a
add build option support 5-byte offset
2019-04-08 22:01:29 -07:00
Chris Lu
7a4b234ea1
divide offset into higher and lower sections
2019-04-08 21:44:06 -07:00
Chris Lu
000ee725fc
refactor Offset into a struct of bytes
2019-04-08 19:40:56 -07:00
Chris Lu
a41ba79119
Merge pull request #914 from jba/gocdk-receive
...
replication: add GoCDK PubSub support
2019-04-08 10:31:02 -07:00
Chris Lu
d14b614407
weed filer.copy: use existing file owner and gropu id
2019-04-07 11:31:50 -07:00
Chris Lu
a32abda1a3
adjust compression to optimize for speed
2019-04-07 11:01:28 -07:00
Chris Lu
c731d9be6d
weed filer.copy: add option to profile the execution
2019-04-07 09:13:24 -07:00
Chris Lu
174bf1e8b2
more efficient client side gzip compression
2019-04-06 14:14:28 -07:00
Chris Lu
868913aa04
printout statistics
2019-04-06 11:12:35 -07:00
Chris Lu
de7626bd22
weed shell: volume.list add summary statistics
2019-04-06 09:25:29 -07:00
Chris Lu
94302935b2
weed volume: only store compressed data if compression is effective
2019-04-06 00:10:52 -07:00
Chris Lu
b2d92a2992
weed shell: fs.tree fix directory tree listing
2019-04-05 23:42:36 -07:00
Chris Lu
24c020104a
weed filer.copy: parallelize the copying to increase throughput
...
improvement https://github.com/chrislusf/seaweedfs/issues/915
2019-04-05 23:35:30 -07:00
Chris Lu
c789b496d8
use cached grpc client
2019-04-05 20:31:58 -07:00
Chris Lu
5808caa2f5
use cached grpc client
2019-04-05 20:26:52 -07:00
Chris Lu
cd6d35aa52
weed shell: fs.tree improvements
2019-04-05 11:40:54 -07:00
Jonathan Amsterdam
72920efc20
added entry to scaffold
2019-04-05 13:43:38 -04:00
Jonathan Amsterdam
a6a5d80401
fix typo
2019-04-05 08:13:32 -04:00
Chris Lu
300b39b5af
weed filer.copy: use cached filer grpc connection
...
another attemp for https://github.com/chrislusf/seaweedfs/issues/915
2019-04-05 01:09:06 -07:00
Chris Lu
921231268b
weed filer.copy: skip files that can not be opened
...
such as socket
2019-04-05 00:04:00 -07:00
Chris Lu
4f714cef22
default weed filer.copy and weed upload chunk size to 32MB
2019-04-04 23:14:32 -07:00
Chris Lu
c94823f52c
set default http idle connection per host
...
possible fix https://github.com/chrislusf/seaweedfs/issues/915
2019-04-04 20:28:40 -07:00
Chris Lu
0bd7ced7c2
fix test
2019-04-04 20:11:59 -07:00
Chris Lu
9cc73f4a9a
fix test
2019-04-04 19:34:15 -07:00
Chris Lu
af37b374cb
weed shell: fs.cd change current directory
2019-04-04 19:27:51 -07:00
Chris Lu
766396d249
weed master: atomic volume counting
...
possible fix for https://github.com/chrislusf/seaweedfs/issues/913
2019-04-04 19:27:00 -07:00
Jonathan Amsterdam
c0c9a8bad5
replication: add GoCDK PubSub support
2019-04-04 17:22:45 -04:00
Chris Lu
715a38da1e
weed shell: add fs.cd, fs.pwd to change to a directory and print current directory
2019-04-03 00:20:00 -07:00
Chris Lu
20dcb44077
fix tests
2019-04-01 23:59:31 -07:00
Chris Lu
beaa2bd71a
weed filer: properly delete cached directory entry
...
fix https://github.com/chrislusf/seaweedfs/issues/910
2019-04-01 17:03:04 -07:00
Chris Lu
78b9db34d5
weed s3: generate time format acceptable by aws cli s3
...
fix https://github.com/chrislusf/seaweedfs/issues/911
2019-04-01 16:13:29 -07:00
Chris Lu
2a52e70a41
weed filer: increase default mysql/postgres dir or name length to 65535
2019-04-01 12:37:54 -07:00
Chris Lu
b4c276263f
weed filer: mysql/postgres use ReadCommitted transaction isolation level for atomic rename
2019-04-01 11:03:04 -07:00
Chris Lu
c5a3ff1c35
weed s3: multipart upload goes to the right bucket
...
fix https://github.com/chrislusf/seaweedfs/issues/908
2019-03-31 11:10:19 -07:00
Chris Lu
189c890715
weed replicate: replicate atomic rename to other systems
2019-03-31 08:10:47 -07:00
Chris Lu
8c823abe1f
1.28
2019-03-30 23:22:41 -07:00
Chris Lu
78ac2bef3c
go fmt
2019-03-30 23:09:16 -07:00
Chris Lu
97406333a5
support atomic renaming for mysql/postgres filer store
2019-03-30 23:08:29 -07:00
Chris Lu
920b4e56aa
refactoring
2019-03-29 22:53:35 -07:00
Chris Lu
f60d383947
weed filer.copy: support multiple masters
2019-03-29 22:50:14 -07:00
Chris Lu
487ca51491
weed filer.export: fix pagination error
2019-03-28 00:46:15 -07:00
Chris Lu
07775998a9
some comments
2019-03-28 00:38:30 -07:00
Chris Lu
53b81fcfcd
weed shell: add fs.tree
2019-03-28 00:05:04 -07:00
Chris Lu
c7e7b6229f
detect mime type
2019-03-27 14:25:18 -07:00
Chris Lu
65757ae2fd
weed filer: set content-type consistent with filer store
2019-03-27 11:41:11 -07:00
Chris Lu
37ce4c5269
weed shell: add fs.ls
2019-03-26 12:43:51 -07:00
Chris Lu
7a14cdc90c
refactoring, go fmt
2019-03-25 23:18:40 -07:00
Chris Lu
19728fe3f6
remove deprecated code
2019-03-25 23:12:14 -07:00
Chris Lu
df95ce0b6c
weed backup: efficient delta backup
...
fix https://github.com/chrislusf/seaweedfs/issues/399
2019-03-25 23:01:53 -07:00
Chris Lu
c913583e2d
Merge branch 'master' into add_volume_follow
2019-03-25 21:19:34 -07:00
Chris Lu
91b9a68493
udpate glide
2019-03-25 09:39:54 -07:00
Chris Lu
a32797518b
weed master: redirect to leader for default admin UI
...
fix https://github.com/chrislusf/seaweedfs/issues/898
2019-03-25 09:20:05 -07:00
Chris Lu
70815e9124
WIP
2019-03-25 09:16:12 -07:00
Chris Lu
eaa42c3865
1.27
2019-03-23 14:14:40 -07:00
Chris Lu
cd8a3b99bb
textual changes
2019-03-23 12:57:35 -07:00
Chris Lu
bd1c0735e0
weed shell: adjust help text format
2019-03-23 11:54:26 -07:00
Chris Lu
6b70b36105
weed shell: add "volume.fix.replication"
2019-03-23 11:34:09 -07:00
Chris Lu
95e0520182
weed volume: add grpc operation to relicate a volume to local
2019-03-23 11:33:34 -07:00
Chris Lu
a3490b600c
weed filer, weed master: add option to disable http
2019-03-21 16:00:46 -07:00
Chris Lu
531add52c2
weed master: skip proxied file read in http read only mode
2019-03-21 09:49:04 -07:00
Jonathan Amsterdam
e966033098
Merge branch 'master' of https://github.com/chrislusf/seaweedfs into gocdk
2019-03-20 16:02:38 -04:00
Chris Lu
977b30e992
weed filer: http HEAD response header add ETag
...
fix https://github.com/chrislusf/seaweedfs/issues/892
2019-03-20 10:13:53 -07:00
Jonathan Amsterdam
8db82e2b75
notification: add Go CDK pubsub support
...
Add the gocdk_pub_sub package, which supports the Go Cloud Development
Kit pubsub API.
Link in all current providers.
Update the notification scaffold.
2019-03-20 07:57:58 -04:00
chenwanli
45a52b17fd
benchmark: add replication flag
2019-03-20 16:38:11 +08:00
Chris Lu
88ab932f7d
refactoring function parameter
2019-03-19 23:01:23 -07:00
Chris Lu
5ae4b963a4
avoid using global rand
2019-03-19 22:20:14 -07:00
Chris Lu
f3d316a846
weed shell: adding file system disk usage fs.du
...
fix https://github.com/chrislusf/seaweedfs/issues/889
echo "fs.du http://localhost:8888/some/path " | weed shell
2019-03-19 21:58:00 -07:00
Chris Lu
cea5c53bf7
fix length checking
...
fix https://github.com/chrislusf/seaweedfs/issues/890
2019-03-19 20:56:27 -07:00
Chris Lu
da871896c3
weed filer: set grpc port to port + 10000
2019-03-19 05:47:41 -07:00
Chris Lu
916b809c08
add error checking
2019-03-19 05:34:43 -07:00
Chris Lu
eb4a54d9fe
refactoring
2019-03-19 05:19:37 -07:00
Chris Lu
ece9d13312
volume info collect compact revision number
2019-03-18 09:32:21 -07:00
Chris Lu
104922a3db
text wrapping
2019-03-18 00:35:15 -07:00
Chris Lu
44647a46c0
needle scanner read appendAtNs
2019-03-17 23:28:43 -07:00
Chris Lu
8b26d15740
refactoring: simplify function parameter
2019-03-17 22:32:01 -07:00
Chris Lu
2fcc88116e
go fmt
2019-03-17 20:27:36 -07:00
Chris Lu
aca653c08b
weed shell: list volumes
2019-03-17 20:27:08 -07:00
Chris Lu
22fbbf023b
adjust collection list output
2019-03-17 17:31:05 -07:00
Chris Lu
e48267e287
adjust output
2019-03-17 17:28:29 -07:00
Chris Lu
657dd2e6c9
add shell command to list all collections
2019-03-16 13:43:16 -07:00
Chris Lu
b92122b885
fix test
2019-03-16 09:50:57 -07:00
Chris Lu
36b632ebcb
fix tests
2019-03-16 09:50:21 -07:00
Chris Lu
42cb05c088
add context.Context
2019-03-15 17:23:02 -07:00
Chris Lu
55bab1b456
add context.Context
2019-03-15 17:20:24 -07:00
Chris Lu
cece860bfd
add context to all filer APIs
2019-03-15 15:55:34 -07:00
Chris Lu
55c85f3a66
adding context
2019-03-15 15:26:09 -07:00
Chris Lu
d30600e6d6
fix filer starting under "weed server" when ip is not localhost
...
fix https://github.com/chrislusf/seaweedfs/issues/886
2019-03-15 09:57:33 -07:00
Chris Lu
a4e8aa98a0
weed server: mkdir first, and set the master ip address
...
fix https://github.com/chrislusf/seaweedfs/issues/885
2019-03-15 00:32:42 -07:00
Chris Lu
95ef4513c8
print out memory size
2019-03-13 23:14:40 -07:00
Chris Lu
346541a101
print out per entry memory usage
2019-03-13 23:07:24 -07:00
Chris Lu
99ce10daeb
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2019-03-11 00:58:17 -07:00
Chris Lu
2ce4dae795
move to 1.26
2019-03-11 00:58:12 -07:00
chenwanli
c668e55e07
Revert "fix https://github.com/chrislusf/seaweedfs/issues/780 "
...
This reverts commit 31038acccf
.
2019-03-11 11:48:11 +08:00
Chris Lu
6fe071175d
mount: add option to disable allow others
...
fix https://github.com/chrislusf/seaweedfs/issues/877
2019-03-10 19:45:40 -07:00
Chris Lu
ad08a52ab6
synchronously upload files
...
fix https://github.com/chrislusf/seaweedfs/issues/807
2019-03-09 13:24:22 -08:00
Chris Lu
4773497d2c
add back "/cluster/status"
...
fix https://github.com/chrislusf/seaweedfs/issues/870
2019-03-06 12:10:45 -08:00
Chris Lu
ab32556a51
weed server: fix error coming from moving raft to grpc
...
fix https://github.com/chrislusf/seaweedfs/issues/872
2019-03-06 10:21:49 -08:00
Chris Lu
0898e6c38e
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2019-03-04 13:00:10 -08:00
Chris Lu
d312c55bbe
file path supports windows, avoiding back slashes
...
fix https://github.com/chrislusf/seaweedfs/issues/868
2019-03-04 13:00:08 -08:00
Chris Lu
f9dcf56465
adjust log level for loading configurations
2019-03-04 09:02:58 -08:00
chenwanli
f233bbe9c0
Fix https://github.com/chrislusf/seaweedfs/issues/825#issuecomment-469097375
2019-03-04 12:43:43 +08:00
Chris Lu
219b651bc3
jwt check the base file id
...
fix https://github.com/chrislusf/seaweedfs/issues/867
2019-03-03 10:17:44 -08:00
Chris Lu
2812c14520
master: add option to disable http operations
2019-03-02 05:10:05 -08:00
chenwanli
900aad4935
Fix https://github.com/chrislusf/seaweedfs/issues/864
2019-02-28 11:37:44 +08:00
Chris Lu
cb07d15254
add namespace for s3
2019-02-27 00:21:37 -08:00
Chris Lu
70ac2f6ea8
add namespace for ListAllMyBucketsResult
2019-02-26 23:23:04 -08:00
chenwanli
fd27ed7755
Fix https://github.com/chrislusf/seaweedfs/issues/861
2019-02-26 17:12:39 +08:00
Chris Lu
344caf3cd7
1.25
2019-02-25 00:44:48 -08:00
Chris Lu
7bbe24dd28
volume server directly support https
2019-02-25 00:43:36 -08:00
Chris Lu
c892b898a1
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2019-02-25 00:34:25 -08:00
Chris Lu
0d84e32f56
update library versions
2019-02-25 00:34:21 -08:00
chenwanli
0bc3cdd26a
Fix https://github.com/chrislusf/seaweedfs/issues/825
2019-02-25 10:47:41 +08:00
Chris Lu
0a106c1757
updated glide
2019-02-20 09:09:47 -08:00
Chris Lu
e108688990
avoid grpc 5 seconds timeout
...
some operations may take longer than 5 seconds.
only keep the timeout for raft operations
2019-02-20 01:01:01 -08:00
Chris Lu
0fa1be8c4b
adjust formatting and comments
2019-02-19 21:10:10 -08:00
Chris Lu
72c2e6bb8f
remove unused file
2019-02-19 14:16:19 -08:00
Chris Lu
07af52cb6f
raft change from http to grpc
...
master grpc port is fixed to http port + 10000
2019-02-18 22:38:14 -08:00
Chris Lu
448645203a
remove unused variables
2019-02-18 20:05:55 -08:00
Chris Lu
d37c3ab7a5
adjust error message
2019-02-18 18:03:27 -08:00
Chris Lu
2442d56671
adjust package name
2019-02-18 18:03:16 -08:00
Chris Lu
9a4dda3011
Merge pull request #855 from chrislusf/add_jwt
...
Add jwt
2019-02-18 15:05:32 -08:00
Chris Lu
a1c7dc3806
avoid changing attributes for mount directory
2019-02-18 12:14:28 -08:00
Chris Lu
77b9af531d
adding grpc mutual tls
2019-02-18 12:11:52 -08:00
Chris Lu
55761ae806
skip nil options
2019-02-16 12:49:58 -08:00
Chris Lu
98a03b38e5
avoid util package depends on security package
2019-02-16 12:23:35 -08:00
Chris Lu
31aa60ce05
Merge branch 'master' into add_jwt
2019-02-16 08:55:38 -08:00
Chris Lu
ad257ae179
simplify isLeader() logic
2019-02-16 08:55:23 -08:00
Chris Lu
712a0e19c8
Merge branch 'master' into add_jwt
2019-02-15 10:01:16 -08:00
Chris Lu
2ec6a679c2
avoid "fchmod failed" during cp
2019-02-15 10:00:27 -08:00
Chris Lu
157c0f7c01
add jwt for writes
2019-02-15 09:59:22 -08:00
Chris Lu
7103c1ab7e
go fmt
2019-02-15 00:09:48 -08:00
Chris Lu
74fb237727
benchmark can work in secure mode
2019-02-15 00:09:19 -08:00
Chris Lu
a3b0e39b06
weed mount deletion always goes to weed filer
2019-02-14 07:07:38 -08:00
Chris Lu
215cd27b37
add authorizing fileId write access
...
need to secure upload/update/delete for benchmark/filer/mount
need to add secure grpc
2019-02-14 00:08:20 -08:00
Chris Lu
4ff4a147b2
cleanup security.Secret
2019-02-09 21:56:32 -08:00
Chris Lu
501bd72b1c
wip: add security.toml file
2019-02-09 21:07:12 -08:00
Chris Lu
729d71fd4c
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2019-02-09 12:52:12 -08:00
Chris Lu
8afc632484
raft: use the first master to bootstrap the election
2019-02-09 12:52:09 -08:00
Chris Lu
744abc2690
log raft state changes
2019-02-09 12:51:14 -08:00
Sergey
aa5ccff6d2
fixing of typos
2019-02-06 18:59:15 +05:00
Chris Lu
1334507595
Revert "randomize based on self address"
...
This reverts commit 6230eb28a6
.
2019-01-28 12:12:51 -08:00
Chris Lu
6230eb28a6
randomize based on self address
...
fix #851
2019-01-28 11:55:33 -08:00
Chris Lu
221105eea3
Revert "use the first entry to bootstrap master cluster"
...
This reverts commit 40c8725ffa
.
2019-01-28 11:46:46 -08:00
Chris Lu
7a493bbefa
better way to detect a master isLeader
2019-01-28 10:36:16 -08:00
Chris Lu
40c8725ffa
use the first entry to bootstrap master cluster
...
fix https://github.com/chrislusf/seaweedfs/issues/851
2019-01-28 10:35:28 -08:00
Chris Lu
adcfe66034
1.24
2019-01-26 00:15:50 -06:00
Chris Lu
834f414af9
add a timeout
2019-01-26 00:15:42 -06:00
Chris Lu
09471b46f9
fix related tests
2019-01-26 00:10:29 -06:00
Chris Lu
5668ed798d
memory needle map mark size to be TombstoneFileSize
...
fix https://github.com/chrislusf/seaweedfs/issues/850
2019-01-26 00:05:44 -06:00
Chris Lu
d3f675abc6
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2019-01-24 22:39:41 -08:00
Chris Lu
e70f740deb
set file attributes
...
fix https://github.com/chrislusf/seaweedfs/issues/847
2019-01-24 19:38:04 -08:00
bingoohuang
cf0fc797ac
use tempFile.Truncate to reset file size to speedup test
2019-01-23 16:24:51 +08:00
Chris Lu
3f56b12ed4
raft: adding idle connection time out
...
another attempt to fix https://github.com/chrislusf/seaweedfs/issues/825
2019-01-22 09:25:25 -08:00
chenwanli
f47f8be912
Fix block-waiting for leader when there are no volumes
2019-01-22 15:58:37 +08:00
Chris Lu
8ec57e18c7
1.23
2019-01-18 14:27:34 -08:00
Chris Lu
a6bffc58f9
go fmt
2019-01-18 14:15:10 -08:00
Chris Lu
67e2ea72be
master add separate grpc port
...
due to https://github.com/soheilhy/cmux/issues/64
fix https://github.com/chrislusf/seaweedfs/issues/820
fix https://github.com/chrislusf/seaweedfs/issues/840
fix https://github.com/chrislusf/seaweedfs/issues/841
2019-01-18 14:14:47 -08:00
Chris Lu
dfae0f4e9d
1.22
2019-01-18 00:25:20 -08:00
Chris Lu
e84dcb0bfe
fix timeout issue, but can not reproduce with local built binaries
...
fix https://github.com/chrislusf/seaweedfs/issues/841
2019-01-18 00:24:40 -08:00
Chris Lu
1d103e3ed5
timeout http connections
...
possible fix for https://github.com/chrislusf/seaweedfs/issues/825
2019-01-17 23:38:33 -08:00
Chris Lu
b881e11a16
1.21
2019-01-16 20:44:38 -08:00
bingoohuang
ab6be025d7
go fmt and fix some typo
2019-01-17 09:17:19 +08:00
bingoohuang
6ddfaf33cb
extract VolumeFileScanner for ScanVolumeFile
2019-01-16 17:48:59 +08:00
bingoohuang
27093bc2e5
fix needle Append return offset to avoid uint32 overflow
2019-01-16 16:48:03 +08:00
bingoohuang
219838db87
fail fast when failed to write local volume
2019-01-15 15:47:43 +08:00
Chris Lu
2ff95ead57
Merge pull request #832 from PapaYofen/fix-830
...
Fix https://github.com/chrislusf/seaweedfs/issues/830
2019-01-12 17:26:51 -08:00
chenwanli
b71d6f4b88
Fix https://github.com/chrislusf/seaweedfs/issues/830
2019-01-12 10:07:19 +08:00
chenwanli
80cf99db1b
Set rpc dial timeout to 5s
2019-01-11 18:23:31 +08:00
Chris Lu
8613e4f558
Merge pull request #827 from suilongfei/master
...
change log_dir to logdir for the "flag redefined:log_dir" error when use another package with "github.com/golang/glog"
2019-01-10 09:10:07 -08:00
Chris Lu
2d23d86fd3
no timeout for volume vacuum
...
revert changes on volume vacuum timemout from https://github.com/chrislusf/seaweedfs/pull/829
2019-01-10 09:07:40 -08:00
chenwanli
39c7455881
Set http timeout to 5s
2019-01-10 20:21:08 +08:00
chenwanli
0a3e83a36a
Set timeout for master and volume non-streaming rpc
2019-01-10 19:41:03 +08:00
隋龙飞@视频结构化
b864cf7e07
change log_dir flag to logdir
...
change log_dir flag to logdir
2019-01-10 17:17:32 +08:00
隋龙飞@视频结构化
ea4f29d1e7
change log_dir flag to logdir
...
change log_dir flag to logdir
2019-01-10 17:16:58 +08:00
Chris Lu
2a75a36b27
set cassandra ttl with entry.TTLSec
2019-01-08 20:26:19 -08:00
byungjo.yoon
58fd2e5a19
Set redis key ttl with entry.TTLSec
2019-01-09 11:48:19 +09:00
Chris Lu
badd8fdf8f
shortcut for empty file
2019-01-08 09:03:28 -08:00
chenwanli
41fe31b5d0
Speed up heartbeat to new master leader
2019-01-07 16:09:15 +08:00
Chris Lu
e768e64fbc
1.20
2019-01-06 10:52:26 -08:00
Chris Lu
bd32108a90
disable async file deletion
2019-01-05 22:51:50 -08:00
Chris Lu
be9a7592a1
incrementally calculate visible intervals
2019-01-05 22:23:44 -08:00
Chris Lu
6b5d6bb5a6
adjust error names
2019-01-05 19:52:38 -08:00
Chris Lu
9383c91eb1
wait to read again if the volume is compacting
2019-01-05 19:52:17 -08:00
Chris Lu
fe50224ea0
refactoring
2019-01-05 15:21:56 -08:00
Chris Lu
6740427638
weed mount: reduce default chunk size
2019-01-05 15:17:03 -08:00
Chris Lu
d9f9aca453
refactoring
2019-01-05 15:16:39 -08:00
Chris Lu
251b596bc5
increase batch deletion size to 1024
2019-01-03 22:21:57 -08:00
Chris Lu
aac11fb832
reduce directory list cache usage
2019-01-03 22:21:39 -08:00
Chris Lu
7f74577db1
add Last-Modified to HEAD requests
...
to work better with restic
2019-01-02 14:23:25 -08:00
Chris Lu
ceafb7d145
1.19
2019-01-02 12:59:20 -08:00
Chris Lu
28a41fda3e
assign new file id even on filer PUT operations
2019-01-02 12:58:26 -08:00
Chris Lu
d67f7ddfaf
adjust log
2019-01-02 12:58:06 -08:00
Chris Lu
e49a38842a
set etag in response
2019-01-02 12:57:54 -08:00
Chris Lu
d5ebfd5b0c
fix s3cmd put
...
1. fix volume put data size
2019-01-02 12:57:33 -08:00
Chris Lu
3339325334
fix s3 ListAllMyBucketsResult to work with s3cmd
2019-01-02 11:36:29 -08:00
Chris Lu
43db7ac123
fix spelling
2019-01-02 02:38:32 -08:00
Chris Lu
9539823c35
fix log
2019-01-01 03:04:44 -08:00
Chris Lu
525db94d9a
async file chunk deletion
2019-01-01 02:33:57 -08:00
Chris Lu
ffb5d3f93a
get bytes from sync pool
2019-01-01 02:14:40 -08:00
Chris Lu
832ac8129a
reduce pointer usage
2018-12-31 15:10:14 -08:00
Chris Lu
1478d7ea21
reduce file seek when writing
2018-12-31 15:08:32 -08:00
Chris Lu
81904ad336
debugging: track all memory allocations
2018-12-31 14:54:13 -08:00
Chris Lu
03cfb4267f
adjust vaccum logging
2018-12-31 00:06:52 -08:00
Chris Lu
d6181076bb
FUSE remove adjust error
2018-12-30 22:29:27 -08:00
Chris Lu
16e8610426
1.18
2018-12-30 00:52:20 -08:00
Chris Lu
0388d421d2
caching visible intervals for read
...
speeds up 4x in single thread mode
speeds up 30% in 32 threads mode
2018-12-30 00:51:44 -08:00
Chris Lu
55f7d8f801
remove extra log
2018-12-29 18:00:19 -08:00
Chris Lu
170ed53da5
weed mount: mac add AutoXattr
2018-12-29 13:55:38 -08:00
Chris Lu
d8c7844d2e
1.17
2018-12-29 13:16:38 -08:00
Chris Lu
9f3836072a
weed mount: no mac finder broswe mode
2018-12-29 13:16:23 -08:00
Chris Lu
d7bf6c5b91
switching_to_seaweedfs_fuse
2018-12-29 10:31:36 -08:00
Chris Lu
0c932d1738
add volume delete grpc API
...
fix https://github.com/chrislusf/seaweedfs/issues/808
2018-12-29 00:03:30 -08:00
Chris Lu
03c7447ad6
set mount point to existing permissions
...
fix https://github.com/chrislusf/seaweedfs/issues/806
2018-12-28 23:36:13 -08:00
Chris Lu
a8af1e3f5f
set attributes include crtime
2018-12-28 22:37:18 -08:00
Chris Lu
ba9b7b62d7
1.16
2018-12-28 15:10:49 -08:00
Chris Lu
79168ab0bb
adjust error and log
2018-12-28 15:10:23 -08:00
Chris Lu
308ac1d0d2
reuse buffer by sync pool
2018-12-28 03:27:48 -08:00
Chris Lu
260b038441
fix url
2018-12-27 14:43:36 -08:00
Chris Lu
2a19511ba8
avoid gzip benchmark data
...
related to https://github.com/chrislusf/seaweedfs/issues/804
2018-12-27 13:10:46 -08:00
Chris Lu
d3660ffce8
avoid gzipping files without file extensions
...
fix https://github.com/chrislusf/seaweedfs/issues/804
2018-12-27 12:17:05 -08:00
Chris Lu
253f190f48
weed mount add symlink support
2018-12-25 22:45:44 -08:00
Chris Lu
9eb2bee86a
avoid error log if not found
2018-12-24 17:45:26 -08:00
Chris Lu
ba923e2864
error if the last arguement does not end with forward slash
2018-12-23 18:20:11 -08:00
Chris Lu
332668a3fd
1.15
2018-12-23 12:54:41 -08:00
Chris Lu
c043fd17cb
detect more gzippable content
2018-12-22 15:05:31 -08:00
Chris Lu
49f40cd145
adjust help message
2018-12-22 15:05:15 -08:00
Chris Lu
852ee21835
avoid .gz auto decompression
2018-12-22 13:58:16 -08:00
Chris Lu
be946c9e54
filer store original data size instead of data size after gzip
2018-12-22 13:11:07 -08:00
Chris Lu
141d302492
fix binarySearchCompactSection
2018-12-22 11:52:58 -08:00
Chris Lu
d3839fe279
go fmt
2018-12-22 11:11:57 -08:00
Chris Lu
5333f2984a
streaming updates for large entries or large index file
...
fix https://github.com/chrislusf/seaweedfs/issues/801
2018-12-22 11:10:08 -08:00
Chris Lu
36d13355bb
compact map fix bug when iterating
2018-12-22 11:05:29 -08:00
Chris Lu
f64dc7465b
remove index file
...
fix https://github.com/chrislusf/seaweedfs/issues/793
2018-12-17 20:33:32 -08:00
Chris Lu
16634235c1
1.14 and HCFS 1.0.3
2018-12-17 10:03:03 -08:00
Chris Lu
52712782a2
adding link
2018-12-17 00:31:23 -08:00
Chris Lu
2fb740a2b9
fix upload path
2018-12-17 00:21:34 -08:00
Chris Lu
7b7f4215aa
filer update existing entry
2018-12-17 00:20:00 -08:00
Chris Lu
6071be104b
adjust log format
2018-12-17 00:03:14 -08:00
Chris Lu
3ac54792e1
paginate when filer deleting and FUSE mount renaming
2018-12-16 23:20:08 -08:00
Chris Lu
93d0ad09af
error log format
2018-12-16 17:37:10 -08:00
Chris Lu
66d04cb83f
1.12
2018-12-15 08:17:24 -08:00
Chris Lu
42cb9b76c4
reduce needle map memory usage by 25%
2018-12-15 05:55:56 -08:00
Chris Lu
42b8f1145a
weed mount ls paginate the results
...
fix https://github.com/chrislusf/seaweedfs/issues/787
2018-12-11 07:41:50 -08:00
Chris Lu
39ea8484c1
adjust proto API
2018-12-11 07:17:10 -08:00
Chris Lu
168f461761
fix MaxUint8
2018-12-10 20:57:51 -08:00
Chris Lu
8f78a069c6
fix possible name overflow when len(name)>255
2018-12-10 20:57:01 -08:00
Chris Lu
4a490fe35a
go fmt
2018-12-09 01:27:25 -08:00
Chris Lu
5741134ca0
change overflow from map to slice
2018-12-09 01:27:11 -08:00
Chris Lu
d9189c857c
remove unused file
2018-12-09 00:30:53 -08:00
Chris Lu
df4b9df142
add memory usage report
2018-12-09 00:15:23 -08:00
Chris Lu
ec5a305624
refactor
2018-12-09 00:12:37 -08:00
Chris Lu
9d3be33e5c
optimize a little bit
2018-12-08 21:45:14 -08:00
Chris Lu
36d1e739e6
1.11
2018-12-08 00:32:34 -08:00
Chris Lu
1bfb96f34d
optimization for reading whole chunk with gzip encoding
2018-12-07 01:57:55 -08:00
Chris Lu
29f1673d97
refactoring
2018-12-07 01:25:01 -08:00
Chris Lu
c28e8a2397
refactoring
2018-12-06 00:44:41 -08:00
Chris Lu
ffa2827ab1
fail fast if two notification queues or inputs are enabled
2018-12-06 00:37:59 -08:00
Chris Lu
63619f8b8e
fail early if two filer stores are defined
2018-12-06 00:32:36 -08:00
Chris Lu
52b24a9902
default "weed server -filer" to same directory as -mdir
2018-12-05 23:24:25 -08:00
Chris Lu
fb43a9d7b9
adjust message
2018-12-05 02:03:03 -08:00
chenwanli
31038acccf
fix https://github.com/chrislusf/seaweedfs/issues/780
2018-12-05 14:06:16 +08:00
Chris Lu
650d1af978
HCFS support change user name and group names
2018-12-04 00:46:00 -08:00
Chris Lu
b0838db5a8
support hdfs user name and group names
2018-12-04 00:13:40 -08:00
Chris Lu
7ace0efd65
revert prev
2018-12-02 23:25:37 -08:00
Chris Lu
738247d47e
do not throw error if not found
2018-12-02 23:20:29 -08:00
Chris Lu
c6b646e92c
skip creation for root
2018-12-02 23:16:17 -08:00
Chris Lu
bea162c34c
add warning for empty attributes
2018-12-02 22:57:59 -08:00
Chris Lu
f0677c5af1
default root to 0755
2018-12-02 19:59:47 -08:00
Chris Lu
db22666a30
add default root directory
2018-12-02 19:42:50 -08:00
Chris Lu
a3def2bbd9
for 1.10
2018-11-29 09:15:04 -08:00
Chris Lu
7a6f49cd42
refactor a bit more
...
fix https://github.com/chrislusf/seaweedfs/issues/777
2018-11-29 00:07:54 -08:00
Chris Lu
ceca078acb
avoid overwriting file or directory
...
fix https://github.com/chrislusf/seaweedfs/issues/777
2018-11-29 00:00:56 -08:00
Chris Lu
1cbd53c01c
WIP SeaweedFileSystem added mkdirs, getFileStatus, listStatus, delete
2018-11-25 13:43:26 -08:00
Chris Lu
871dee4674
fix test
2018-11-23 12:37:09 -08:00
Chris Lu
86dd933596
go fmt
2018-11-23 00:26:15 -08:00
Chris Lu
444dfded84
add fs.FSStatfser for SeaweedFS weed mount
2018-11-23 00:24:51 -08:00
Chris Lu
2e32b44061
add template for fs.FSStatfser
2018-11-22 23:05:22 -08:00
Chris Lu
6344469a9b
set fuse subtype
2018-11-22 23:04:23 -08:00
Chris Lu
d64546c791
adjust logs
2018-11-22 14:31:10 -08:00
Chris Lu
e12b6c0306
add subimit count parameter
...
related to https://github.com/chrislusf/seaweedfs/issues/631
2018-11-22 13:45:29 -08:00
Chris Lu
0ed816d4e9
formatting error log
2018-11-22 00:42:56 -08:00
Chris Lu
19d546938a
forgot to upgrade version number
2018-11-21 23:30:59 -08:00
Chris Lu
60d3681371
default images.fix.orientation to false
2018-11-21 16:50:42 -08:00
Chris Lu
99256100a0
fix tests
2018-11-21 16:35:27 -08:00
Chris Lu
e5ad2223a1
add missing intervals
...
fix https://github.com/chrislusf/seaweedfs/issues/774
2018-11-21 16:25:13 -08:00
Chris Lu
c9b3ef58de
options to control filer.export verbosity
2018-11-21 15:15:40 -08:00
Chris Lu
2f2e417b68
detailed export
2018-11-21 15:12:47 -08:00
Chris Lu
b282e34dc2
async file chunk deletion
2018-11-20 20:56:28 -08:00
Chris Lu
5065d4ab2d
master add grpc API for fileid assigning
2018-11-20 11:35:45 -08:00
Chris Lu
b318167cd3
batch delete file ids
2018-11-20 11:35:08 -08:00
Chris Lu
06474e5990
1.08
2018-11-18 22:43:05 -08:00
Chris Lu
ee3fe07acf
fix sort bug
2018-11-18 21:59:53 -08:00
Chris Lu
469863f7f0
merge interval now is quite faster with reusing slices
2018-11-18 21:24:58 -08:00
Chris Lu
d2924c7748
merge intervals is a bit faster
2018-11-18 20:31:39 -08:00
Chris Lu
cb5a10c6a3
adjust the tests
2018-11-18 17:05:36 -08:00
Chris Lu
f1db22d48b
add benchmark test for merging intervals
2018-11-18 16:49:08 -08:00
Chris Lu
4fcfc9410f
cleanup
2018-11-18 11:51:38 -08:00
Chris Lu
9655dc9ca9
simpler logic
2018-11-18 10:07:30 -08:00
Chris Lu
f8eb988347
1.07
2018-11-18 08:10:15 -08:00
Chris Lu
60421b23f0
reduce weed mount memory usage
2018-11-18 07:49:14 -08:00
Chris Lu
2929563294
1.06
2018-11-14 23:51:15 -08:00
Chris Lu
d3b7965c76
caching grpc clients
2018-11-14 23:50:46 -08:00
Chris Lu
d89774cb7f
cache grpc client connections to improve performance
2018-11-14 23:31:39 -08:00
Chris Lu
2e6a3c7b16
reduce memory usage
2018-11-14 22:48:54 -08:00
Chris Lu
f224b9fe28
1.05
2018-11-13 23:54:17 -08:00
Chris Lu
11b0a9cb76
fix possible error when reusing file handles
2018-11-13 23:53:17 -08:00
Chris Lu
a62adf62ca
a bit faster
2018-11-12 11:19:13 -08:00
Chris Lu
affe708961
a bit faster
2018-11-11 00:43:30 -08:00
Chris Lu
a09eaf8756
adjusting, seems there are some perf benefit
2018-11-11 00:08:22 -08:00
Chris Lu
8178fc9973
adjust logging
2018-11-11 00:07:46 -08:00
Chris Lu
2697d6a4fb
fix deleting bucket
2018-11-08 22:15:21 -08:00
Chris Lu
0d095f72e4
update to 1.04
2018-11-08 21:42:45 -08:00
Chris Lu
a4ceb051a7
use MD5 for ETag to be consistent with Amazon S3
2018-11-08 21:41:02 -08:00
Chris Lu
6e53c38c2f
update to 1.03
2018-11-08 07:37:44 -08:00
Chris Lu
cbd94b18a5
improve "ls -al" performance for large directory
2018-11-08 07:37:34 -08:00
Chris Lu
6e11923551
upgrade to 1.02
2018-11-07 11:37:32 -08:00
Chris Lu
ed8efb5aef
much improved "ls -al" performance
2018-11-07 11:35:13 -08:00
Chris Lu
60db731e36
update to 1.01
2018-11-05 08:54:40 -08:00
Chris Lu
f9dbaf6292
delete file by name instead of referencing the dataFile pointer
2018-11-05 08:53:38 -08:00
alex8224
8d3f9c0b75
fix invalid point when delete collection
2018-11-05 19:53:55 +08:00
Chris Lu
1a64a817ac
update to 1.00
2018-11-04 22:13:16 -08:00
Chris Lu
85d1d99459
default "weed export to current enabled store"
2018-11-04 12:59:29 -08:00
Chris Lu
6c45787a37
selectively export files
2018-11-04 12:07:33 -08:00
Chris Lu
ff4189ab25
go fmt
2018-11-04 11:59:08 -08:00
Chris Lu
8dfac6a4cf
working b2 sink
2018-11-04 11:58:59 -08:00
Chris Lu
f050b22d6c
close file in ScanVolumeFile()
...
fix https://github.com/chrislusf/seaweedfs/issues/761
2018-11-04 00:28:24 -07:00
Chris Lu
93e0dddbf3
add function description
2018-11-03 22:32:21 -07:00
Chris Lu
a64613172d
bootstrap filer replication with weed filer.export -targetStore=notification
2018-11-03 12:43:45 -07:00
Chris Lu
3674ad9f8e
go fmt
2018-11-01 01:12:21 -07:00
Chris Lu
db584ff7f8
separate into notification.toml, add gcp pub/sub message queue
2018-11-01 01:11:09 -07:00
Chris Lu
6219a9ad1f
filer ordered notifications
2018-10-31 21:48:05 -07:00
Chris Lu
4c97ff3717
support AWS SQS as file change notification message queue
2018-10-31 01:11:19 -07:00
Chris Lu
200cbcde62
Merge pull request #756 from zzoe/patch-1
...
Update volume_server_handlers_write.go
2018-10-30 21:52:19 -07:00
zoe
a11c28cb4c
Update volume_server_handlers_write.go
...
size declared and not used
2018-10-31 09:36:32 +08:00
Chris Lu
fe31f389b0
add configuration instructions
2018-10-30 02:29:11 -07:00
zoe
29eff0ecd7
Update volume_server_handlers_write.go
...
返回实际文件大小,而不是needle的大小
2018-10-29 21:05:51 +08:00
Chris Lu
9e98bba206
go fmt
2018-10-24 00:00:01 -07:00
Chris Lu
6a756136ef
go fmt
2018-10-23 23:59:49 -07:00
Chris Lu
c58b73ad9b
adding BackBlaze, Azure
2018-10-23 23:59:40 -07:00
Chris Lu
b861899919
use released version of azblob
...
replace https://github.com/chrislusf/seaweedfs/pull/754
2018-10-23 02:18:46 -07:00
Chris Lu
82a5d39778
add back volume vacuum cleanup
2018-10-18 23:02:58 -07:00
Chris Lu
496a7398d1
adjust logging
2018-10-18 20:47:30 -07:00
Chris Lu
201c24c110
add locking to volume layout vacuum
2018-10-18 20:34:43 -07:00
Chris Lu
46eb77f9bb
move DiskStatus and MemStatus to protobuf
2018-10-15 22:25:28 -07:00
Chris Lu
eec951cad2
migrate volume sync to gRpc
2018-10-15 21:44:41 -07:00
Chris Lu
f8b2d3cacc
move volume mount/unmount on volume server to grpc
2018-10-15 01:48:15 -07:00
Chris Lu
66a353dcb5
remove volume server /admin/volume/delete
2018-10-15 01:26:49 -07:00
Chris Lu
fda771c83f
migrate volume sync status to grpc API on volume server
2018-10-15 01:19:15 -07:00
Chris Lu
b423bb9e2d
migrate assign volume to grpc API on volume server
2018-10-15 00:40:46 -07:00
Chris Lu
333709657c
adjust log level
2018-10-15 00:40:12 -07:00
Chris Lu
f9410bcaa7
adjust log level for deletes
2018-10-15 00:39:56 -07:00
Chris Lu
8301519fb0
migrate delete collection to grpc API on volume server
2018-10-15 00:03:55 -07:00
Chris Lu
db152ca540
fix error log format
2018-10-14 23:50:54 -07:00
Chris Lu
67d1e572ca
rename files
2018-10-14 23:47:29 -07:00
Chris Lu
b1daede91b
move volume vacuum to gRpc
2018-10-14 23:12:43 -07:00
Chris Lu
91ac2e0dd9
go fmt
2018-10-14 00:30:20 -07:00
Chris Lu
249eeccf32
0.99 release
2018-10-14 00:19:04 -07:00
Chris Lu
9db53cf256
use the configurable chunk size
2018-10-14 00:18:52 -07:00
Chris Lu
ff66269b62
use grpc to replace http APIs for batch volume id lookup and batch delete
...
1. remove batch volume id lookup http API /vol/lookup
2. remove batch delete http API /delete
2018-10-14 00:12:28 -07:00
Chris Lu
3ddcd87098
increase weed mount IO perforance
2018-10-14 00:10:17 -07:00
Chris Lu
ab85118233
error handling when kafka not ready
2018-10-13 23:30:00 -07:00
Chris Lu
41aba0c02c
fix when no files in filer
2018-10-12 00:45:28 -07:00
Chris Lu
4af6b4eb0e
fix master static resource display problem
2018-10-12 00:05:15 -07:00
Chris Lu
556382ff5f
adding basic grpc to volume server
2018-10-11 01:16:33 -07:00
Chris Lu
64ed47502b
rename file
2018-10-11 00:21:32 -07:00
Chris Lu
08266b7256
go fmt
2018-10-11 00:08:13 -07:00
Chris Lu
1d7e01b754
reformat
2018-10-11 00:05:54 -07:00
Chris Lu
da6154b29c
refactor volume server to startVolumeServer()
2018-10-11 00:04:31 -07:00
Chris Lu
60d2f1557d
refactor func name
2018-10-10 23:19:54 -07:00
Chris Lu
b7763c1cb6
releasing 0.98
2018-10-10 21:44:10 -07:00
Chris Lu
9f4c2f87d3
adding filer replication sink to Azure
2018-10-09 01:35:48 -07:00
Chris Lu
453d0be4d4
adjust error format
2018-10-09 01:35:30 -07:00
Alex Wang
4db68669b6
[bugfix] Fix interrupt hook overwritten bug
2018-10-09 15:43:32 +08:00
Chris Lu
96a23591e7
remove comments
2018-10-07 22:21:00 -07:00
Chris Lu
dbe33236fb
embed static resources via statik
2018-10-07 21:42:57 -07:00
Chris Lu
fffbc2d490
embed static resources via statik
2018-10-07 10:54:05 -07:00
Chris Lu
a5b5887efc
add filer replication to aws s3
2018-10-06 17:10:15 -07:00
Chris Lu
84d4a2c8a0
add filer replication to google cloud storage
2018-10-06 13:08:38 -07:00
Chris Lu
f8c2704d2b
reformat
2018-10-06 13:04:33 -07:00
Chris Lu
14858f6224
add logging
2018-10-06 13:01:38 -07:00
Chris Lu
04da4c8094
add logging
2018-10-06 13:01:29 -07:00
Chris Lu
4a8ef198d7
add logging
2018-10-06 13:00:33 -07:00
Chris Lu
56c5c7b1b6
add google cloud storage
2018-10-04 01:14:44 -07:00
Chris Lu
e8ef501f02
add s3 replication sink
2018-10-03 23:36:52 -07:00
Chris Lu
56a5d5af8d
ignore duplicated key error if the store has duplicated primary key prevention
...
fix https://github.com/chrislusf/seaweedfs/issues/741
2018-09-30 00:49:52 -07:00
Chris Lu
98384b62fd
adjust description
2018-09-28 01:58:34 -07:00
Chris Lu
f11d531d45
adjust logging
2018-09-28 01:54:10 -07:00
Chris Lu
06c43286cd
skip error when deletion failed
2018-09-28 00:10:49 -07:00
Chris Lu
746017ea4f
releasing 0.97
2018-09-27 21:51:41 -07:00
Chris Lu
f7ff84f17f
Revert "smalller block size for better accuracy"
...
This reverts commit 97ecd20fc9
.
2018-09-27 09:42:24 -07:00
Chris Lu
97ecd20fc9
smalller block size for better accuracy
2018-09-27 09:39:17 -07:00
Chris Lu
f53632a84b
add block size for linux du
...
du already working on Mac
2018-09-27 09:33:36 -07:00
Chris Lu
a362261972
avoid possible racing issue if leveldb or boltdb is used.
...
remove the to-be-generated files during commit
fix https://github.com/chrislusf/seaweedfs/issues/738
2018-09-26 18:45:51 -07:00
Chris Lu
31ed352ab6
replication handle cases when entry already exists
2018-09-25 09:27:03 -07:00
Chris Lu
60c1ada4c1
remove filer grpc API GetEntryAttributes
2018-09-25 09:18:06 -07:00
Chris Lu
578f8a6dec
fix empty file reading on linux
2018-09-23 10:01:00 -07:00
Chris Lu
200d9b9ac2
add file name to log
2018-09-23 09:48:55 -07:00
Chris Lu
a60767c43a
add more instructions
2018-09-23 09:26:39 -07:00
Chris Lu
12ba8b9799
generate useable filer.toml
2018-09-23 09:26:25 -07:00
Chris Lu
b1b8c4ed32
join via filepath
2018-09-23 01:46:50 -07:00
Chris Lu
b16d47030e
save and use last processed kafka offsets
2018-09-23 01:34:40 -07:00
Chris Lu
9fe24991d5
refactoring
2018-09-23 00:40:36 -07:00
Chris Lu
7d6b2a4740
add ETag to upload results and chunks
2018-09-22 22:12:21 -07:00