Chris Lu
c602073395
adjust logs
2019-12-11 21:40:33 -08:00
Chris Lu
1ddfbff2d3
deletion add option skipChunkDeletion
2019-12-11 14:58:22 -08:00
Chris Lu
f06b65a6fd
filer: EXPERT add shouldDeleteChunks parameter for fast deletion
2019-12-11 07:50:53 -08:00
Yoni Nakache
9e45702334
Filer: Enable delete entries without flag volume
...
same as happens on the s3 API on delete bucket,
to reduce the IO requests on the system, you can just clear recursively folders without flagging them in volume server (they will be removed on delete bucket/drop collection)
this will remove the user ability to retrieve data under these folders
2019-12-11 11:30:19 +02:00
Chris Lu
40ff4eea14
fix nil problem
2019-12-10 23:13:14 -08:00
Chris Lu
356bd1b629
adjust text
2019-12-10 22:09:29 -08:00
Chris Lu
5847c5afff
return meaningful error log
2019-12-10 22:09:29 -08:00
Chris Lu
3727d2488f
Merge pull request #1149 from song-zhang/improve-replications-consistency
...
improve data consistency when replication write
2019-12-09 20:13:03 -08:00
Chris Lu
10bd3c6b4b
refactoring
2019-12-08 19:44:16 -08:00
Chris Lu
2b8e20f122
remove unused functions
2019-12-08 18:33:25 -08:00
zhangsong
5eb86aad47
improve data consistency when replication write
2019-12-08 22:21:35 +08:00
Chris Lu
e2a8cdfc48
webdav: fix writing file
...
fix https://github.com/chrislusf/seaweedfs/issues/1114
2019-12-07 07:56:51 -08:00
Chris Lu
0eda75fa2c
add logging
2019-12-07 07:56:05 -08:00
Chris Lu
f81d43442b
filer: speed up filer.meta.save by parallelizing
2019-12-06 22:20:59 -08:00
Chris Lu
f38f90b7ea
cleanly remove .tier volume file
2019-12-06 22:19:47 -08:00
Chris Lu
6827cabfbb
volume: dataFileAccessLock change to RW Lock
2019-12-06 06:59:57 -08:00
Chris Lu
81f424a630
go fmt
2019-12-06 06:57:32 -08:00
Chris Lu
916423e96a
master: submit support "dataCenter" parameter
2019-12-05 15:51:24 -08:00
Chris Lu
e426bd541e
grow volumes on volume servers with slots freed by cloud storage
2019-12-03 21:36:42 -08:00
Chris Lu
7ae8b1cc86
show volume info in volume.list in weed shell
2019-12-02 23:38:56 -08:00
Chris Lu
1becbce657
display remote volumes on volume server ui page
2019-12-02 23:23:54 -08:00
Chris Lu
70648d35ad
go fmt
2019-12-02 20:49:58 -08:00
Chris Lu
6383b45bd0
add lock variable
2019-12-02 20:49:50 -08:00
Chris Lu
caae543a9f
fix test
2019-12-02 15:54:24 -08:00
Chris Lu
126fb492d2
fix test
2019-12-02 15:52:33 -08:00
Chris Lu
ec8de250e2
tiered storage: can copy to s3, read from s3
...
master not aware tiered volume yet, file assigning is not working yet
2019-12-02 15:08:31 -08:00
Chris Lu
228231f3d7
minor
2019-12-02 15:08:31 -08:00
zhangsong
e83c36e26f
fix the bug of volume never be vacuumed
2019-12-02 13:25:32 +08:00
Chris Lu
0da7b894cc
pass backend config from master to volume servers
2019-11-29 01:05:09 -08:00
Chris Lu
a9dbd52e06
Merge pull request #1145 from stlpmo-jn/volume_graceful_stop
...
let volume server graceful stop
2019-11-29 00:11:57 -08:00
stlpmo
ea9d1ebd2f
resolve the Duration.Milliseconds() is not exist in golang v11 & v12
2019-11-29 15:17:17 +08:00
Chris Lu
ea9a7e61d0
only send metrics address during on the first response
2019-11-28 20:48:08 -08:00
Chris Lu
61bc1d6ffd
tweaking
2019-11-28 18:47:51 -08:00
Chris Lu
09c6d8fa32
fix test
2019-11-28 18:44:27 -08:00
Chris Lu
f60154f330
master load backend storage config from master.toml
2019-11-28 18:33:18 -08:00
Chris Lu
641b92f53c
fix scaffold error
2019-11-28 18:04:25 -08:00
Chris Lu
ee2072dff9
volume: avoid inifinite loop reading ec volume info if error happens
2019-11-27 23:07:53 -08:00
Chris Lu
0b822a6fb5
Merge pull request #1142 from iliul/wip-fuse-mount
...
fuse: check mount point available before do runmount
2019-11-27 19:22:51 -08:00
Lei Liu
13b6f09fea
fuse: adjust linux specific logic
...
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-11-28 10:21:31 +08:00
Chris Lu
586798ecc0
go fmt
2019-11-27 12:34:57 -08:00
Chris Lu
46294b42e6
filer: support memsql
...
fix https://github.com/chrislusf/seaweedfs/issues/1141
2019-11-27 12:34:03 -08:00
Lei Liu
1ba3456ac2
go fmt
...
Signed-off-by: Lei Liu <liul.stone@gmail.com>
2019-11-27 23:08:51 +08:00
Lei Liu
4c87b222f1
fuse: check mount point available before do runmount
...
1. Use more readable mount point information
2. Fix some typos
eg:
$ df -Th
Filesystem Type Size Used Avail Use% Mounted on
localhost:8888:/ fuse.seaweedfs 206G 512 206G 1% /mnt/weedfs
$ mount | grep weedfs
localhost:8888:/ on /mnt/weedfs type fuse.seaweedfs (rw,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-11-27 22:56:44 +08:00
Chris Lu
0f9ba84274
s3
2019-11-27 03:09:45 -08:00
Chris Lu
67e5f5b558
typo
2019-11-27 03:09:45 -08:00
Lei Liu
2700ac6d1f
filer: Add attr.mode in query string
...
Signed-off-by: Lei Liu <liul.stone@gmail.com>
2019-11-27 13:57:29 +08:00
Chris Lu
5f6b360eb7
adjust error message
2019-11-24 23:13:40 -08:00
Chris Lu
2cbe79acda
fix typo in command help
2019-11-24 22:17:43 -08:00
Chris Lu
0fca1ae776
adjust build
2019-11-22 23:19:49 -08:00
Chris Lu
bc6cd93431
skip build for 32 bit computers
2019-11-22 22:52:21 -08:00
Chris Lu
249b9d8d5d
1.45
2019-11-22 14:51:16 -08:00
Chris Lu
e4c63ff2f8
go fmt
2019-11-22 14:51:09 -08:00
Chris Lu
fb3c9ed80d
shell: fs.meta.save missed directory data with more than 1024 entries
...
fix https://github.com/chrislusf/seaweedfs/issues/1130
2019-11-22 01:39:50 -08:00
divinerapier
5656d43264
can not break out of for-select block
...
Signed-off-by: divinerapier <poriter.coco@gmail.com>
2019-11-20 08:25:29 +08:00
Chris Lu
9711a6ffaa
WIP
2019-11-18 19:24:37 -08:00
Chris Lu
ba73c053c3
change uuid package
...
fix https://github.com/chrislusf/seaweedfs/issues/1126
2019-11-18 19:24:37 -08:00
chenwanli
295f3beed5
rm ldb and bdb directory
2019-11-19 09:35:06 +08:00
stlpmo
753138a3f7
test passed
2019-11-17 11:40:36 +08:00
Chris Lu
de0c983808
refactoring scaffolding for master.toml
2019-11-14 22:20:38 -08:00
Chris Lu
c6e8225a20
adjust memory mapped file size
...
related to https://github.com/chrislusf/seaweedfs/pull/1087
2019-11-12 21:05:48 -08:00
Chris Lu
66acc2c84f
passing in averageShardsPerEcNode instead of expectedTotalEcShards
...
addressing https://github.com/chrislusf/seaweedfs/issues/1111
2019-11-12 15:36:18 -08:00
Chris Lu
40514c5362
add sortEcNodesByFreeslotsDecending and sortEcNodesByFreeslotsAscending
...
addressing https://github.com/chrislusf/seaweedfs/issues/1111
2019-11-12 13:47:36 -08:00
Chris Lu
717ec47fa2
avoid double counting ec slot
...
addressing issue found in https://github.com/chrislusf/seaweedfs/issues/1111
The ec slots are already counted on the server side.
2019-11-12 09:33:51 -08:00
Lei Liu
46755ea1e1
fix master maintenance logic
...
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-11-12 14:46:10 +08:00
Chris Lu
ee90236a97
Merge branch 'master' into refactoring_dat_backend
2019-11-11 21:30:14 -08:00
Chris Lu
d7852ebb3b
Merge pull request #1102 from stlpmo-jn/add_ETCD_sequencer
...
Add etcd sequencer
2019-11-11 12:43:24 -08:00
stlpmo
d07701fa75
delete unused function
2019-11-11 18:08:48 +08:00
stlpmo
62d393d6c9
ut pass
2019-11-11 10:52:21 +08:00
stlpmo
802a0eb3fe
move from cmd-line to scaffold
2019-11-11 09:15:17 +08:00
Chris Lu
9c2f3b1b0f
fix rack_not_found case
...
fix rack_not_found in https://github.com/chrislusf/seaweedfs/issues/1111
2019-11-10 12:06:41 -08:00
Chris Lu
e6ab75972f
use constant
2019-11-10 12:05:58 -08:00
Chris Lu
471902cd51
change parameter name
2019-11-10 12:05:45 -08:00
zhangsong
61fa485700
add volume number param in assign operation
2019-11-10 20:11:03 +08:00
Chris Lu
85f8649320
refactor memory mapped file into backend storage
2019-11-09 00:10:59 -08:00
Chris Lu
c5c1d83d91
Merge branch 'master' into refactoring_dat_backend
2019-11-08 22:54:41 -08:00
Chris Lu
c34ffed43f
go fmt
2019-11-08 22:47:50 -08:00
Chris Lu
84c503c6a7
adjust ec reading log level
2019-11-08 22:41:02 -08:00
Chris Lu
6cc9e0d788
volume: fix ec shard reading
...
fix https://github.com/chrislusf/seaweedfs/issues/1106
2019-11-08 22:41:02 -08:00
zhangsong
1dd101f782
use read lock to avoid io hang during heartbeat
2019-11-08 20:27:55 +08:00
stlpmo
1c8bed3810
delete the var etcdClient and comments
...
refactor the code
add sequencer cmd-line
delete nerver used codes
2019-11-05 15:45:25 +08:00
stlpmo
364f7200ad
Create etcd_sequencer.go
...
the 1st version
2019-11-04 16:36:06 +08:00
Lei Liu
1294999d8b
return error when garbageThreshold is not a valid float number
...
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-10-31 20:43:20 +08:00
Lei Liu
1bcef02a6c
fix dir/lookup and col/delete api
...
1, Fix Layouts first letter capitalized
2, Return http 204 when delete a collection
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-10-31 16:55:45 +08:00
Lei Liu
f2f90436ef
fix leader master /dir/lookup api
...
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-10-30 16:38:40 +08:00
Chris Lu
db30a46050
skip memory mapped file when compacting
2019-10-29 23:34:38 -07:00
Chris Lu
5b950c735e
Merge branch 'master' into refactoring_dat_backend
2019-10-29 23:18:41 -07:00
Chris Lu
57e441d67b
fix compaction logic
2019-10-29 23:18:01 -07:00
Chris Lu
b7156291a8
adjust variable names
2019-10-29 23:16:43 -07:00
Chris Lu
4b5ba4927f
adjust NewDiskFile() paramaters
2019-10-29 22:37:36 -07:00
Lei Liu
c2884cace2
misc updated
...
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-10-29 21:28:28 +08:00
Chris Lu
cd7ce720c7
fix test
2019-10-29 00:37:15 -07:00
Chris Lu
19b6a16003
changed from os.file to backend.DataStorageBackend
2019-10-29 00:35:16 -07:00
Chris Lu
eb2172f63f
FUSE: add os specific mount options
...
related to https://github.com/chrislusf/seaweedfs/issues/1094
2019-10-27 09:12:10 -07:00
Chris Lu
71eb8efd20
fix error logs
2019-10-25 09:44:58 -07:00
Chris Lu
55a2221eba
master: add logging for volume server disconnection
...
add logging for https://github.com/chrislusf/seaweedfs/issues/1093
2019-10-25 09:35:19 -07:00
Chris Lu
69de05f6cb
go fmt
2019-10-25 07:45:12 -07:00
Chris Lu
288c45a690
fix iitial filer url
2019-10-25 07:44:37 -07:00
Chris Lu
05fe7a2366
Merge pull request #1092 from binbinshi/master
...
fix: weed shell filer config is constant
2019-10-25 07:08:59 -07:00
shibinbin
fe1f469978
fix : change default filer.port to 8888
2019-10-25 17:39:29 +08:00
shibinbin
3464f21b5c
fix: weed shell add filer runtime attributes
2019-10-25 17:34:03 +08:00
Chris Lu
7a51a9a582
refactor: simplifying to ReadAt() and WriteAt()
2019-10-25 00:11:25 -07:00
Chris Lu
46ed2ca902
refactoring
2019-10-24 23:41:32 -07:00
Chris Lu
9573c0f4b3
filer: filer.copy cleanup in case of failed uploads
2019-10-24 10:16:01 -07:00
Chris Lu
1b71608b3c
Merge pull request #1091 from divinerapier/fix/abused-404
...
fix abused 404 status code
2019-10-24 09:35:54 -07:00
Chris Lu
5d78871487
filer: parallize weed filer.copy for single file
2019-10-24 07:26:23 -07:00
divinerapier
f1ee6b7a54
fix abused 404 status code
...
Signed-off-by: divinerapier <poriter.coco@gmail.com>
2019-10-24 16:51:26 +08:00
Chris Lu
253dd20bc6
go fmt
2019-10-23 22:25:59 -07:00
Chris Lu
f937933b3d
refactoring
2019-10-23 22:25:53 -07:00
Chris Lu
c358f96aaa
volume: redirection follow public url
...
fix https://github.com/chrislusf/seaweedfs/issues/1089
2019-10-23 22:11:42 -07:00
Chris Lu
c0776aa40f
trying to compile
2019-10-23 01:27:08 -07:00
Chris Lu
23240efa63
filer: add tikv support
2019-10-23 00:31:31 -07:00
Chris Lu
3c865ee39b
fix missing needle header read
2019-10-22 00:57:47 -07:00
Chris Lu
c71fddee2b
go fmt
2019-10-22 00:50:50 -07:00
Chris Lu
fc412e428b
refactor ScanVolumeFileFrom()
2019-10-22 00:50:30 -07:00
Chris Lu
c9a183eb69
refactor memory map related code
2019-10-22 00:49:42 -07:00
Chris Lu
fec07c829d
go fmt
2019-10-21 23:03:48 -07:00
Chris Lu
faec9076a4
adjust parameter names
2019-10-21 22:57:01 -07:00
Chris Lu
d16c450682
fix unreachable code
2019-10-21 22:36:41 -07:00
Chris Lu
c262526d8a
Merge pull request #1087 from joeslay/master
...
Storing files in in-memory collections on windows
2019-10-21 22:30:35 -07:00
j.laycock
2e2fe00dbd
Comma, no space?
2019-10-18 11:31:25 +01:00
j.laycock
248f3be6e3
using a space instead of a comma to hopefully fix the build!
2019-10-18 11:23:02 +01:00
j.laycock
2c455841ea
Make volumeCreate more consistent between all 3 implementations.
2019-10-18 11:01:45 +01:00
j.laycock
b0ddad6889
Fix volume_create breaking the build
2019-10-18 10:32:07 +01:00
joeslay
d53aee179b
Merge pull request #10 from chrislusf/master
...
merge seaweed master
2019-10-14 16:03:40 +01:00
zhangsong
65f1095a54
fix benchmark cmd usage info
2019-10-14 20:23:32 +08:00
Chris Lu
50e885da45
1.44
2019-10-11 23:56:58 -07:00
Chris Lu
7a707617e5
follow the redirected master
...
fix https://github.com/chrislusf/seaweedfs/issues/1070
2019-10-09 21:00:40 -07:00
Chris Lu
87e5a02a99
fix logic error
2019-10-09 00:07:18 -07:00
Chris Lu
939e4b57a8
go fmt
2019-10-09 00:03:18 -07:00
Chris Lu
09874f0d16
volume: return error if superblock is not initialized
...
fix https://github.com/chrislusf/seaweedfs/issues/1079
2019-10-09 00:02:50 -07:00
Chris Lu
f8d4b7d1c0
support basic json filtering and selection
2019-10-06 22:35:08 -07:00
Chris Lu
e26670c67a
add some basic sql types
...
copied some code from vitness
2019-10-06 22:35:08 -07:00
Chris Lu
cf47f657af
scaffold for volume server query feature
2019-10-06 22:35:08 -07:00
joeslay
9d03aa6a38
Merge pull request #8 from chrislusf/master
...
merge seaweed master
2019-10-01 17:55:31 +01:00
j.laycock
eb27c2b037
Make releaseMemory private and return byte array instead, fix other platform compilation issues, reduce in-memory chunk size.
2019-10-01 12:21:44 +01:00
Lei Liu
2cb348a2c2
s3 api: fix listbucket common_prefixes issue
...
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-09-29 21:15:40 +08:00
Lei Liu
3cc084269c
master api: return http 404 when volumeId not exist
...
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-09-29 14:23:39 +08:00
j.laycock
d5f5acb734
limit locking physical memory to 80% of max physical memory
2019-09-20 12:44:29 +01:00
joeslay
48b093b3d2
Merge pull request #7 from chrislusf/master
...
merge seaweed master
2019-09-19 12:00:05 +01:00
j.laycock
476140fd6b
minor change to setProcessWorkingSetSize function
2019-09-19 11:59:00 +01:00
Chris Lu
972e881d48
add more logs
...
add more logs
2019-09-16 09:48:30 -07:00
Chris Lu
20ae559437
add logs
...
add logs for https://github.com/chrislusf/seaweedfs/issues/1064
2019-09-16 08:19:10 -07:00
Chris Lu
ad3efbb197
tweaking data types
2019-09-14 01:21:51 -07:00
Chris Lu
b0e4771135
Merge pull request #1060 from divinerapier/master
...
fix: non-thread-safe rand will panic
2019-09-14 01:06:14 -07:00
divinerapier
bb31462b52
fix: thread unsafe
...
Signed-off-by: divinerapier <poriter.coco@gmail.com>
2019-09-13 20:06:02 +08:00
j.laycock
6fc6322c90
Change joeslay paths to chrislusf paths
2019-09-12 14:18:21 +01:00
joeslay
d8c34b032f
Merge pull request #6 from chrislusf/master
...
merge seaweed master
2019-09-12 11:44:09 +01:00
Chris Lu
ae53f63680
filer: recursive deletion optionally ignoring any errors
...
fix https://github.com/chrislusf/seaweedfs/issues/1062
2019-09-11 20:26:20 -07:00
Chris Lu
5e9c65469e
volume: skip readonly checking when Destroy()
...
fix https://github.com/chrislusf/seaweedfs/issues/1063
2019-09-11 09:45:09 -07:00
j.laycock
c1a928887a
Add some comments/documentation
2019-09-11 10:42:37 +01:00
joeslay
9ea9a5e97a
Merge pull request #5 from chrislusf/master
...
merge seaweed master
2019-09-11 10:25:27 +01:00
j.laycock
15a4c91c28
Remove Finalizer
2019-09-10 17:05:54 +01:00
j.laycock
9c9dff7386
Virtual lock memory pages and Set Process Working set size to hint to windows harder not to write pages to disk as much, add finalize function, minor renaming
2019-09-10 16:18:51 +01:00
Chris Lu
02710f9624
refactoring: remove unused functions
2019-09-09 10:03:41 -07:00
Chris Lu
0f861d23a2
avoid nil needle map
...
fix https://github.com/chrislusf/seaweedfs/issues/1061
2019-09-09 09:48:08 -07:00
joeslay
d757ed2fe3
Merge pull request #4 from chrislusf/master
...
merge seaweed master
2019-09-06 10:49:27 +01:00
Chris Lu
803345e6af
remove glide
2019-09-06 00:15:59 -07:00
j.laycock
63acc8c972
rename mem_buffer to mBuffer
2019-09-05 13:46:29 +01:00
j.laycock
44ae041e80
rename more memorymapped variables to memoryMapMaxSizeMB
2019-09-05 13:42:23 +01:00
j.laycock
5885ab67b3
rename volume property MemoryMap to MemoryMapMaxSizeMB
2019-09-05 11:56:48 +01:00
j.laycock
5a6dfb05fc
Only use truncate flag when creating non memory map files, rename and use memory map size for reserving max file size
2019-09-05 11:46:12 +01:00
j.laycock
f6146d6ad2
use 0 instead of false for max memory flag
2019-09-05 11:43:18 +01:00
j.laycock
4257582db5
Allocate in 16MB chunks, make creation of memory maps always aligned to 16MB chunks
2019-09-05 11:15:01 +01:00
j.laycock
84f2dc3b53
Remove test code
2019-09-05 11:14:02 +01:00
Chris Lu
20d90dea5a
filer: avoid hard-coded upload timeout
2019-09-05 01:26:59 -07:00
Tom Maxwell
4a878c0006
Changed the InMemory bool to a uint32 so that it can be used to alter how much space to reserve
2019-09-04 15:27:14 +01:00
joeslay
6ee65356e3
Merge pull request #3 from chrislusf/master
...
merge seaweed master
2019-09-04 14:58:04 +01:00
j.laycock
cc756ddd79
Rename End_of_file variable, implement platform indepenent memory_map.go
2019-09-04 14:24:17 +01:00
j.laycock
9aa0859697
Remove inmemory from vaccum stuff
2019-09-03 18:20:45 +01:00
j.laycock
de5a7e60a2
remove temp hack
2019-09-03 18:20:09 +01:00
j.laycock
cbd0a98fa1
Add InMemory to Volume Grow structure
2019-09-03 18:19:02 +01:00
Chris Lu
ee8bcb0c12
remove println
2019-09-03 09:22:56 -07:00
j.laycock
1f01eb78e8
Rename mem_map to mMap, remove some in_memory variables being passed around, added MemoryMapped member to volume struct
2019-09-03 17:00:59 +01:00
Tom Maxwell
d637d86d22
Changes to try and pass the URL parameters through - in memory flag not working still
2019-09-03 15:41:28 +01:00
j.laycock
9a459d984b
Do not vacuum memory mapped files
2019-09-03 15:05:43 +01:00
Chris Lu
89a0cb2e11
1.43
2019-09-03 01:31:58 -07:00
j.laycock
0123c7a898
Change to max 2GB for now
2019-09-02 17:46:22 +01:00
joeslay
e91139348e
Merge pull request #2 from chrislusf/master
...
merge seaweed master
2019-09-02 11:39:40 +01:00
j.laycock
595a1beff0
Swap imports to use joeslay
2019-09-02 11:28:40 +01:00
Chris Lu
60c9215a00
HCFS: avoid verbose file not found exception stack trace
2019-09-01 21:40:26 -07:00
j.laycock
1e62a2b233
Fix memory not being unmapped
2019-08-30 16:55:33 +01:00
j.laycock
840ccdc35d
Refactor to pass memory maps by reference instead of value, fix memory maps not being created properly or written to properly
2019-08-30 12:30:00 +01:00
j.laycock
dc50701e7d
Overload createFile function, expose File in memory_map struct, delete from memory_map map on delete
2019-08-30 12:30:00 +01:00
j.laycock
523f3a12b3
use Os.File/uintptr instead of windows.Handle in memory map struct
2019-08-30 12:30:00 +01:00
j.laycock
0e5d3b1a70
Move memory_map_windows to new folder, intercept most of the read and write calls to volume dat files
2019-08-30 12:30:00 +01:00
Chris Lu
58168a8c52
volume: truncate dat files when creating
2019-08-26 13:01:57 -07:00
joeslay
36bde84000
Merge pull request #1 from chrislusf/master
...
merge changes from chrislusf/seaweed master
2019-08-23 17:03:37 +01:00
j.laycock
1e89e719b3
Add windows memory map functions to storage package
2019-08-23 16:19:00 +01:00
Chris Lu
35d471a818
Revert "use std encoding lib"
2019-08-23 07:59:09 -07:00
Chris Lu
fa160e00f3
Merge pull request #1051 from xushuxun/master
...
volume: read dat files' last modified time correctly
2019-08-20 22:25:37 -07:00
xushuxun
9af9f36b89
volume: read dat files' last modified time no matter dat files are catWrite or readonly
2019-08-21 11:26:01 +08:00
divinerapier
e798e4e9c5
use std encoding lib
...
Signed-off-by: divinerapier <poriter.coco@gmail.com>
2019-08-19 18:07:54 +08:00
xushuxun
5904d78bd4
weed backup: add ttl and replication parameter
2019-08-16 11:05:22 +08:00
Chris Lu
f9c1cd7370
avoid dead lock from dataFileAccessLock
...
fix https://github.com/chrislusf/seaweedfs/issues/1043
2019-08-15 09:24:54 -07:00
xushuxun
46ae35cd7e
wdclient: clear client cache when failed to connect to master leader
2019-08-15 15:13:16 +08:00
Chris Lu
d829df4f59
volume: protect against nil needle map
...
fix @mastak reported nil problem in https://github.com/chrislusf/seaweedfs/issues/1037
2019-08-14 01:08:01 -07:00
Chris Lu
e40634e6b4
volume: fail the volume deletion if compaction is in progress
...
fix https://github.com/chrislusf/seaweedfs/issues/1035
2019-08-12 00:53:50 -07:00
Chris Lu
6f75df8660
server: fix starting default master list
2019-08-10 03:10:03 -07:00
Chris Lu
0dc43ecdca
skip writing content if not modified
...
related to https://github.com/chrislusf/seaweedfs/issues/1036
2019-08-09 14:38:54 -07:00
Chris Lu
b9f6236f8a
filer: fix typo
...
fix https://github.com/chrislusf/seaweedfs/issues/1031
2019-08-06 00:42:19 -07:00
Chris Lu
c79274f7eb
notes on supported meta data store
2019-08-05 16:45:21 -07:00
Feng Ye
fc0498798b
Add etcd filestore
2019-08-01 10:16:45 +08:00
Chris Lu
ca2b81a944
fix test error
2019-07-31 02:09:13 -07:00
Chris Lu
19360ab10a
master: redirect clients to the new leader
2019-07-31 01:54:42 -07:00
Chris Lu
8afd8d35b3
master: followers can also lookup and redirect
...
improve scalability
2019-07-28 03:58:13 -07:00
Chris Lu
2c6cf72e73
refactoring
2019-07-28 01:55:05 -07:00
Chris Lu
fb90d63589
shell: Added minor changes to fs.meta.save
...
* added an option to set the output file name
* added an option to see verbose output, default to non-verbose mode
related to: https://github.com/chrislusf/seaweedfs/issues/1019
2019-07-26 09:35:22 -07:00
divinerapier
9e3b77c3ad
avoid double warpping
...
Signed-off-by: divinerapier <poriter.coco@gmail.com>
2019-07-24 16:41:45 +08:00
Chris Lu
344ce90315
remove weedfuse
2019-07-24 00:42:12 -07:00
Chris Lu
5956dfd08d
mount: add umask option
...
related to https://github.com/chrislusf/seaweedfs/issues/978
2019-07-24 00:03:05 -07:00
Chris Lu
3f851feb59
1.42
2019-07-21 21:52:48 -07:00
Chris Lu
862c2cb8e6
go fmt
2019-07-21 21:51:38 -07:00
Chris Lu
79762385bd
master: ensure only one exclusive vacuum process
...
fix https://github.com/chrislusf/seaweedfs/issues/1011
2019-07-21 21:49:10 -07:00
Chris Lu
f3b99cbfe5
fix wrong alignment fix
2019-07-21 13:50:42 -07:00
Chris Lu
644c5c27b4
refactoring
2019-07-21 13:50:24 -07:00
Chris Lu
33b3366043
volume: avoid compacting wait group
2019-07-21 13:49:59 -07:00
Chris Lu
4b15c8f0c4
volume: lock writables changes
2019-07-21 13:49:09 -07:00
Chris Lu
fd16adbde1
stricter checking before writes
2019-07-21 11:21:30 -07:00
Chris Lu
5a135fef87
purge old code
2019-07-21 10:45:36 -07:00
Chris Lu
c54d9221b9
write requests also checks cookie if overwrites
...
protect against edge cases, avoid https://github.com/chrislusf/seaweedfs/issues/1014
2019-07-17 23:57:34 -07:00
Chris Lu
898d943b25
refactoring
2019-07-17 23:43:48 -07:00
Chris Lu
0264a7f50a
set the max file key when ever possible
2019-07-17 23:23:01 -07:00
Chris Lu
c33f423955
stop early if compaction fails
...
fix https://github.com/chrislusf/seaweedfs/issues/1015
2019-07-17 23:22:01 -07:00
Chris Lu
aff911c00d
skip all azuresb
2019-07-17 01:24:20 -07:00
Chris Lu
0ea98dc6a2
disable azure due to api changes
2019-07-17 00:39:36 -07:00
Chris Lu
a7b1b23c58
fix wrong volume count
...
fix https://github.com/chrislusf/seaweedfs/issues/1013
2019-07-17 00:03:15 -07:00
Chris Lu
9e47c4749a
shell: add fs.mv
...
fix https://github.com/chrislusf/seaweedfs/issues/954
2019-07-11 09:29:04 -07:00
Chris Lu
1d1b355f9e
filer: set file name when uploading to a directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1005
2019-07-11 04:20:03 -07:00
Chris Lu
61b7a650f4
filer.copy default to 32MB chunks
2019-07-10 21:48:27 -07:00
Chris Lu
476a7c269c
detect a filer path is a file or a directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1004
2019-07-10 12:11:18 -07:00
Chris Lu
1e2127e261
s3: adding Location for multipart upload
...
fix https://github.com/chrislusf/seaweedfs/issues/1002
2019-07-08 12:37:45 -07:00
Chris Lu
62843991f2
s3: adjust object key
2019-07-08 12:37:20 -07:00
Chris Lu
cf2804eebd
1.41
2019-07-07 20:40:20 -07:00
Chris Lu
7872fc38ea
filer leveldb2: correct directory listing
...
fix https://github.com/chrislusf/seaweedfs/issues/1001
2019-07-07 20:37:48 -07:00
Chris Lu
fe2882ce7d
fix test
2019-07-02 21:42:06 -07:00
Chris Lu
36250154cc
remove println
2019-07-02 21:33:23 -07:00
Chris Lu
4d579f7f57
adjust
2019-07-02 21:28:51 -07:00
Chris Lu
23d36dfbaf
1.40
...
enable azure replication
2019-07-02 21:27:05 -07:00
Chris Lu
dcc331cf5a
parallelize leveldb
2019-07-02 21:25:53 -07:00
Chris Lu
88fed85bba
temp skipping azure
...
can not release due to https://github.com/Azure/azure-pipeline-go/issues/23
2019-06-30 23:17:45 -07:00
Chris Lu
ce5aab66e9
add versions
2019-06-30 22:43:10 -07:00
Chris Lu
36022f9416
1.39
2019-06-30 00:50:17 -07:00
Chris Lu
15ecf4e3bf
filer: default to leveldb2
2019-06-30 00:44:57 -07:00
Chris Lu
ee933cd859
filer: add metrics for go memory, gc stats
2019-06-29 03:02:28 -07:00
Chris Lu
92c7f7e069
fix compilation error
2019-06-28 11:19:08 -07:00
Chris Lu
9f3f2f7c79
protect locations slice
...
fix https://github.com/chrislusf/seaweedfs/issues/995
2019-06-28 09:47:29 -07:00
Chris Lu
327336ecf3
filer: avoid concurrent modification to result slice
...
fix https://github.com/chrislusf/seaweedfs/issues/972
2019-06-28 01:12:41 -07:00
Chris Lu
a3d1296ed9
go fmt
2019-06-27 12:18:59 -07:00
Chris Lu
ec75b2d761
volume: fix bug with 8000GB version if using in memory index
...
fix https://github.com/chrislusf/seaweedfs/issues/994
2019-06-27 12:18:45 -07:00
Chris Lu
6883f9e322
mark volume readonly before ec encoding
2019-06-26 23:02:22 -07:00
Chris Lu
c2960c7a86
defaulting file ttl to volume ttl
...
fix https://github.com/chrislusf/seaweedfs/issues/992
2019-06-26 11:18:23 -07:00
Lei Liu
1d9b75b536
weed.go: remove unused parameter
...
Signed-off-by: Lei Liu <liul.stone@gmail.com>
2019-06-26 10:46:32 +08:00
Chris Lu
359563e95a
refactoring
2019-06-25 09:49:27 -07:00
Chris Lu
fdff927b37
1.38
2019-06-24 00:36:46 -07:00
Chris Lu
a8b4fe6eae
adjust help text
2019-06-24 00:26:03 -07:00
Chris Lu
c777102da9
go fmt
2019-06-23 15:30:16 -07:00
Chris Lu
d5560f2705
simplify metrics settings
2019-06-23 15:29:49 -07:00
Chris Lu
6f8b335007
fix duplicated flags
2019-06-23 03:11:21 -07:00
Chris Lu
f16375621f
big refactoring
2019-06-23 03:08:27 -07:00
Chris Lu
6c01fb6d2d
go fmt
2019-06-23 01:57:51 -07:00
Chris Lu
8da5d5b094
filer.copy: use filer settings, avoid unnecessary command line options
...
fix https://github.com/chrislusf/seaweedfs/issues/968
2019-06-23 01:57:35 -07:00
Chris Lu
d1cd8f8c5b
add metrics, refactoring
2019-06-22 22:53:52 -07:00
Chris Lu
a14bd31f5b
go fmt
2019-06-22 20:05:25 -07:00
Chris Lu
3fa1f150d9
refactoring
2019-06-22 20:04:56 -07:00
Chris Lu
1babec00e7
check deleted chunks faster
2019-06-22 13:22:22 -07:00
Chris Lu
cd45ab072a
fix compilation error
2019-06-22 12:30:08 -07:00
Chris Lu
c369e5a13b
add metrics for filer store
2019-06-22 12:23:25 -07:00
Chris Lu
6bc3dee5b3
refactoring
2019-06-22 10:56:54 -07:00
Chris Lu
95ad56d99d
1.37
2019-06-21 20:56:34 -07:00
Chris Lu
a111f26fe6
avoid nil
...
fix https://github.com/chrislusf/seaweedfs/issues/988
2019-06-21 20:56:27 -07:00
Chris Lu
308102f023
fix cassandra creation problem
...
fix https://github.com/chrislusf/seaweedfs/issues/986
2019-06-21 17:10:38 -07:00
Chris Lu
fe4eb6bffa
1.36
2019-06-21 13:11:40 -07:00
Chris Lu
059ef879a8
fix issue 986
...
fix issue 986
2019-06-21 13:06:04 -07:00
Chris Lu
ff97acae62
more logs
2019-06-21 12:14:40 -07:00
Chris Lu
d0ce5b020b
logging
2019-06-21 12:05:00 -07:00
Chris Lu
78994024ff
adjust logging
2019-06-21 11:46:12 -07:00
Chris Lu
f88a8bda7b
ec deletion works
2019-06-21 01:14:10 -07:00
Chris Lu
613a2e8060
delete garbage only when successful
2019-06-20 23:46:00 -07:00
Chris Lu
11be0b5e91
more logging
2019-06-20 23:45:30 -07:00
Chris Lu
4b0f084b5d
delete chunks only when file writing is successful
2019-06-20 21:58:35 -07:00
Chris Lu
eebc44cce3
refactoring
2019-06-20 21:57:17 -07:00
Chris Lu
3106065b58
fix startup problem with metrics
2019-06-20 09:56:49 -07:00
Chris Lu
c74dc2b306
go fmt
2019-06-20 00:55:52 -07:00
Chris Lu
e63317fb08
ec deletion code complete, not tested yet
2019-06-20 00:55:30 -07:00
Chris Lu
4cea8aefd0
add grpc VolumeEcBlobDelete
2019-06-20 00:17:11 -07:00
Chris Lu
856da7aae2
ec volume support deletes
2019-06-19 22:57:14 -07:00
Chris Lu
115558e5f5
adjust counters
2019-06-17 21:02:50 -07:00
Chris Lu
d8ed73926d
volume servers get metrics address and interval from the master
2019-06-17 14:51:47 -07:00
Chris Lu
68d1bef236
instance format
2019-06-16 21:57:32 -07:00
Chris Lu
935639b908
metrics: disk size for <collection, normal/EC>
2019-06-16 21:56:41 -07:00
Chris Lu
0fdb1e705d
collect volume disk usage metrics
2019-06-16 02:44:20 -07:00
Chris Lu
289fd7eb39
count number of volumes and ec shards
2019-06-16 02:24:15 -07:00
Chris Lu
ff6a6dd11e
refactoring
2019-06-15 21:46:55 -07:00
Chris Lu
8b43679ae3
refactoring
2019-06-15 12:21:44 -07:00
Chris Lu
5336008dcd
refactoring
2019-06-14 13:06:01 -07:00
Chris Lu
d67189921f
go fmt
2019-06-14 00:55:12 -07:00
Chris Lu
5f6c9825f8
volume server: adds basic metrics
2019-06-14 00:54:56 -07:00
Chris Lu
a11525fe4e
filer: adds basic metrics pushing to Prometheus gateway
2019-06-13 02:01:54 -07:00
Chris Lu
be3f1f84a2
1.35
2019-06-10 21:34:06 -07:00
Chris Lu
57092d69ac
go fmt
2019-06-10 21:33:32 -07:00