Chris Lu
4a5e381feb
fail fast if filer configured wrong masters
...
fix https://github.com/chrislusf/seaweedfs/issues/1257
2020-04-03 00:47:33 -07:00
Chris Lu
aaf5e658cc
adjust parameter name
2020-04-03 00:41:05 -07:00
Chris Lu
6c96e51414
re-ordering
2020-04-03 00:40:54 -07:00
Chris Lu
d2f8e50459
make error a little bit more clearer
...
fix https://github.com/chrislusf/seaweedfs/issues/1259
2020-04-03 00:24:09 -07:00
Chris Lu
5f881d2fd5
mount: fail early rather than after mounted the directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1258
2020-04-03 00:21:03 -07:00
Chris Lu
fee3319cbf
handle gzip for image resizing
2020-04-02 23:44:24 -07:00
Chris Lu
73bc286377
volume: passing ext info via URL
2020-04-02 21:23:15 -07:00
Chris Lu
cea52a4faf
volume copying adds cleaning up on error
...
fix https://github.com/chrislusf/seaweedfs/issues/1253
2020-04-02 15:36:55 -07:00
Chris Lu
c981975627
volume.fix.replication: add test cases for complicated moving
...
fix https://github.com/chrislusf/seaweedfs/issues/1253
2020-04-02 02:16:16 -07:00
Chris Lu
ae3d08883f
volume.fsck: fix for erasure encoded volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/1254
2020-04-01 22:10:13 -07:00
Chris Lu
c446438ca5
Merge pull request #1255 from levenlabs/ignore
...
Added treat_replication_as_minimums master toml option
2020-04-01 16:48:58 -07:00
James Hartig
eae3f27c80
Added treat_replication_as_minimums master toml option
2020-04-01 19:08:48 -04:00
Chris Lu
2fc2eb74dd
refactoring
2020-03-30 22:57:45 -07:00
Chris Lu
c22094d340
adjust logging
2020-03-30 13:03:43 -07:00
Chris Lu
50a5018b7f
writing meta logs is working
2020-03-30 01:19:33 -07:00
Chris Lu
9dc0b1df8f
refactoring to compile for windows OS
...
windows os does not like to work with fuse
2020-03-29 21:07:55 -07:00
Chris Lu
be6c2f8511
1.70
2020-03-29 13:40:55 -07:00
Chris Lu
54768d0761
fix: delete a file and then create a directory with the same name
2020-03-29 01:39:48 -07:00
Chris Lu
057722bbf4
return part of the chunk if chunkview is not the full chunk
2020-03-29 00:54:39 -07:00
Chris Lu
af6d9976ad
adjust print out
2020-03-28 14:07:25 -07:00
Chris Lu
a75d50bbb8
FUSE: add configurable in memory chunk cache size
2020-03-28 14:07:16 -07:00
Chris Lu
826bc0b7e3
FUSE: add chunk cache for recently accessed file chunks
2020-03-28 13:43:31 -07:00
Chris Lu
4aa82c95e6
refactor
2020-03-28 13:42:35 -07:00
Chris Lu
eedd33dda3
refactoring
2020-03-28 13:41:58 -07:00
Chris Lu
005b4ab3fe
1.69
2020-03-27 04:51:31 -07:00
Chris Lu
d1439c5bd3
fix FUSE read for large files
...
FUSE expects ReadAt do not return partial filled buffer with a nil error.
2020-03-27 04:50:51 -07:00
Chris Lu
f06ca04451
avoid overflow
2020-03-27 04:35:31 -07:00
Chris Lu
e1911760a7
refactoring
2020-03-27 00:30:55 -07:00
Chris Lu
2f5ed29c37
clean up
2020-03-27 00:30:39 -07:00
Chris Lu
2286eda575
sampling whether the data can be gzipped
2020-03-26 23:50:48 -07:00
Chris Lu
5463560912
small test
2020-03-26 23:50:07 -07:00
Chris Lu
34a0d424f8
FUSE mount: report max volume count available disk space
2020-03-26 23:06:21 -07:00
Chris Lu
a5ce359999
refactoring
2020-03-26 22:46:49 -07:00
Chris Lu
0a05052b5c
1.68
2020-03-26 22:22:36 -07:00
Chris Lu
e7e8691720
add some tests
2020-03-26 22:21:52 -07:00
Chris Lu
86398d1efe
S3 API: set deletion to recursive
...
fix https://github.com/chrislusf/seaweedfs/issues/1251
2020-03-26 11:05:54 -07:00
Chris Lu
1fef598347
set target Dir object when moving
2020-03-26 10:56:40 -07:00
Chris Lu
9e2e07b238
tweak
2020-03-26 10:56:18 -07:00
Chris Lu
cbce793753
rename: set new name for renamed files
2020-03-26 03:30:23 -07:00
Chris Lu
c763a492f5
refactor
2020-03-26 03:30:02 -07:00
Chris Lu
712720864d
error checking for remove
2020-03-26 02:21:30 -07:00
Chris Lu
8d9a76005d
add back running output
2020-03-26 02:14:48 -07:00
Chris Lu
dd5b582d05
go fmt
2020-03-26 00:09:01 -07:00
Chris Lu
7f0d87b206
tree structured fs cache
...
FsCache for FsNode, wrapping fs.Node
2020-03-26 00:08:14 -07:00
Chris Lu
2e4fadd10a
fix fscache move
2020-03-25 22:19:19 -07:00
Chris Lu
966df2ae27
purging skip EC volumes
2020-03-25 02:41:22 -07:00
Chris Lu
6d87a0c5f5
batch delete orphan data by file keys
2020-03-25 02:21:15 -07:00
Chris Lu
5ebc95b69b
refactoring
2020-03-25 02:20:19 -07:00
Chris Lu
d79f0a76b1
format output
2020-03-25 00:56:47 -07:00
Chris Lu
d9b56ed88b
shell: volume.fsck add options
2020-03-24 22:38:36 -07:00
Chris Lu
e63a79ade8
better handle lock in case of exception
2020-03-24 18:41:25 -07:00
Chris Lu
4d5554b16f
fix compilation on travis
2020-03-24 18:40:52 -07:00
Chris Lu
d83e648c0b
auto add prefix "fs."
2020-03-24 02:40:51 -07:00
Chris Lu
a875f67a08
fix pwd printout
2020-03-24 02:40:29 -07:00
Chris Lu
00d3f01876
better output format
2020-03-24 02:34:28 -07:00
Chris Lu
0820935290
clean up
2020-03-24 02:20:37 -07:00
Chris Lu
49440cbd2f
shell: add volume.fsck
...
volume.fsck can find out how much space is not used by the filer
2020-03-24 02:18:13 -07:00
Chris Lu
0f10833e0b
fix compilation
2020-03-24 00:16:12 -07:00
Chris Lu
3794f2721f
fix logic
2020-03-24 00:09:12 -07:00
Chris Lu
443b229ef7
disable chunk file output
2020-03-24 00:08:02 -07:00
Chris Lu
bb9b97e2b4
add comments
2020-03-23 23:07:11 -07:00
Chris Lu
8047ec2f51
shell: fs.meta.save add option to export all fileIds for all files
2020-03-23 22:54:46 -07:00
Chris Lu
782d776d2a
refactoring
2020-03-23 22:54:02 -07:00
Chris Lu
38e73463f1
fix du block and byte couting
2020-03-23 21:37:04 -07:00
Chris Lu
b51fa81f0e
fix directory checking
2020-03-23 21:36:39 -07:00
Chris Lu
e666aeece2
simplify parsing filer host and port
2020-03-23 21:26:15 -07:00
Chris Lu
d151185b7e
shell: desupport filer url in the arguments
2020-03-23 20:46:17 -07:00
Chris Lu
40601953bf
comments
2020-03-23 02:54:49 -07:00
Chris Lu
daeb18be30
scaffold: fix master.toml typo
...
fix https://github.com/chrislusf/seaweedfs/issues/1247
2020-03-23 01:45:43 -07:00
Chris Lu
b97768c51c
refactoring
2020-03-23 01:30:22 -07:00
Chris Lu
45ee3736aa
refactoring
2020-03-23 01:25:38 -07:00
Chris Lu
cddcb5cafe
same booleans as before
2020-03-23 01:25:23 -07:00
Chris Lu
f84c288852
purge
2020-03-23 01:14:55 -07:00
Chris Lu
a5aa8be19b
Update webdav_server.go
2020-03-23 01:14:42 -07:00
Chris Lu
7f1e3c843d
refactoring
2020-03-23 01:14:21 -07:00
Chris Lu
c07bcd5065
refactoring
2020-03-23 00:30:02 -07:00
Chris Lu
654a69ff52
refactoring
2020-03-23 00:06:24 -07:00
Chris Lu
c0f0fdb3ba
refactoring
2020-03-23 00:01:34 -07:00
Chris Lu
fbca6b29bd
refactoring
2020-03-22 23:52:55 -07:00
Chris Lu
e39e78ea8d
remove println
2020-03-22 18:37:12 -07:00
Chris Lu
c16dc57a58
fix max volume count reporting
2020-03-22 18:33:19 -07:00
Chris Lu
d6412f27a0
logging
2020-03-22 18:32:56 -07:00
Chris Lu
35208711e5
logging
2020-03-22 18:32:49 -07:00
Chris Lu
e32999108a
add auto configured volume max count help messge
2020-03-22 16:27:15 -07:00
Chris Lu
3137777d83
volume: automatically detect max volume count
2020-03-22 16:21:42 -07:00
Chris Lu
0bf148f49d
logging
2020-03-22 16:19:00 -07:00
Chris Lu
bda13ed593
commented prints
2020-03-22 13:09:51 -07:00
Chris Lu
74e23e8d64
FUSE mount: rename correctly across folders
2020-03-22 13:09:33 -07:00
Chris Lu
64000f5c45
FUSE mount: rename under root
...
fix https://github.com/chrislusf/seaweedfs/issues/1242
2020-03-22 13:09:16 -07:00
Chris Lu
9f2c9b666b
adjust log level
2020-03-22 01:39:08 -07:00
Chris Lu
ae2ee379c0
consistent 64bit size
2020-03-22 01:37:46 -07:00
Chris Lu
2bdd936fb6
purge code
2020-03-22 01:33:40 -07:00
Chris Lu
e93e986727
webdav: stream read
2020-03-22 01:30:27 -07:00
Chris Lu
65d2ea9fb0
FUSE mount: stream read data with buffer
...
fix https://github.com/chrislusf/seaweedfs/issues/1244
2020-03-22 01:00:36 -07:00
Chris Lu
82bfad5b86
refactoring
2020-03-21 23:48:11 -07:00
Chris Lu
6ff9e2835e
filer: skip resizing image if width or height larger than original image
...
fix https://github.com/chrislusf/seaweedfs/issues/1239#issuecomment-602140779
2020-03-21 22:16:00 -07:00
Chris Lu
7c111f7b75
FUSE mount: make "nonempty" optional
...
https://github.com/chrislusf/seaweedfs/issues/1094
2020-03-21 19:14:25 -07:00
Chris Lu
3b3e063f25
1.67
2020-03-20 23:42:49 -07:00
Chris Lu
d3e4a31058
filer: fix where deletion can miss under high concurrency
2020-03-20 23:39:32 -07:00
Chris Lu
cbfe31a9a8
idx file sync before compaction
2020-03-20 23:38:46 -07:00
Chris Lu
3505b06023
report deletion error in the log
2020-03-20 23:38:24 -07:00
Chris Lu
4bc19f1e52
master: fix connections count reporting
2020-03-20 22:41:37 -07:00
Chris Lu
e1b5ac5c04
1.66
2020-03-20 20:39:48 -07:00
Chris Lu
308688c8d0
filer: add back image resizing capability
2020-03-20 20:31:11 -07:00
Chris Lu
f251d03673
refactoring
2020-03-20 15:46:16 -07:00
Chris Lu
11372dd548
refactoring
2020-03-20 15:38:04 -07:00
Chris Lu
c4bea45099
S3 API: fix DeleteMultipleObjectsHandler
...
fix https://github.com/chrislusf/seaweedfs/issues/1241
2020-03-20 14:17:31 -07:00
Chris Lu
165b0d22a4
1.65
2020-03-20 00:05:42 -07:00
Chris Lu
81797a059a
volume: sync volume file right before compaction
...
fix https://github.com/chrislusf/seaweedfs/issues/1237
2020-03-19 23:54:52 -07:00
Chris Lu
709f231e23
tiered storage: add s3 endpoint for private s3 implementation
...
fix https://github.com/chrislusf/seaweedfs/issues/1238
2020-03-19 21:13:56 -07:00
Chris Lu
d848d08944
use public url also for deletions
2020-03-18 11:16:45 -07:00
Chris Lu
6566c8e114
weed upload: add usePublicUrl option
...
fix https://github.com/chrislusf/seaweedfs/issues/1236
2020-03-18 10:50:53 -07:00
Chris Lu
db9854fa0a
1.64
2020-03-17 10:58:35 -07:00
Chris Lu
bec6ec7db6
go fmt
2020-03-17 10:01:55 -07:00
Chris Lu
b964bbab3d
fix compilation
2020-03-17 10:01:24 -07:00
Chris Lu
c3cb6fa1d7
volume: compaction can cause readonly volumes
...
address https://github.com/chrislusf/seaweedfs/issues/1233
2020-03-17 09:43:57 -07:00
Chris Lu
f9b3750ad1
volume: handle repeated writes response
2020-03-15 04:16:00 -07:00
Chris Lu
22400c6633
consistent gzip logic
...
local store gzip same as replicated writes
2020-03-15 04:15:40 -07:00
Chris Lu
560df51def
refactoring
2020-03-15 03:11:26 -07:00
Chris Lu
7edbee6f57
volume: proxy writes to remote volume server, with replication or not
...
the panic is triggered by uploading a file to a volume server not holding the designated replica.
2020-03-15 10:20:14.365488 I | http: panic serving 127.0.0.1:57124: runtime error: invalid memory address or nil pointer dereference
goroutine 119 [running]:
net/http.(*conn).serve.func1(0xc0001a8000)
/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:1772 +0x139
panic(0x2316fe0, 0x3662900)
/home/travis/.gimme/versions/go1.14.linux.amd64/src/runtime/panic.go:973 +0x396
github.com/chrislusf/seaweedfs/weed/topology.getWritableRemoteReplications(0xc00009c000, 0x2, 0x7ffeefbffbd2, 0xe, 0x0, 0xa, 0x0, 0x0, 0xbb4bf1f7)
/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/topology/store_replicate.go:157 +0x53
github.com/chrislusf/seaweedfs/weed/topology.ReplicatedWrite(0x7ffeefbffbd2, 0xe, 0xc00009c000, 0xc000000002, 0xc000472750, 0xc0001b2200, 0x0, 0x1, 0x0)
/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/topology/store_replicate.go:29 +0xc7
github.com/chrislusf/seaweedfs/weed/server.(*VolumeServer).PostHandler(0xc0001513f0, 0x292bde0, 0xc0001fe2a0, 0xc0001b2200)
/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/volume_server_handlers_write.go:52 +0x56f
github.com/chrislusf/seaweedfs/weed/server.(*VolumeServer).privateStoreHandler(0xc0001513f0, 0x292bde0, 0xc0001fe2a0, 0xc0001b2200)
/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/volume_server_handlers.go:37 +0x21f
net/http.HandlerFunc.ServeHTTP(0xc0004420e0, 0x292bde0, 0xc0001fe2a0, 0xc0001b2200)
/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:2012 +0x44
net/http.(*ServeMux).ServeHTTP(0xc0001fc800, 0x292bde0, 0xc0001fe2a0, 0xc0001b2200)
/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:2387 +0x1a5
net/http.serverHandler.ServeHTTP(0xc0001781c0, 0x292bde0, 0xc0001fe2a0, 0xc0001b2200)
/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:2807 +0xa3
net/http.(*conn).serve(0xc0001a8000, 0x2934420, 0xc000212400)
/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:1895 +0x86c
created by net/http.(*Server).Serve
/home/travis/.gimme/versions/go1.14.linux.amd64/src/net/http/server.go:2933 +0x35c
Eg:
server A (datacenter 1) and server B (datacenter 2) hold replica (100) for volume 1.
If you upload a file with a key 1,xxxxx to server C (datacenter 3) will trigger the panic on server C.
The server C should either proxy upload file to the correct volume server or should return an HTTP error code and not panic.
2020-03-15 02:50:42 -07:00
Chris Lu
7b37178716
filer: close stores if interrupted
2020-03-14 20:30:26 -07:00
Chris Lu
d022b6bc0e
fix compilation
2020-03-14 16:32:16 -07:00
Chris Lu
0576a27f44
protect against possible nil
...
which is unlikely to happen though
2020-03-14 16:06:06 -07:00
Chris Lu
cc52e8ca87
1.63
2020-03-14 14:25:07 -07:00
Chris Lu
ee635dcc36
refactoring
2020-03-14 14:08:00 -07:00
Chris Lu
18cda6adbb
cleaner fix
2020-03-14 13:55:32 -07:00
Chris Lu
17efa361f9
fix possible nil
2020-03-14 13:53:03 -07:00
Chris Lu
91e0a987e4
1.62
2020-03-14 01:03:59 -07:00
Chris Lu
e2e691d9c2
clean up, add test
2020-03-13 23:53:15 -07:00
Chris Lu
3cc9e85895
volume: vacuum pass preallocate variable
2020-03-13 16:17:44 -07:00
Chris Lu
4ce23e8809
fix compilation error
2020-03-13 15:51:38 -07:00
Chris Lu
c90eb0da1f
volume: handling readonly volumes after compaction
...
ensure readonly volumes are not added as writable
2020-03-13 15:41:27 -07:00
Chris Lu
e85da50ad4
remove tikv, move its support to "tikv" branch
2020-03-11 20:59:00 -07:00
Chris Lu
d439d83772
volume: follow compactionBytePerSecond
...
related to https://github.com/chrislusf/seaweedfs/issues/1108
2020-03-11 10:32:17 -07:00
Chris Lu
4237a813cc
skip printouts
2020-03-11 01:13:54 -07:00
Chris Lu
ebc739afb5
filer store postgres: skip empty parameters
...
better fix https://github.com/chrislusf/seaweedfs/issues/1227
2020-03-11 01:13:54 -07:00
HongyanShen
81610ed006
fix : #1226
2020-03-11 14:37:14 +08:00
Chris Lu
60f5f05c78
1.61
2020-03-10 13:52:49 -07:00
Chris Lu
7f3ce7b6d6
adjust tikv build env
2020-03-10 13:36:45 -07:00
Chris Lu
d3e1ef64b3
ensure error checking on windows
2020-03-10 13:33:26 -07:00
Chris Lu
88a110e67e
release file handle
2020-03-10 13:32:22 -07:00
Chris Lu
43e6261579
volume: fix memory leak
2020-03-10 13:31:23 -07:00
Chris Lu
d214cefc2e
filer: list diretory pagination make up for the expired entries
2020-03-09 23:28:01 -07:00
Chris Lu
e6de42f888
1.60
2020-03-09 22:32:40 -07:00
Chris Lu
e73b0c2457
remove not ready status
2020-03-09 22:31:14 -07:00
Chris Lu
0871d2cff0
volume: fix memory leak during compaction
...
fix https://github.com/chrislusf/seaweedfs/issues/1222
2020-03-09 22:29:02 -07:00
Chris Lu
14ae33d642
adjust logging
2020-03-09 10:36:03 -07:00
Chris Lu
89eb05b50f
filer: support TTL for all filer stores
2020-03-09 01:02:01 -07:00
Chris Lu
8a899992f2
filer: fix ttl parsing
...
fix https://github.com/chrislusf/seaweedfs/issues/1225
2020-03-09 00:16:10 -07:00
Chris Lu
c32f95c380
1.59
2020-03-08 22:02:34 -07:00
Chris Lu
3934c69757
clean up
2020-03-08 21:54:47 -07:00
Chris Lu
2e3f6ad3a9
filer: remember content is gzipped or not
2020-03-08 21:39:33 -07:00
Chris Lu
5ac6297c68
adjust parameter names
2020-03-08 17:03:37 -07:00
Chris Lu
e04c1a1361
filer: remove chunk manifest file support
...
This is not needed for filer
2020-03-08 17:03:27 -07:00
Chris Lu
1dc30214cb
mark encryptVolumeData as not ready
2020-03-08 17:02:47 -07:00
Chris Lu
9b3109a5d8
filer: processing all response headers, no pass through to volume server
...
* filer calculate MD5 etag
* filer handle response headers, instread of pass it to volume servers
2020-03-08 15:42:44 -07:00
Chris Lu
11fceaf2f7
refactoring
2020-03-07 18:09:30 -08:00
Chris Lu
1a75fbfce0
refactoring
2020-03-07 18:06:48 -08:00
Chris Lu
afb20de14c
breaks dependency loop
2020-03-07 17:01:39 -08:00
Chris Lu
8645283a7b
fuse mount: avoid lookup nil entry
...
fix https://github.com/chrislusf/seaweedfs/issues/1221
2020-03-07 16:51:46 -08:00
Chris Lu
936e7cdbfb
pass in filer.copy cipher option
2020-03-07 12:46:00 -08:00
Chris Lu
9228ff192c
encryption works
2020-03-07 11:08:57 -08:00
Chris Lu
0375ce2c2e
filer: set mime type from volume server
2020-03-07 07:25:15 -08:00
Chris Lu
1ae83c2938
go fmt
2020-03-07 06:12:57 -08:00
Chris Lu
564629444b
format
2020-03-07 06:12:20 -08:00
Chris Lu
8c3e25b38a
mimee type default "application/octet-stream" to empty string
2020-03-07 06:08:08 -08:00
Chris Lu
dba35404e4
filer: HEAD response add content type
2020-03-07 06:08:08 -08:00
Chris Lu
ea1169dc80
filer cipher: single chunk http POST and PUT and read
2020-03-07 06:08:08 -08:00
Chris Lu
e3b8bf5588
Merge pull request #1219 from song-zhang/master
...
schedule new volume by free volume number of nodes
2020-03-07 05:54:06 -08:00
zhangsong
40f70481cd
schedule new volume by free volume number of nodes
2020-03-07 21:41:42 +08:00
Chris Lu
13e215ee5c
filer: option to encrypt data on volume server
2020-03-06 00:49:47 -08:00
Chris Lu
31c481e3fc
fix typo
2020-03-05 16:00:20 -08:00
Chris Lu
465d18930b
filer redis: directory listing follow ttl
...
fix https://github.com/chrislusf/seaweedfs/issues/1217
2020-03-05 10:35:21 -08:00
Chris Lu
e031605248
purge code
2020-03-04 00:44:27 -08:00
Chris Lu
f90c43635d
refactoring
2020-03-04 00:39:47 -08:00
Chris Lu
9b7dae8037
filer: fix uploaded chunked file size reporting bug.
2020-03-02 20:27:14 -08:00
Chris Lu
7335e62199
volume: PUT also conditionally gzip compress
2020-03-01 23:37:02 -08:00
Chris Lu
410bce3925
go fmt
2020-03-01 22:39:08 -08:00
Chris Lu
dc40413847
fix error
2020-03-01 22:21:13 -08:00
Chris Lu
6a8484b4ae
master able to list all master clients by type
2020-03-01 22:13:47 -08:00
Chris Lu
0ca68a2a6d
WIP
2020-03-01 22:13:15 -08:00
Chris Lu
4532640ffd
1.58
2020-03-01 17:59:12 -08:00
Chris Lu
151114ff59
volume: fix readonly status reporting
2020-03-01 17:57:37 -08:00
Chris Lu
1df13d0d6d
adjust instructions
2020-02-27 13:15:21 -08:00
Chris Lu
f87f2045b3
delete old file first on windows
...
fix https://github.com/chrislusf/seaweedfs/issues/1210
2020-02-27 00:59:35 -08:00
Chris Lu
757c7d67ad
avoid nil DataBackend
2020-02-27 00:07:25 -08:00
Chris Lu
555413d9fc
weed queue starts
2020-02-27 00:07:13 -08:00
Chris Lu
430b5a49e2
unused code
2020-02-26 19:35:00 -08:00
Chris Lu
8c1da71402
remove dead code
2020-02-26 17:27:49 -08:00
Chris Lu
ed0acd1722
go fmt
2020-02-26 16:52:57 -08:00
Chris Lu
37a3628b2e
grpc add PermitWithoutStream to client and server options
2020-02-26 16:51:38 -08:00
Chris Lu
556dd76303
s3: wait to connect to filer
2020-02-26 16:49:47 -08:00
Chris Lu
0156e2975a
mount: add mode to run external to SeaweedFS container cluster
2020-02-26 16:46:01 -08:00
Chris Lu
543cf1c80e
fix compilation error
2020-02-25 23:59:30 -08:00
Chris Lu
b1f377f822
mount: fail fast when connecting to filer
...
fix https://github.com/chrislusf/seaweedfs/issues/1034
2020-02-25 23:51:34 -08:00
Chris Lu
36d6595658
remove ctx
2020-02-25 22:38:27 -08:00
Chris Lu
fd9612d66e
remove ctx
2020-02-25 22:37:54 -08:00
Chris Lu
86cce3eb58
fix test
2020-02-25 22:29:01 -08:00
Chris Lu
97ab8a1976
remove ctx if possible
2020-02-25 22:23:59 -08:00
Chris Lu
892e726eb9
avoid reusing context object
...
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu
bd3254b53f
adjust logging
2020-02-25 17:24:08 -08:00
Chris Lu
0841bedb15
move filer assign volume grpc errror to response
2020-02-25 17:15:09 -08:00
Chris Lu
bc38b72a20
s3: implemented DeleteMultipleObjects
2020-02-25 14:38:36 -08:00
Chris Lu
35dde56711
refactoring
2020-02-25 12:58:45 -08:00
Chris Lu
986d63cd5e
s3: increase list objects or parts limit from 1000 to 10000
...
10x of Amazon S3 limits
2020-02-25 11:45:40 -08:00
Chris Lu
7d10fdf737
fix directory lookup nil
2020-02-25 11:13:06 -08:00
Chris Lu
e86da5a491
minor
2020-02-25 00:42:48 -08:00
Chris Lu
0644d63748
shell: add commands for bucket
2020-02-24 23:30:01 -08:00
Chris Lu
6ab7368ef2
filer: dynamically create bucket under /buckets folder
2020-02-24 22:28:45 -08:00
Chris Lu
5bcb44eda9
filer: move buckets folder to filer.toml since it should not be changed often
2020-02-24 14:42:57 -08:00
Chris Lu
d8dec2323b
s3: move buckets folder configuration to filer
2020-02-24 14:34:14 -08:00
Chris Lu
b06b7ca6e6
adjust UI
2020-02-23 22:01:03 -08:00
Yoni Nakache
0d60bb4427
display bug, header is Free but data was Used
2020-02-24 07:47:27 +02:00
Chris Lu
e96f85ed8a
make pb files
2020-02-23 18:06:34 -08:00
Chris Lu
346a1cf0b9
Merge pull request #1207 from LazyDBA247-Anyvision/update-stats-and-status-volume-server
...
Update stats and status volume server
2020-02-23 18:05:25 -08:00
Chris Lu
bbb163c5ad
add for https://github.com/chrislusf/seaweedfs/pull/1207
2020-02-23 18:04:31 -08:00
LazyDBA247-Anyvision
4ff513d64d
staus route: add DiskStatuses for disk in the volume server status
...
whem monitoring server, better to know the status of the disks & volumes in a single route.
2020-02-23 23:27:09 +02:00
LazyDBA247-Anyvision
ab91dbe1d7
fillInDiskStatus: update function to calculate new stats
2020-02-23 23:25:56 +02:00
LazyDBA247-Anyvision
ec8619176a
volume server protobuf: add precentFree & percentUsage to the DiskStatus
2020-02-23 23:24:24 +02:00
Yoni Nakache
cd4c9a365b
DiskStats: adding Total & Percent Usage
...
making relevant data visible and readable.
2020-02-23 22:33:47 +02:00
Chris Lu
6ea394b802
1.57
2020-02-22 21:34:49 -08:00
Chris Lu
4ed6b584e2
s3: access control limited by bucket
2020-02-22 21:34:18 -08:00
Chris Lu
e83bfd0a35
adjust log level
2020-02-22 21:23:30 -08:00
Chris Lu
48b7ad5fa8
s3: deny anonymous type
2020-02-22 14:01:04 -08:00
Chris Lu
f55f49970f
move volume server stats to grpc
2020-02-21 21:45:03 -08:00
Chris Lu
3a1d017de2
shell: ensure dc and rack does not change for replicated volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/1203
2020-02-21 21:23:25 -08:00
Chris Lu
621cdbdf58
filer: avoid possible timeouts for updates and deletions
2020-02-20 15:44:17 -08:00
Chris Lu
45156cc2fe
add test cases for replication=011
2020-02-20 01:21:11 -08:00
Chris Lu
b07089cff2
webdav out of "unstable"
2020-02-18 19:38:59 -08:00
Chris Lu
dbabdd418e
add unhandled request auth type
...
fix
2020-02-18 11:43:57.396699 I | http: panic serving 172.28.0.43:50658: runtime error: invalid memory address or nil pointer dereference
goroutine 595 [running]:
net/http.(*conn).serve.func1(0xc0001fe3c0)
/usr/lib/go/src/net/http/server.go:1767 +0x13b
panic(0x55c4e35f3820, 0x55c4e48b3c40)
/usr/lib/go/src/runtime/panic.go:679 +0x1b6
github.com/chrislusf/seaweedfs/weed/s3api.(*IdentityAccessManagement).authRequest(0xc0004b84e0, 0xc000115900, 0xc0000bb650, 0x1, 0x1, 0x55c4e399d740)
/go/src/github.com/chrislusf/seaweedfs/weed/s3api/auth_credentials.go:143 +0x11c
github.com/chrislusf/seaweedfs/weed/s3api.(*IdentityAccessManagement).Auth.func1(0x55c4e3994c40, 0xc0007808c0, 0xc000115900)
/go/src/github.com/chrislusf/seaweedfs/weed/s3api/auth_credentials.go:111 +0x5e
net/http.HandlerFunc.ServeHTTP(0xc0004b87e0, 0x55c4e3994c40, 0xc0007808c0, 0xc000115900)
/usr/lib/go/src/net/http/server.go:2007 +0x46
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0004ba000, 0x55c4e3994c40, 0xc0007808c0, 0xc000115700)
/root/go/pkg/mod/github.com/gorilla/mux@v1.7.3/mux.go:212 +0xe4
net/http.serverHandler.ServeHTTP(0xc00011e0e0, 0x55c4e3994c40, 0xc0007808c0, 0xc000115700)
/usr/lib/go/src/net/http/server.go:2802 +0xa6
net/http.(*conn).serve(0xc0001fe3c0, 0x55c4e399d680, 0xc000894180)
/usr/lib/go/src/net/http/server.go:1890 +0x877
created by net/http.(*Server).Serve
/usr/lib/go/src/net/http/server.go:2927 +0x390
2020-02-18 09:16:04 -08:00
Chris Lu
443a68484a
1.56
2020-02-17 12:54:45 -08:00
Chris Lu
0a57db7026
Merge pull request #1200 from chrislusf/add_s3_acl
...
Add s3 ACL
2020-02-17 12:53:30 -08:00
Chris Lu
4cdde5f569
configuration stores the identity list
2020-02-17 12:31:59 -08:00
Chris Lu
9232d3ac68
ignore error when counting closed connections
2020-02-14 10:28:02 -08:00
Chris Lu
96c1ae8471
refactoring the close http response
2020-02-14 09:46:36 -08:00
Chris Lu
cf5064d702
properly close http response
2020-02-14 09:09:15 -08:00
Chris Lu
9b6296e77a
volume: add grpc file read operation
...
This is added more for performance benchmarking.
2020-02-14 00:37:32 -08:00
panyc16
7b3764fd9e
make loadExistingVolume() return bool
2020-02-13 15:19:00 +08:00
Chris Lu
62c34454d8
1.55
2020-02-11 23:00:31 -08:00
Chris Lu
83836f4299
filer: recursively batch delete file chunks
...
fix https://github.com/chrislusf/seaweedfs/issues/1197
2020-02-11 22:54:10 -08:00
Chris Lu
2a6db0fd43
volume: return 204 for unchanged file uploads
...
fix https://github.com/chrislusf/seaweedfs/issues/1196
2020-02-11 09:45:02 -08:00
Chris Lu
33b3bd467c
Revert "HEAD operation changes to fasthttp"
...
This reverts commit 58f126fd27
.
2020-02-10 13:43:53 -08:00
Chris Lu
58f126fd27
HEAD operation changes to fasthttp
2020-02-10 09:13:29 -08:00
Chris Lu
29945fad51
1.54
2020-02-09 18:07:17 -08:00
Chris Lu
b4abe3c081
unused
2020-02-09 18:02:17 -08:00
Chris Lu
f3ce3166ad
add streaming v4
2020-02-09 17:42:17 -08:00
Chris Lu
b90ad6f452
add v2 support
2020-02-09 16:02:05 -08:00
Chris Lu
e7b63d50b1
add s3.config to server.go
2020-02-09 14:31:51 -08:00
Chris Lu
9ed364f053
support acl
2020-02-09 14:30:02 -08:00
Chris Lu
8d94564f41
refactor
2020-02-04 21:16:34 -08:00
Chris Lu
c1288e9eb4
volume: sdx generation uses memdb instead of compactMap
...
fix https://github.com/chrislusf/seaweedfs/issues/1194
2020-02-04 21:12:09 -08:00
Chris Lu
08e4702542
sync before closing index file
...
merge from 0181f87d91
2020-02-04 10:38:12 -08:00
Chris Lu
3a35632d58
sync before closing file
...
merge changes from 1747fc2d52
2020-02-04 10:37:14 -08:00
Chris Lu
dc786a63ac
master: add configurable volume growth toml setting
2020-02-03 18:15:16 -08:00
Chris Lu
382ff218d3
filer: set file size, streaming chunk file uploading
...
fix https://github.com/chrislusf/seaweedfs/issues/1193
2020-02-03 17:04:06 -08:00
Chris Lu
40ae533fa3
shell: add volume.configure.replication to change replication for a volume
...
fix https://github.com/chrislusf/seaweedfs/issues/1192
2020-02-02 15:37:23 -08:00
Chris Lu
fb19263a71
fix build
2020-01-31 00:59:48 -08:00
Chris Lu
beb0a1599e
1.53
2020-01-31 00:32:09 -08:00
Chris Lu
b2743afaee
remove println
2020-01-31 00:21:18 -08:00
Chris Lu
a80ecbfe84
s3: add s3 copy
...
fix https://github.com/chrislusf/seaweedfs/issues/1190
2020-01-31 00:11:12 -08:00
Chris Lu
3b043ead49
Merge pull request #1189 from eryx67/compression-wav
...
Support wav files compression
2020-01-29 22:37:35 -08:00
Chris Lu
abf90ad7b7
remove testing for viper
2020-01-29 21:38:53 -08:00
Chris Lu
20b1fb9ab7
fix test
2020-01-29 21:24:06 -08:00
eryx67
2a87087590
support wav files compression
2020-01-30 10:11:47 +05:00
Chris Lu
8925f3305d
adjust example
2020-01-29 09:11:07 -08:00
Chris Lu
d335f04de6
support env variables to overwrite toml file
2020-01-29 09:09:55 -08:00
Chris Lu
27b94cb65b
fix wrong url
...
fix https://github.com/chrislusf/seaweedfs/issues/1187
2020-01-28 00:49:47 -08:00
Chris Lu
f724057051
reduce logs
2020-01-27 21:20:41 -08:00
Chris Lu
c8d543ef51
avoid retry delay
2020-01-27 00:58:26 -08:00
Chris Lu
081bc1ea25
mount: able to write to any part of a file
2020-01-27 00:54:52 -08:00
Chris Lu
2f6bb57979
logging
2020-01-27 00:54:21 -08:00
Chris Lu
234f69452b
1.52
2020-01-26 21:04:40 -08:00
Chris Lu
72a64a5cf8
use the same context object in order to retry
2020-01-26 14:42:11 -08:00
Chris Lu
08e4b56a8a
mount: able to handle large git clone
2020-01-26 13:01:11 -08:00
Chris Lu
19a05ad174
add test cases
2020-01-26 12:47:23 -08:00
Chris Lu
835da19c09
add logging
2020-01-26 00:50:18 -08:00
Chris Lu
c48fc8b4de
grpc send error via response instead of grpc error
2020-01-25 09:17:19 -08:00
Chris Lu
ccf3859152
log
2020-01-25 08:16:43 -08:00
Chris Lu
4433c7bdca
logs
2020-01-25 07:34:09 -08:00
Chris Lu
8cccccce9f
mount: reuse the entry object for the new directory
...
avoid mkdir and then query for the same directory
reduces these "context canceled" issues
attr read directory:"/seaweedfs/other/java/s3copier/src/main/java/com" name:"seaweedfs" : rpc error: code = Canceled desc = context canceled
2020-01-25 01:15:54 -08:00
Chris Lu
90e30c0170
log
2020-01-25 00:32:18 -08:00
Chris Lu
8a42aa8221
adjust logging
2020-01-25 00:31:53 -08:00
Chris Lu
28d1b34856
logging
2020-01-24 22:13:06 -08:00
Chris Lu
6e25acc681
add logging
2020-01-24 22:01:51 -08:00
Chris Lu
b6e6ca8595
adjust logs
2020-01-24 20:06:58 -08:00
Chris Lu
d4cde5df49
return empty response if not found
2020-01-24 18:07:34 -08:00
Chris Lu
3cdb27fafc
adjust logging
2020-01-24 17:55:39 -08:00
Chris Lu
6107509c64
adjust logging
2020-01-24 17:26:18 -08:00
Chris Lu
e6f9f5da3a
adjust logging
2020-01-24 17:26:03 -08:00
Chris Lu
24f62ebe76
minor changes
2020-01-24 10:02:53 -08:00
Chris Lu
5f1109f143
reduce log
2020-01-24 01:41:51 -08:00
Chris Lu
9d93c83654
mount: return IO error for mkdir
2020-01-24 01:41:31 -08:00
Chris Lu
107e8a56ea
retry context canceled request
2020-01-24 01:40:51 -08:00
Chris Lu
2f75264ec7
mount: avoid use uint64 for greater than zero test
2020-01-23 22:12:57 -08:00
Chris Lu
c936a12afa
mount: update the file size so file Attr() can read
...
if file Attr() during file writes, the OS, at least for Mac, may try to resend the data to write, which usually are empty.
2020-01-23 21:59:58 -08:00
Chris Lu
c2e589f202
mount: better combines connected intervals to write to volume servers
2020-01-22 23:00:04 -08:00
Chris Lu
6a5c037099
fix http range requests
2020-01-22 22:59:12 -08:00
Chris Lu
72e1aff237
add lock on dirty pages
2020-01-22 15:38:25 -08:00
Chris Lu
66ded8804a
adjust logging
2020-01-22 15:37:59 -08:00
Chris Lu
6b48d246a5
mount: read data that is just written
...
able read on data not flushed
multiple file open shares the same file handle
fix https://github.com/chrislusf/seaweedfs/issues/1182 on linux
2020-01-22 13:42:03 -08:00