mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-29 05:49:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,584 Commits 32 Branches 296 Tags 148 MiB
e8d4be579d
Commit Graph

433 Commits

Author SHA1 Message Date
Chris Lu
be0e88a606 fix chunk size limit for default 0 2018-05-28 13:44:27 -07:00
Chris Lu
5c4480ec6c add mountOptions.chunkSizeLimitMB, remove cmdMount.IsDebug 2018-05-28 13:42:25 -07:00
Chris Lu
8ab7dd9d08 weed mount add options for collection and replication 2018-05-28 13:24:48 -07:00
Chris Lu
d0b238d2db cache local writes before flushing to volume server 2018-05-28 12:30:17 -07:00
Chris Lu
07e0d13d2d filer support reading multiple chunks, with range support 2018-05-28 05:39:12 -07:00
Chris Lu
dd5661b297 adding favicon to filer also 2018-05-28 02:35:58 -07:00
Chris Lu
9b603f5ffa split filer read into 2 files 2018-05-28 02:24:14 -07:00
Chris Lu
8db9319a06 refactoring: go fmt, reorg 2018-05-27 23:59:49 -07:00
Chris Lu
480a073f1f refactoring: split into 4 files 2018-05-27 23:53:10 -07:00
Chris Lu
5b844d7011 pass reference of master server 
SendHeartbeat passes lock by value: weed_server.MasterServer contains
sync.Mutex
 2018-05-27 11:58:00 -07:00
Chris Lu
8b0718ac92 go vet 2018-05-27 11:56:49 -07:00
Chris Lu
458ada173e go fmt 2018-05-27 11:52:26 -07:00
Chris Lu
8647191bee removing filer package! 2018-05-27 11:52:15 -07:00
Chris Lu
03d76479a5 add reds 2018-05-27 11:14:29 -07:00
Chris Lu
a6a6b87f1b adjust readme 2018-05-27 11:14:12 -07:00
Chris Lu
ab86d263c9 add Cassandra 2018-05-27 00:01:15 -07:00
Chris Lu
603de2d5db skip deletion if entry not found 2018-05-27 00:00:56 -07:00
Chris Lu
f124ebab1d fix ordering 2018-05-27 00:00:11 -07:00
Chris Lu
1c0a2a7db8 reformat 2018-05-26 23:59:56 -07:00
Chris Lu
869161a261 support both mysql and postgres 2018-05-26 22:02:49 -07:00
Chris Lu
87b3b84471 simplifying dirhash to 64bit integer 2018-05-26 21:24:03 -07:00
Chris Lu
955eae3500 this works great and support long file names 2018-05-26 14:08:55 -07:00
Chris Lu
2da84ed331 working fine now 
but index length can be improved
 2018-05-26 13:35:56 -07:00
Chris Lu
68bcaff14d mysql can compile, not yet tested! 2018-05-26 05:32:15 -07:00
Chris Lu
c1e353491a refactoring variable names 2018-05-26 04:50:55 -07:00
Chris Lu
828e4a5ace fix test error 2018-05-26 04:13:50 -07:00
Chris Lu
9e77563c99 add leveldb store 
1. switch to viper for filer store configuration
2. simplify FindEntry() return values, removing “found”
3. add leveldb store
 2018-05-26 03:49:46 -07:00
Chris Lu
c34feca59c refactoring 2018-05-25 23:27:06 -07:00
Chris Lu
6de84c64c6 adding create time 2018-05-25 23:26:40 -07:00
Chris Lu
72fc001d0d remove unused code 2018-05-25 01:30:16 -07:00
Chris Lu
ac66c133a5 do not read attributes when file is opened 2018-05-25 01:27:21 -07:00
Chris Lu
6d1bcd4b8c use existing attributes instead of fetching from filer 2018-05-25 01:22:31 -07:00
Chris Lu
0a223838bd refactoring 2018-05-25 00:57:25 -07:00
Chris Lu
f8776ad5cd add breadcrumb to filer UI 2018-05-24 23:42:13 -07:00
Chris Lu
a01557f4cb adjust filer web page 2018-05-24 23:21:12 -07:00
Chris Lu
c11d84f314 fix reading from a url 2018-05-24 23:20:56 -07:00
Chris Lu
7b81cf3762 better logs 2018-05-24 23:20:26 -07:00
Chris Lu
9f4f8de9ad skip printout 2018-05-24 23:20:12 -07:00
Chris Lu
e18c7e160c avoid empty chunk view 2018-05-24 23:19:56 -07:00
Chris Lu
d773e11c7a file handler directly read from volume servers 
this mostly works fine now!

next: need to cache files to local disk
 2018-05-24 01:22:37 -07:00
Chris Lu
00d0274fd7 prepare to read from multiple file chunks 2018-05-23 22:28:54 -07:00
Chris Lu
849b6ec28d seems editing already working 
Need to handle multiple chunks read.
Need to cache local file changes.
 2018-05-23 20:55:24 -07:00
Chris Lu
d84b80b795 similar changes as master 2018-05-23 03:23:47 -07:00
Chris Lu
536559f62d copy works, edit somehow still fails 2018-05-23 03:08:46 -07:00
Chris Lu
1675243f29 maybe speed up a little when loading index 2018-05-23 02:38:11 -07:00
Chris Lu
e97c60cc10 avoid overwriting variables 2018-05-23 02:36:19 -07:00
Chris Lu
468514f525 reduce logs 2018-05-23 02:36:06 -07:00
Chris Lu
69b9d8c3c2 fix boltdb variable usage 2018-05-22 10:18:09 -07:00
Chris Lu
873868cc10 not working now 
need to add file handler
 2018-05-22 04:31:44 -07:00
Chris Lu
7362de9a18 weed mount can work well 
TODO: somehow filer url is returning empty content
 2018-05-22 03:26:38 -07:00
Chris Lu
9dd228747c filer copy added uid/gid 2018-05-21 01:25:30 -07:00
Chris Lu
f07482382b able to update file content 
having some issue when vi reports file changed.
 2018-05-21 00:00:28 -07:00
Chris Lu
cd47528a75 remove unused code 2018-05-20 17:08:54 -07:00
Chris Lu
cc66e25cd2 merge intervals 2018-05-20 17:06:09 -07:00
Chris Lu
7ca5052942 create files correctly! 2018-05-19 13:51:44 -07:00
Chris Lu
793dd81ca2 skip permission checking when creating dir or files 2018-05-19 12:40:24 -07:00
Chris Lu
ab4ddb1e0e fix directory creation, directory listing 2018-05-19 12:07:15 -07:00
Chris Lu
e31c514b00 adding modified time to file chunk 2018-05-16 00:54:44 -07:00
Chris Lu
6bf31467c7 adding empty fsync 
less error now. But still:

vi on write: E514: write error (file system full?)
cp: Input/output error
 2018-05-16 00:54:27 -07:00
Chris Lu
b303a02461 cp file can work 
1. consolidate to filer_pb.FileChunk
2. dir add file, mkdir
3. file flush, write

updates having issue
 2018-05-16 00:08:44 -07:00
Chris Lu
c7a71d35b0 fix on pagination 2018-05-14 20:27:48 -07:00
Chris Lu
58954bf46f pass file attributes from filer to mount 2018-05-14 02:02:17 -07:00
Chris Lu
67401f1908 fix gRpc nil response 2018-05-14 00:17:22 -07:00
Chris Lu
c5cf9bd290 properly working filer 2018-05-13 23:56:16 -07:00
Chris Lu
f01d5616b3 add better listing directory entries 2018-05-13 14:02:29 -07:00
Chris Lu
a4740ca836 fix compilation error for windows 2018-05-13 00:49:59 -07:00
Chris Lu
2dd5b777f7 prepare for 0.77 release 
right before changing to filer2
 2018-05-13 00:26:23 -07:00
Chris Lu
75a2b05aa3 remove Size from Attr 2018-05-13 00:11:26 -07:00
Chris Lu
fc7f81f6e7 remove nlink property from directory node 2018-05-12 22:42:28 -07:00
Chris Lu
9ecbf92569 more tests 2018-05-12 13:50:11 -07:00
Chris Lu
81af1bafba file store adds memdb implementation 2018-05-12 13:45:29 -07:00
Chris Lu
a808df5019 visit parent folder first 2018-05-11 02:27:57 -07:00
Chris Lu
8a1d640dc4 starting filer2 2018-05-11 02:20:15 -07:00
Chris Lu
9f345da20f mv filer proto to filer_pb 2018-05-09 23:18:02 -07:00
Chris Lu
942c2cbd7b mv pb to master_pb 2018-05-09 23:11:54 -07:00
Chris Lu
57c5405657 add todo 2018-05-09 04:18:31 -07:00
Chris Lu
6db1c8bafb Revert "adjust the needle mapper" 
This reverts commit e2c7be3a2e.
 2018-05-09 03:57:07 -07:00
Chris Lu
e2c7be3a2e adjust the needle mapper 
as discussed on
https://github.com/chrislusf/seaweedfs/pull/642#issuecomment-387606960
 2018-05-09 03:50:14 -07:00
Chris Lu
43a69d20bf change filer API to gRPC 2018-05-08 01:59:43 -07:00
Chris Lu
4936d6c342 adjusting for FUSE 2018-05-07 00:37:47 -07:00
Chris Lu
cb6b1d8d41 fix broken master health checking 2018-05-07 00:37:33 -07:00
Chris Lu
be58993f47 ensure the same Node is returned 2018-05-05 23:50:34 -07:00
Chris Lu
b857cf9d9b format 2018-05-05 23:39:29 -07:00
Chris Lu
456738ba64 refactoring fuse 2018-05-05 22:47:16 -07:00
Chris Lu
2cada598c6 fix test 2018-05-05 15:08:23 -07:00
Chris Lu
f3e07c231b adding folder icon 2018-05-05 14:41:40 -07:00
Chris Lu
7f49514f6f fix filer UI 2018-05-05 14:10:50 -07:00
Chris Lu
fffb14bc87 better support FUSE Lookup() 2018-05-05 02:01:50 -07:00
Chris Lu
050ab19264 add design 2018-05-04 23:38:42 -07:00
Chris Lu
94a35f25f3 simple refactoring 2018-05-03 00:16:02 -07:00
WenLong LI
b9068c1304
use volumes variable 
fix leveldb mount bug
 2018-05-03 12:13:53 +08:00
liwenlong05
02a1597642 fix leveldb mount bug 2018-05-03 11:20:38 +08:00
mtolman
e4b1e5c3c8
Fix for issue #635 
Fix for the following issue on master: https://github.com/chrislusf/seaweedfs/issues/635
 2018-04-17 13:26:48 -06:00
Chris Lu
d49e91cd78 cassandra use local quorum for better performance 2018-04-16 01:33:05 -07:00
Chris Lu
15b8499f8d refactoring 2018-04-16 01:16:06 -07:00
Chris Lu
d712c87ec0 update glide 2018-04-16 00:59:10 -07:00
Chris Lu
6448e0c25d update glide 2018-04-16 00:38:49 -07:00
Chris Lu
8c64d06b3a update redis library 2018-04-16 00:27:35 -07:00
Chris Lu
9cd6b33e97 fix https://github.com/chrislusf/seaweedfs/issues/603 2018-02-27 14:18:00 -08:00
Chris Lu
4b9b9547d4 unused option 2018-02-27 13:57:15 -08:00
Chris Lu
fc3924ac70 set file name to base file name 
fixing https://github.com/chrislusf/seaweedfs/issues/632
 2018-02-26 08:01:23 -08:00
tengattack
371ad2e397 fix reserve volume on node 2018-01-17 19:53:41 +08:00
Chris Lu
61ff330e99 adjust instructions 2018-01-15 20:33:12 -08:00
Chris Lu
1d09291f95 update upstream 2018-01-15 20:32:57 -08:00
Chris Lu
70f6740309 better fix for single master restart without peers changing 2017-12-06 00:14:14 -08:00
Chris Lu
9026b3e86e always remember the max volume id 2017-11-28 17:08:59 -08:00
Chris Lu
97f757c669 add debug message 
display the error message for debugging
https://github.com/chrislusf/seaweedfs/issues/603
 2017-11-25 01:31:13 -08:00
Chris Lu
ef6224e45d filer default to auto chunking file size to 32MB 2017-09-15 08:24:30 -07:00
Yang Pengcheng
a2112e1be0 fix ldb dir deleting 2017-08-30 22:55:03 +08:00
Chris Lu
3288d729ae fix compilation error 2017-08-30 02:24:19 -07:00
Chris Lu
58344980e4 preallocate disk space during compaction also, add cleanup for failed compaction 2017-08-29 23:59:53 -07:00
Chris Lu
f7c22f0159 fix bug when collecting vacuum errors 2017-08-29 23:11:20 -07:00
chanyoung
52fc476a6f filer: fixup print error message when make new directory 2017-08-15 15:54:12 +09:00
Chris Lu
dbee816205 fix typo 
fix https://github.com/chrislusf/seaweedfs/issues/549
 2017-08-10 10:26:39 -07:00
Chris Lu
3a8746ca54 better error message 2017-08-10 10:26:39 -07:00
kelgon
3bf883327e (fix #543)added body to error when Post encounter 4xx response 2017-07-28 16:11:23 +08:00
Chris Lu
e0dfa7a069 ensure 30GB or less 
avoid https://github.com/chrislusf/seaweedfs/issues/535
 2017-07-16 21:40:55 -07:00
Xinran Xu
4a0939a4e4 Fix panic error when target collection is not exist. 
Fix: interface conversion: interface {} is nil, not *topology.Collection.
 2017-07-14 20:04:33 +08:00
Chris Lu
f9a1769df1 fix nil raft server caused crash 
fix https://github.com/chrislusf/seaweedfs/issues/524
 2017-07-03 22:35:28 -07:00
Chris Lu
1fbb8723dc add cpu and mem pprof 2017-06-22 01:33:58 -07:00
Chris Lu
2c7dad589d stop when dn.parent is nil 
fix https://github.com/chrislusf/seaweedfs/issues/473
 2017-06-16 08:27:50 -07:00
Chris Lu
72e89b615b 301 is reported as 404 for http post 
fix https://github.com/chrislusf/seaweedfs/issues/512
 2017-06-15 21:21:32 -07:00
Chris Lu
5047bdb4a2 skip bytes cache 2017-06-03 11:44:24 -07:00
Chris Lu
f5bed84340 add volume id to error message 2017-06-03 01:58:28 -07:00
Chris Lu
83efe1547d revert webp support due to "go test" 
related to https://github.com/chai2010/webp/issues/9
 2017-05-28 10:25:40 -07:00
Chris Lu
478fe0ecf2 filer add readonly public port 2017-05-27 20:14:22 -07:00
Chris Lu
7ecc0f4b11 rename variables 2017-05-27 18:11:18 -07:00
Chris Lu
021ef0d14b adjust version to 0.76 2017-05-26 23:04:38 -07:00
Chris Lu
e3fc676260 add doc for 0.75 2017-05-26 23:01:40 -07:00
Chris Lu
82c3ccc8dd add btree for volume index 2017-05-26 22:51:25 -07:00
Chris Lu
bd97cbc523 skip busy reporting back to master 
possible fix for https://github.com/chrislusf/seaweedfs/issues/499
 2017-05-23 21:12:03 -07:00
Chris Lu
a7faefd4ce fix compilation error 2017-05-22 23:04:43 -07:00
Chris Lu
b3513d3167 skip only readonly volumes 
fix https://github.com/chrislusf/seaweedfs/issues/405
 2017-05-22 17:05:27 -07:00
Chris Lu
4ce6586710 use local random generator to avoid global lock 2017-05-19 23:33:36 -07:00
rmn
366fe0d394 Scale images to fit or fill 2017-05-05 12:17:30 +03:00
Vitaly Baev
7ab9160ee1 WebP resize support 2017-03-31 09:56:00 +03:00
Chris Lu
a337b844ec properly send heartbeats 
fix https://github.com/chrislusf/seaweedfs/issues/476
 2017-03-27 10:11:54 -07:00
sparklxb
c46e91d229 complement weed mount: add read and delete 2017-03-17 15:02:27 +08:00
Chris Lu
37b79174d1 Merge pull request #469 from sparklxb/master 
complement weed mount
 2017-03-14 08:21:29 -07:00
Stepan Bujnak
14ec3bfc1d Show contents of manifest file when cm=false is passed 2017-03-12 18:40:24 -07:00
Stepan Bujnak
a72c92e55f Added locking on the filer move operation 2017-03-11 02:52:04 -08:00
sparklxb
3a67b6ddc1 complement weed mount 2017-03-10 14:46:00 +08:00
sparklxb
7f85e15990 complement weed mount 2017-03-10 10:28:19 +08:00
sparklxb
813a4f7456 Fixes #266 2017-03-06 13:04:52 +08:00
sparklxb
4049c72e6c fix bug: replication, metadata not replicated 2017-03-04 11:24:12 +08:00
Chris Lu
30f138a538 fix tests 2017-02-12 22:18:48 -08:00
Chris Lu
41f686ab9d remove weeds.conf 2017-02-12 21:58:44 -08:00
Chris Lu
98d66a569b avoid nil 
fix https://github.com/chrislusf/seaweedfs/issues/452
 2017-02-11 10:21:28 -08:00
vancepym
ee6067e98c Fix: the maximum-length character exceed 255 
The length of a CHAR column is fixed to the length that you declare when you create the table. The length can be any value from 0 to 255. When CHAR values are stored, they are right-padded with spaces to the specified length. 

see https://dev.mysql.com/doc/refman/5.7/en/char.html
 2017-01-23 20:33:00 +08:00
Chris Lu
c2ce97785f set the volume size limit 
fixing https://github.com/chrislusf/seaweedfs/issues/447
 2017-01-22 21:36:36 -08:00
Chris Lu
f302c9fda2 always send the leader info on each heartbeat 2017-01-21 13:58:56 -08:00
Chris Lu
3cb0071025 supporting reading file of size zero 2017-01-21 13:45:16 -08:00
Chris Lu
043b7a7c65 add locking to access l.volumes 2017-01-20 10:18:43 -08:00
brstgt
0656838fe5 Don't return actual file size from CheckVolumeDataIntegrity, it will be 0 if last needle is a tombstone, so it's not reliable anyway 2017-01-20 16:37:45 +01:00
brstgt
4fb5bb09b6 Remove obsolete property Volume.dataFileSize 2017-01-20 16:31:11 +01:00
brstgt
e074a54a20 Delete volumes online without restarting volume server 2017-01-20 13:02:37 +01:00
brstgt
492f93416d Mount and unmount volumes online without restarting volume server 2017-01-20 12:49:20 +01:00
brstgt
18b3afc97a Log volumeId if dataFileSize != actual data file size - improve log types 2017-01-20 12:22:25 +01:00
brstgt
0c898df430 Log volumeId if dataFileSize != actual data file size 2017-01-20 11:52:23 +01:00
Chris Lu
c8f54aad8b adjust timing of leader election 2017-01-18 09:54:43 -08:00
Chris Lu
8de0027df5 Release 0.74 2017-01-18 09:39:47 -08:00
Chris Lu
5b0f5f456e volume server get notified of leader change 2017-01-18 09:34:27 -08:00
Chris Lu
59022b6fe0 fix byte counter on loading index file 
fix https://github.com/chrislusf/seaweedfs/issues/441
 2017-01-18 08:40:39 -08:00
Chris Lu
76da296799 upgrade to 0.73 2017-01-15 12:29:53 -08:00
Chris Lu
3065506b38 volume servers always connect to the master leader 2017-01-13 10:20:40 -08:00
Chris Lu
d953ed442c auto detect volume server ip address 2017-01-12 13:42:53 -08:00
Chris Lu
1be354bdce default IP to "localhost" 
so that the master can know the ip address.
 2017-01-10 23:57:59 -08:00
Chris Lu
adc8d344a5 update proto buffer 2017-01-10 23:57:16 -08:00
Chris Lu
e767c3ea4f disable master connection timeout 
temporarily disable master connection timeout due to heartbeat
connection timeout
 2017-01-10 01:30:00 -08:00
Chris Lu
e46c3415f7 gRpc for master~volume heartbeat 2017-01-10 01:01:12 -08:00
Chris Lu
4beaaa0650 simplify the gzip optimization logic 
fix https://github.com/chrislusf/seaweedfs/issues/436
 2017-01-09 19:31:58 -08:00
Chris Lu
9fa648e570 Merge pull request #435 from sparklxb/master 
support uploading files to specific dataCenter
 2017-01-09 10:07:07 -08:00
sparklxb
66e7013dfe suport uploading files to specific dataCenter 2017-01-09 23:34:47 +08:00
wangjie
90a6f43c56 fix the bug than we can't get filename when download file. 2017-01-09 14:55:21 +08:00
Chris Lu
53cf1b4900 change the basic unit to MB 2017-01-08 14:57:32 -08:00
Chris Lu
ed44f12f6d support Fallocate on linux 2017-01-08 11:01:46 -08:00
Chris Lu
7b6837cbc2 move back the section 2017-01-08 10:35:47 -08:00
sparklxb
da9b672d1b support additional header name-value pairs 2017-01-08 22:34:42 +08:00
sparklxb
86a7c56275 support additional header name-value pairs 2017-01-08 09:16:40 +08:00
Chris Lu
13e7069eb9 keep track of total data file size 2017-01-06 10:22:20 -08:00
Chris Lu
e61c9af523 check Accept instead of content type 
fix https://github.com/chrislusf/seaweedfs/issues/429
 2017-01-04 19:58:27 -08:00
Chris Lu
dc42d15352 release 0.72 2017-01-03 21:32:59 -08:00
Chris Lu
4d3353750e fix build issue 2017-01-03 21:28:30 -08:00
Chris Lu
297f64e286 Merge pull request #428 from sparklxb/master 
fix bug: upload big .gz file more than maxMB
 2017-01-03 21:23:08 -08:00
Chris Lu
8fb29e5230 change to 0.71 version 2017-01-03 21:17:54 -08:00
lixianbin
d96d0a87cf fix bug: upload big .gz file more than maxMB 2017-01-04 11:23:40 +08:00
listeng
5b18cf09e7 Return json data when content-type is "application/json" 
Before #344, the filer will return json data, and cschiano make a template that render to html. But sometimes need json data, so I add some code will return json data when content-type is "application/json".
 2016-12-26 16:49:43 +08:00
Mike Tolman
43e655afc8 Updating PostgreSQL file store integration to support directories. 2016-12-12 13:03:32 -07:00
Mike Tolman
e88795be13 Adding PostgreSQL Support 2016-12-07 17:24:40 -07:00
Mike Tolman
b18ec59c56 Revert "Adding PostgreSQL Filer Support" 
This reverts commit 10853e4d2f.
 2016-12-07 17:16:10 -07:00
Mike Tolman
10853e4d2f Adding PostgreSQL Filer Support 2016-12-07 17:13:03 -07:00
Mike Tolman
a1783a14cc Revert "Adding PostgreSQL Filer Support" 
This reverts commit 855058858a.
 2016-12-07 16:54:06 -07:00
Mike Tolman
855058858a Adding PostgreSQL Filer Support 2016-12-07 16:51:36 -07:00
Chris Lu
64509dd6ca fix text. 2016-11-28 20:07:27 -08:00
Chris Lu
924f797c58 Merge pull request #406 from eshujiushiwo/master 
Fix the filer connection of cassandra
 2016-11-28 20:05:45 -08:00
eshujiushiwo
e025fc00a4 let filer use all cassandra server 
let filer use all cassandra server
 2016-11-28 18:23:39 +08:00
eshujiushiwo
5ee7fdc0cc fix hosts strings 
fix hosts strings
 2016-11-28 16:03:39 +08:00
eshujiushiwo
3450eff288 fix info log 
fix info log
 2016-11-28 15:32:16 +08:00
eshujiushiwo
3e29d0f75b Update cassandra_store.go 2016-11-28 15:14:19 +08:00
eshujiushiwo
809aa028ec fix the connection of cassandra 
fix the connection of cassandra ,let filer can connect to multi cassandra nodes which are in the one cluster
 2016-11-28 15:01:04 +08:00