mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-27 11:27:56 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,636 Commits 32 Branches 296 Tags 153 MiB
e9cee59c8c
Commit Graph

1579 Commits

Author SHA1 Message Date
Chris Lu
76cbe8bf33 instant notification of new volumes added or deleted 2018-08-24 01:26:56 -07:00
Chris Lu
5ccf8e8078 reformat 2018-08-24 00:30:03 -07:00
Chris Lu
98b8f8649d adding filer.export prototype 2018-08-22 23:54:22 -07:00
Chris Lu
bb92fd1db0 fix test error 2018-08-19 18:52:50 -07:00
Chris Lu
906472b21d filer add upload and drag&dorp 2018-08-19 18:42:40 -07:00
Chris Lu
95b997fb35 go fmt 2018-08-19 15:18:37 -07:00
Chris Lu
f827ada811 merge notification config with filer.toml 2018-08-19 15:17:55 -07:00
Chris Lu
5c467083e5 go fmt 2018-08-13 01:22:32 -07:00
Chris Lu
f036ef8a3c add filer notification 2018-08-13 01:20:49 -07:00
Chris Lu
75d63db60d randomize raft server startup 
also some go fmt
 2018-08-12 14:27:14 -07:00
Chris Lu
b931ced0a3 readable file size 2018-08-10 23:47:31 -07:00
Chris Lu
032f1226db fix compilation error 2018-07-29 02:25:03 -07:00
Chris Lu
d3205a0070 go fmt 2018-07-28 21:02:56 -07:00
Chris Lu
888eb2abb5 filer read write all via locations from MasterClient 2018-07-28 14:51:36 -07:00
Chris Lu
1d779389cb MasterClient replicates all vid locations 2018-07-28 14:22:46 -07:00
Chris Lu
1ab8232b55 filer only talk to the master leader 2018-07-28 01:30:03 -07:00
Chris Lu
e8d4be579d send initial list of volume location 2018-07-28 01:17:35 -07:00
Chris Lu
a12c7b86b0 broadcast messages of new and deleted volumes 2018-07-27 23:09:55 -07:00
Chris Lu
e4b7e31902 add back recursively delete a folder 2018-07-24 22:33:26 -07:00
Chris Lu
4f317c7e3d working version3 2018-07-24 01:36:04 -07:00
Chris Lu
0b99557c29 fix compilation error 2018-07-24 01:09:32 -07:00
Chris Lu
9bf93e698e refactoring: avoid common code 2018-07-24 00:58:52 -07:00
Chris Lu
aae77e8ea3 initialize read seeker 2018-07-24 00:54:15 -07:00
Chris Lu
5a237af973 fix nil if not images 2018-07-24 00:49:53 -07:00
Chris Lu
ff331178aa
Merge pull request #695 from zhangmingfeng/master 
增加chunk图片文件支持width和height
 2018-07-22 19:54:04 -07:00
Chris Lu
7e76a7d21b
Merge pull request #694 from himyxia/inprove_switch_case 
better way to handler switch case
 2018-07-22 10:29:37 -07:00
Jimmy
1c77d323b0 better way to handler switch case 2018-07-23 01:27:10 +08:00
Chris Lu
056871242b remove println 2018-07-22 01:17:49 -07:00
Chris Lu
6319d84f42 s3 API add ListObjectsV1 2018-07-22 01:14:36 -07:00
Chris Lu
8f240276db s3 add HEAD DELETE 2018-07-21 18:49:47 -07:00
Chris Lu
bfa87b4f50 adjust log level 2018-07-21 18:47:23 -07:00
Chris Lu
648626115a fix printing 2018-07-21 18:47:07 -07:00
Chris Lu
12e6ccc581 remove unused code 2018-07-21 17:54:14 -07:00
Chris Lu
702d7ac424 delegating old chunk purging to CreateEntry 
1. filer add file user id and group id
2. auto-chunking set the file name correctly
3. delegating old chunk purging to CreateEntry
 2018-07-21 17:47:59 -07:00
Chris Lu
852af28f91 needle priting format 2018-07-21 17:41:21 -07:00
Chris Lu
7e2031b18f go fmt 2018-07-21 17:39:10 -07:00
Chris Lu
dc8dd8b486 Merge branch 'master' into add_s3 2018-07-21 15:59:18 -07:00
Chris Lu
8480008a9a add s3 upload, and removing mono and multi part upload analyzer 
removing mono and multi part upload analyzer, which were used just to determine the file name
 2018-07-21 10:39:02 -07:00
Chris Lu
80d80daf64 set filer2.ErrNotFound for not found entry 2018-07-20 02:14:18 -07:00
Chris Lu
e73b859b1d fix compilation error 2018-07-20 00:46:02 -07:00
Chris Lu
568132c0c9 delete bucket also removes collection data 2018-07-20 00:10:01 -07:00
Chris Lu
a09ef6002a adjust name to avoid confusion 2018-07-19 01:46:18 -07:00
Chris Lu
c7228fefa7 add bucket creation and deletion 
1. option for "weed s3 -filer.dir.buckets" to choose a folder for buckets
2. create a bucket
3. delete a bucket, recursively delete all metadata on filer
 2018-07-19 01:21:44 -07:00
Chris Lu
85eae57e4c Store and replicate timestamp of file deletion in tombstone marker 
merging changes from https://github.com/chrislusf/seaweedfs/pull/609/files
 2018-07-14 20:26:45 -07:00
Chris Lu
67763fb375 fix data center setting error 2018-07-14 14:09:45 -07:00
Chris Lu
842dab07b4 weed mount can request to prioritize to write to a data center 2018-07-14 13:36:28 -07:00
Chris Lu
007c4a1965 alt request for dc preferred requests 2018-07-10 00:20:50 -07:00
Chris Lu
6c4fee85e3 add primary request and backup request when requesting volumes for a data center 2018-07-09 23:18:20 -07:00
Chris Lu
50d351944f filer add data center option 2018-07-09 02:22:48 -07:00
Chris Lu
d4d7ced922 refactoring: add type for needle id, offset 
later the type size can possibly be adjusted
 2018-07-08 02:28:04 -07:00
Chris Lu
702ff48cda add dir list limit option 2018-07-07 02:18:47 -07:00
Chris Lu
2ad45ca04f refactor 2018-07-07 02:03:25 -07:00
zhangmingfeng
79d18c69b4 增加chunk图片文件支持width和height 2018-07-05 10:34:17 +08:00
Chris Lu
77fc8c5914 keep alive for gRpc calls 2018-07-03 19:07:55 -07:00
Chris Lu
0d83c1b91e refactoring 2018-06-25 00:01:53 -07:00
Chris Lu
ef98d26c9e ensuring removing registered datanode when disconnecting 2018-06-24 23:20:27 -07:00
Chris Lu
03f50180f3 simplifying the leader election by raft 
fixing https://github.com/chrislusf/seaweedfs/issues/629
 2018-06-12 01:54:09 -07:00
Chris Lu
17918f0b82 fix compilation error on go tip 2018-06-11 23:51:36 -07:00
Chris Lu
5bd72696ac weed mount add ttl option 2018-06-11 23:13:33 -07:00
Chris Lu
98110c1697 add collection, replication, ttl to each file 2018-06-10 16:57:32 -07:00
Chris Lu
5c25d29272 support renaming files 2018-06-06 22:11:01 -07:00
Chris Lu
95fe745a0c compatible with previous json rendering 2018-06-04 19:28:59 -07:00
Chris Lu
ebf58709d3 remove unused register file function 2018-06-01 23:24:34 -07:00
Chris Lu
43e3f5724c use fixed list of masters in both filer and volume servers 2018-06-01 00:39:39 -07:00
Chris Lu
a6f7f9b0b8 add missing modify and create times 2018-05-31 22:50:19 -07:00
Chris Lu
7efeb146c5 fix log 2018-05-31 22:49:55 -07:00
Chris Lu
0301504184 add mime, use simple insert and update filer store API 
1. add mime type to file in filer
2. purge old chunks if overwrite during insert
 2018-05-30 20:24:57 -07:00
Chris Lu
78aabf66ce add error details 2018-05-29 23:46:17 -07:00
Chris Lu
07e0d13d2d filer support reading multiple chunks, with range support 2018-05-28 05:39:12 -07:00
Chris Lu
dd5661b297 adding favicon to filer also 2018-05-28 02:35:58 -07:00
Chris Lu
9b603f5ffa split filer read into 2 files 2018-05-28 02:24:14 -07:00
Chris Lu
8db9319a06 refactoring: go fmt, reorg 2018-05-27 23:59:49 -07:00
Chris Lu
480a073f1f refactoring: split into 4 files 2018-05-27 23:53:10 -07:00
Chris Lu
5b844d7011 pass reference of master server 
SendHeartbeat passes lock by value: weed_server.MasterServer contains
sync.Mutex
 2018-05-27 11:58:00 -07:00
Chris Lu
458ada173e go fmt 2018-05-27 11:52:26 -07:00
Chris Lu
8647191bee removing filer package! 2018-05-27 11:52:15 -07:00
Chris Lu
03d76479a5 add reds 2018-05-27 11:14:29 -07:00
Chris Lu
ab86d263c9 add Cassandra 2018-05-27 00:01:15 -07:00
Chris Lu
1c0a2a7db8 reformat 2018-05-26 23:59:56 -07:00
Chris Lu
869161a261 support both mysql and postgres 2018-05-26 22:02:49 -07:00
Chris Lu
68bcaff14d mysql can compile, not yet tested! 2018-05-26 05:32:15 -07:00
Chris Lu
9e77563c99 add leveldb store 
1. switch to viper for filer store configuration
2. simplify FindEntry() return values, removing “found”
3. add leveldb store
 2018-05-26 03:49:46 -07:00
Chris Lu
6de84c64c6 adding create time 2018-05-25 23:26:40 -07:00
Chris Lu
f8776ad5cd add breadcrumb to filer UI 2018-05-24 23:42:13 -07:00
Chris Lu
a01557f4cb adjust filer web page 2018-05-24 23:21:12 -07:00
Chris Lu
d773e11c7a file handler directly read from volume servers 
this mostly works fine now!

next: need to cache files to local disk
 2018-05-24 01:22:37 -07:00
Chris Lu
536559f62d copy works, edit somehow still fails 2018-05-23 03:08:46 -07:00
Chris Lu
873868cc10 not working now 
need to add file handler
 2018-05-22 04:31:44 -07:00
Chris Lu
7362de9a18 weed mount can work well 
TODO: somehow filer url is returning empty content
 2018-05-22 03:26:38 -07:00
Chris Lu
9dd228747c filer copy added uid/gid 2018-05-21 01:25:30 -07:00
Chris Lu
f07482382b able to update file content 
having some issue when vi reports file changed.
 2018-05-21 00:00:28 -07:00
Chris Lu
cc66e25cd2 merge intervals 2018-05-20 17:06:09 -07:00
Chris Lu
ab4ddb1e0e fix directory creation, directory listing 2018-05-19 12:07:15 -07:00
Chris Lu
e31c514b00 adding modified time to file chunk 2018-05-16 00:54:44 -07:00
Chris Lu
b303a02461 cp file can work 
1. consolidate to filer_pb.FileChunk
2. dir add file, mkdir
3. file flush, write

updates having issue
 2018-05-16 00:08:44 -07:00
Chris Lu
c7a71d35b0 fix on pagination 2018-05-14 20:27:48 -07:00
Chris Lu
58954bf46f pass file attributes from filer to mount 2018-05-14 02:02:17 -07:00
Chris Lu
67401f1908 fix gRpc nil response 2018-05-14 00:17:22 -07:00
Chris Lu
c5cf9bd290 properly working filer 2018-05-13 23:56:16 -07:00
Chris Lu
a4740ca836 fix compilation error for windows 2018-05-13 00:49:59 -07:00
Chris Lu
9f345da20f mv filer proto to filer_pb 2018-05-09 23:18:02 -07:00
Chris Lu
942c2cbd7b mv pb to master_pb 2018-05-09 23:11:54 -07:00
Chris Lu
43a69d20bf change filer API to gRPC 2018-05-08 01:59:43 -07:00
Chris Lu
cb6b1d8d41 fix broken master health checking 2018-05-07 00:37:33 -07:00
Chris Lu
f3e07c231b adding folder icon 2018-05-05 14:41:40 -07:00
Chris Lu
7f49514f6f fix filer UI 2018-05-05 14:10:50 -07:00
Chris Lu
fffb14bc87 better support FUSE Lookup() 2018-05-05 02:01:50 -07:00
Chris Lu
9cd6b33e97 fix https://github.com/chrislusf/seaweedfs/issues/603 2018-02-27 14:18:00 -08:00
Chris Lu
70f6740309 better fix for single master restart without peers changing 2017-12-06 00:14:14 -08:00
Chris Lu
9026b3e86e always remember the max volume id 2017-11-28 17:08:59 -08:00
Chris Lu
3288d729ae fix compilation error 2017-08-30 02:24:19 -07:00
Chris Lu
58344980e4 preallocate disk space during compaction also, add cleanup for failed compaction 2017-08-29 23:59:53 -07:00
Chris Lu
f9a1769df1 fix nil raft server caused crash 
fix https://github.com/chrislusf/seaweedfs/issues/524
 2017-07-03 22:35:28 -07:00
Chris Lu
72e89b615b 301 is reported as 404 for http post 
fix https://github.com/chrislusf/seaweedfs/issues/512
 2017-06-15 21:21:32 -07:00
Chris Lu
5047bdb4a2 skip bytes cache 2017-06-03 11:44:24 -07:00
Chris Lu
f5bed84340 add volume id to error message 2017-06-03 01:58:28 -07:00
Chris Lu
83efe1547d revert webp support due to "go test" 
related to https://github.com/chai2010/webp/issues/9
 2017-05-28 10:25:40 -07:00
Chris Lu
478fe0ecf2 filer add readonly public port 2017-05-27 20:14:22 -07:00
Chris Lu
7ecc0f4b11 rename variables 2017-05-27 18:11:18 -07:00
rmn
366fe0d394 Scale images to fit or fill 2017-05-05 12:17:30 +03:00
Vitaly Baev
7ab9160ee1 WebP resize support 2017-03-31 09:56:00 +03:00
Chris Lu
a337b844ec properly send heartbeats 
fix https://github.com/chrislusf/seaweedfs/issues/476
 2017-03-27 10:11:54 -07:00
sparklxb
c46e91d229 complement weed mount: add read and delete 2017-03-17 15:02:27 +08:00
Chris Lu
37b79174d1 Merge pull request #469 from sparklxb/master 
complement weed mount
 2017-03-14 08:21:29 -07:00
Stepan Bujnak
14ec3bfc1d Show contents of manifest file when cm=false is passed 2017-03-12 18:40:24 -07:00
sparklxb
3a67b6ddc1 complement weed mount 2017-03-10 14:46:00 +08:00
sparklxb
7f85e15990 complement weed mount 2017-03-10 10:28:19 +08:00
Chris Lu
41f686ab9d remove weeds.conf 2017-02-12 21:58:44 -08:00
Chris Lu
98d66a569b avoid nil 
fix https://github.com/chrislusf/seaweedfs/issues/452
 2017-02-11 10:21:28 -08:00
Chris Lu
c2ce97785f set the volume size limit 
fixing https://github.com/chrislusf/seaweedfs/issues/447
 2017-01-22 21:36:36 -08:00
Chris Lu
f302c9fda2 always send the leader info on each heartbeat 2017-01-21 13:58:56 -08:00
Chris Lu
3cb0071025 supporting reading file of size zero 2017-01-21 13:45:16 -08:00
brstgt
e074a54a20 Delete volumes online without restarting volume server 2017-01-20 13:02:37 +01:00
brstgt
492f93416d Mount and unmount volumes online without restarting volume server 2017-01-20 12:49:20 +01:00
Chris Lu
c8f54aad8b adjust timing of leader election 2017-01-18 09:54:43 -08:00
Chris Lu
5b0f5f456e volume server get notified of leader change 2017-01-18 09:34:27 -08:00
Chris Lu
3065506b38 volume servers always connect to the master leader 2017-01-13 10:20:40 -08:00
Chris Lu
d953ed442c auto detect volume server ip address 2017-01-12 13:42:53 -08:00
Chris Lu
e767c3ea4f disable master connection timeout 
temporarily disable master connection timeout due to heartbeat
connection timeout
 2017-01-10 01:30:00 -08:00
Chris Lu
e46c3415f7 gRpc for master~volume heartbeat 2017-01-10 01:01:12 -08:00
Chris Lu
ed44f12f6d support Fallocate on linux 2017-01-08 11:01:46 -08:00
sparklxb
da9b672d1b support additional header name-value pairs 2017-01-08 22:34:42 +08:00
sparklxb
86a7c56275 support additional header name-value pairs 2017-01-08 09:16:40 +08:00
Chris Lu
e61c9af523 check Accept instead of content type 
fix https://github.com/chrislusf/seaweedfs/issues/429
 2017-01-04 19:58:27 -08:00
listeng
5b18cf09e7 Return json data when content-type is "application/json" 
Before #344, the filer will return json data, and cschiano make a template that render to html. But sometimes need json data, so I add some code will return json data when content-type is "application/json".
 2016-12-26 16:49:43 +08:00
Mike Tolman
43e655afc8 Updating PostgreSQL file store integration to support directories. 2016-12-12 13:03:32 -07:00
Mike Tolman
e88795be13 Adding PostgreSQL Support 2016-12-07 17:24:40 -07:00
Mike Tolman
b18ec59c56 Revert "Adding PostgreSQL Filer Support" 
This reverts commit 10853e4d2f.
 2016-12-07 17:16:10 -07:00
Mike Tolman
10853e4d2f Adding PostgreSQL Filer Support 2016-12-07 17:13:03 -07:00
Chris Lu
dcaf1796fe add option to enable caching 2016-11-16 07:09:57 -08:00
Yang zhixiang
57005ed7f8 fix bug : join post is empty 2016-11-08 11:00:46 +08:00
Yang zhixiang
4194ae33fd fix raft_server.go isPeersChanged bug 2016-11-03 15:46:59 +08:00
霍晓栋
78474409a5 filer mysqlstore bug fix 2016-09-08 11:35:54 +08:00
霍晓栋
3aa021a812 refactoring mysql store code 2016-09-05 14:10:22 +08:00
霍晓栋
b0035747e3 add filer support 2016-08-31 11:32:30 +08:00
Mike Tolman
01d3f69c52 Adding AutoChunk/MaxMB Support to Filer API 
This is related to the following issue I added to chrislusf/seaweedfs:
https://github.com/chrislusf/seaweedfs/issues/342
 2016-08-05 16:01:30 -06:00
Mike Tolman
a89a3c86d0 Revert "Add AutoChunking to the Filer API, so that you can upload really large files through the filer API." 
This reverts commit 09059bfdcc.
 2016-08-05 15:47:46 -06:00
Mike Tolman
0f4c7dd8fd Revert "Ooops. Missed a line." 
This reverts commit 14d4252904.
 2016-08-05 15:46:08 -06:00
Mike Tolman
ce99bb927d Revert "Adding HTTP verb whitelisting options." 
This reverts commit 34837afc7a.
 2016-08-05 15:45:48 -06:00
Mike Tolman
34837afc7a Adding HTTP verb whitelisting options. 2016-08-05 15:23:43 -06:00
Mike Tolman
14d4252904 Ooops. Missed a line. 2016-08-05 15:22:46 -06:00
Mike Tolman
09059bfdcc Add AutoChunking to the Filer API, so that you can upload really large files through the filer API. 2016-08-05 15:19:10 -06:00
Chris Lu
cdae9fc680 add "weed copy" command to copy files to filer 2016-07-20 23:45:55 -07:00
霍晓栋
574cc41f0b recover filer read handler logic 2016-07-21 12:20:22 +08:00
cschiano
af905a3ff7 Add limit parameter and pagination for files 2016-07-20 10:46:28 +02:00
cschiano
c63cebaee1 Add a visualization for the filer 2016-07-18 10:28:24 +02:00
霍晓栋
339856eb78 fix bug when choosing the healthy master in filer service 2016-06-30 19:54:05 +08:00
霍晓栋
079fdbad13 fix volume_layout bug, add new feature for volume etag write,filer pass through query params' 2016-06-29 16:05:00 +08:00
霍晓栋
1f63094542 refactor Volume Assign function 2016-06-26 10:50:18 +08:00
Chris Lu
cc01c352c3 refactoring 2016-06-13 15:07:17 -07:00
霍晓栋
3a25af223f refactor work for filer write handler 2016-06-08 15:46:14 +08:00
霍晓栋
26bb9094d4 made a little change for Amazon S3 API's compatibility 2016-06-08 12:30:20 +08:00
Chris Lu
0e3140c54c a little refactoring 2016-06-06 23:50:27 -07:00
霍晓栋
d514b6f3f9 read write handler update 2016-06-07 11:45:00 +08:00
霍晓栋
1ef81ac518 filer could detect master nodes healthy status and choose a working one when encountering SPOF 2016-06-07 11:38:40 +08:00
Chris Lu
b456173eb8 delete file storage when POST to update 
PUT is used to update. But sometimes POST is also used to post the same
file again. In this case, the actual file storage was not marked as
deleted.

This fix will try its best to delete the orphan files.
 2016-06-03 00:24:55 -07:00
Chris Lu
535b1610c8 add fid and url to filer post result 
fix https://github.com/chrislusf/seaweedfs/issues/274
 2016-06-02 20:44:50 -07:00
Chris Lu
4f47b8cff9 refactoring 2016-06-02 20:05:34 -07:00
Chris Lu
5ce6bbf076 directory structure change to work with glide 
glide has its own requirements. My previous workaround caused me some
code checkin errors. Need to fix this.
 2016-06-02 18:09:14 -07:00