2551 Commits

Author	SHA1	Message	Date
Chris Lu	0d99a5da3c	c*: support kv operations	2020-09-02 18:39:24 -07:00
Chris Lu	645a4af3db	mongodb: support kv operations	2020-09-02 17:19:14 -07:00
Chris Lu	ecaa30c408	better error message	2020-09-02 17:17:44 -07:00
Chris Lu	7c770b727c	Update abstract_sql_store_kv.go	2020-09-02 10:13:48 -07:00
Chris Lu	05a6e2dc7f	sql kv upsert	2020-09-02 10:12:44 -07:00
Chris Lu	ed62f52452	fix sql insert with duplicated primary key	2020-09-02 10:09:49 -07:00
Chris Lu	9ea290aa12	fix error message	2020-09-01 23:21:41 -07:00
Chris Lu	06a1188267	mysql/postgres: support kv operations	2020-09-01 22:48:36 -07:00
Chris Lu	aa40295f06	leveldb: report not found entry	2020-09-01 22:48:23 -07:00
Chris Lu	87d7312bf6	mysql/postgres: properly report entry not found	2020-09-01 22:47:57 -07:00
Chris Lu	72b0a5f1d1	mysql or postgres: log find error	2020-09-01 22:25:17 -07:00
Chris Lu	2815bbe6c0	go fmt	2020-09-01 21:59:26 -07:00
Chris Lu	37234bf3f8	filer store adds kv support ... can compile now, need to implement those unimplemented	2020-09-01 21:59:00 -07:00
James Hartig	8e54e34576	volume: Don't unmount before deleting volume in copy ... If we unmount first and then delete, the delete fails because the volume was unmounted. Delete ends up doing the same thing as the unmount anyways.	2020-09-01 22:00:07 -04:00
Chris Lu	2b14ae5819	fix tests	2020-09-01 01:33:43 -07:00
Chris Lu	d91ec535b3	fix tests	2020-09-01 01:29:13 -07:00
Chris Lu	eb7929a971	rename filer2 to filer	2020-09-01 00:21:19 -07:00
Chris Lu	38e06d783d	volume: check disk space before compaction ... fix https://github.com/chrislusf/seaweedfs/issues/1440	2020-08-31 18:10:53 -07:00
Chris Lu	dbb10e0f0b	testing with more than 30days	2020-08-31 17:59:29 -07:00
Chris Lu	dd176cdb8b	avoid tiny step pagination	2020-08-31 11:28:03 -07:00
Chris Lu	97733c3dff	typo	2020-08-31 10:49:17 -07:00
Chris Lu	c8398bdb09	adjust metrics	2020-08-31 10:41:05 -07:00
Chris Lu	8a1c8e41b3	simplify if else logic	2020-08-31 10:39:24 -07:00
Chris Lu	f0c89cfacd	go fmt	2020-08-31 10:23:31 -07:00
Chris Lu	edb9d65e05	Merge pull request #1431 from kmlebedev/wip-prefix-search ... Wip prefix search	2020-08-31 10:22:14 -07:00
Konstantin Lebedev	9a195bebfd	accurate limit	2020-08-31 22:13:13 +05:00
Konstantin Lebedev	18b98f9747	not convert err to string	2020-08-31 21:55:18 +05:00
Konstantin Lebedev	22f32e75c4	rename	2020-08-31 21:52:05 +05:00
Konstantin Lebedev	a8de332beb	fix	2020-08-31 21:45:44 +05:00
Konstantin Lebedev	7f013d77d6	add var UnsupportedListDirectoryPrefixedErr	2020-08-31 21:43:32 +05:00
Konstantin Lebedev	60a86cfe04	add const	2020-08-31 21:35:16 +05:00
Konstantin Lebedev	82ea121d09	rm func	2020-08-31 18:50:06 +05:00
Konstantin Lebedev	2ea638f865	filer store wrapper can implement the logic to filter by prefi	2020-08-31 17:13:56 +05:00
Chris Lu	408e339c53	also delete the manifest chunk itself	2020-08-31 00:16:03 -07:00
Chris Lu	dd52037c27	1.92	2020-08-30 22:56:35 -07:00
Chris Lu	33d8c6c617	change log to warning	2020-08-30 21:01:44 -07:00
Chris Lu	e62d7f221f	a bit more logs	2020-08-30 21:00:24 -07:00
Chris Lu	22fe4ae573	mount: fix for UrBackup	2020-08-30 21:00:09 -07:00
Chris Lu	ae84a9ee2e	add logs	2020-08-30 20:59:04 -07:00
Chris Lu	a41588279a	change log level 5 to 4	2020-08-30 20:12:04 -07:00
Chris Lu	56244fb9a1	fix hard coded host address	2020-08-30 10:23:35 -07:00
Chris Lu	f2a8574448	filer and mount deletion resolves manifest chunks also	2020-08-30 02:07:14 -07:00
Chris Lu	99ecf63276	go fmt	2020-08-29 22:28:33 -07:00
Chris Lu	ef75ce8a34	use UTC for all time related folders	2020-08-29 21:01:14 -07:00
Chris Lu	b69cb74c03	read meta logs by timestamp ... pass in event ts when moving logs meta aggregator reads in memory logs only	2020-08-29 17:37:19 -07:00
Chris Lu	063c9ddac5	adjust logs	2020-08-29 11:56:22 -07:00
Chris Lu	ca658a97c5	add signatures to messages to avoid double processing	2020-08-28 23:48:48 -07:00
Chris Lu	63ad1abcce	watch: follow changes for a pattern	2020-08-28 19:43:04 -07:00
Chris Lu	b7a654a318	file mime avoid saving application/octet-stream	2020-08-28 19:42:40 -07:00
Chris Lu	707192f966	fix help message	2020-08-27 00:13:38 -07:00
Chris Lu	0be6863c87	rename	2020-08-26 22:40:15 -07:00
Chris Lu	50f7e2e89f	add log messages	2020-08-26 09:51:20 -07:00
Chris Lu	6bc618051c	log message	2020-08-26 09:32:56 -07:00
Chris Lu	9dae4e9d3e	fix logs	2020-08-26 09:31:11 -07:00
Chris Lu	4c1a3187bf	fix log error	2020-08-26 09:25:54 -07:00
Chris Lu	ab759f0ec2	erasure coding: fix EC error if multiple disks are configured in one volume server	2020-08-26 09:16:58 -07:00
Chris Lu	d1cf39f180	fix logging	2020-08-26 08:55:15 -07:00
Chris Lu	83012f543a	s: avoid possible completeMultipartUpload re-tries that can overwrite the uploaded file	2020-08-26 08:50:24 -07:00
Konstantin Lebedev	464d4c82ec	stop send heartbeat before stop volume server	2020-08-26 16:16:11 +05:00
Chris Lu	1901f15cd2	volume server: remove whitelist for status checking ... Volume liveness/readiness not work if jwt enabled, I try use param -whiteList but jwt breaks	2020-08-25 02:27:47 -07:00
Chris Lu	81e5124faf	s3: list bucket permission change from admin to read ... fix https://github.com/chrislusf/seaweedfs/issues/1430	2020-08-24 11:22:45 -07:00
Chris Lu	5f55a87101	close http response	2020-08-24 00:32:44 -07:00
Chris Lu	0b301b0b47	consume all response body in order to release requests	2020-08-23 21:32:29 -07:00
Chris Lu	2cdad77d82	fix checkin error	2020-08-23 18:30:11 -07:00
Chris Lu	81a8aa6581	1.91	2020-08-23 17:28:40 -07:00
Chris Lu	51346a5930	always keep the manifest list of chunks	2020-08-23 17:15:12 -07:00
Chris Lu	c1d1677a28	keep manifest chunks forever	2020-08-23 16:59:01 -07:00
Chris Lu	98175548c2	adjust deletion ordering	2020-08-23 16:27:12 -07:00
Chris Lu	f0e50e7b26	fix test	2020-08-23 16:12:34 -07:00
Chris Lu	aee27ccbe1	multiple fixes ... * adjust isOpen count * move ContinuousDirtyPages lock to filehandle * fix problem with MergeIntoVisibles, avoid reusing slices * let filer delete the garbage	2020-08-23 15:48:02 -07:00
Chris Lu	c19245886c	simpler test	2020-08-23 14:20:27 -07:00
Chris Lu	df816a58fe	add tests	2020-08-23 14:09:25 -07:00
Chris Lu	77393d3d30	add file handle locking when changing file entry	2020-08-23 00:35:50 -07:00
Chris Lu	d60bcbf08a	sorting chunks	2020-08-23 00:00:36 -07:00
Chris Lu	5e6b714836	add random test	2020-08-22 16:33:00 -07:00
Chris Lu	42ddbcc3a2	SQL insert falls back to update ... fix https://github.com/chrislusf/seaweedfs/issues/1429	2020-08-20 07:52:46 -07:00
Chris Lu	f48567c5c6	remove unused function	2020-08-19 22:53:49 -07:00
Chris Lu	c45ba5d7d4	fix listObjectsV2 response format ... fix https://github.com/chrislusf/seaweedfs/issues/1426 issue 1	2020-08-19 12:08:04 -07:00
James Hartig	3ccfa4c6ad	Added VolumeMarkWritable and VolumeStatus grpc methods ... This is necessary for copy to mark as read-only and then restore the original state afterwards.	2020-08-19 11:42:56 -04:00
Chris Lu	3b4b1d4a77	fix tests	2020-08-19 01:37:56 -07:00
Chris Lu	839634097f	also do flush on release	2020-08-19 01:27:10 -07:00
Chris Lu	c27e18aa6a	read possible old deleted chunks	2020-08-19 00:43:07 -07:00
Chris Lu	a78772d5ea	avoid shutdown in the middle of running	2020-08-19 00:42:02 -07:00
Chris Lu	ae9bc4a508	logs	2020-08-18 23:42:26 -07:00
Chris Lu	ed4b43b419	adjust logs	2020-08-18 23:42:09 -07:00
Chris Lu	9f1e0aeef5	delete chunks in the last step	2020-08-18 23:40:53 -07:00
Chris Lu	99d05f758c	adjust logs	2020-08-18 23:39:18 -07:00
Chris Lu	fe01191b5b	support read option readDeleted=true	2020-08-18 19:22:16 -07:00
Chris Lu	6ccd7f0a4d	refactoring	2020-08-18 18:01:37 -07:00
Chris Lu	c026eb0592	refactoring	2020-08-18 17:39:29 -07:00
Chris Lu	7e91ae592c	pass in option to read deleted entries ... not working yet	2020-08-18 17:37:26 -07:00
Chris Lu	51ecb49db3	for debugging	2020-08-18 17:35:39 -07:00
Chris Lu	ee11d98650	refactoring	2020-08-18 17:35:19 -07:00
Chris Lu	6a92f0bc7a	refactoring to typed Size ... Go is amazing with refactoring!	2020-08-18 17:04:28 -07:00
Chris Lu	618b2f6829	release resources only when needed to	2020-08-18 12:53:08 -07:00
Chris Lu	208849702d	logs	2020-08-18 12:52:54 -07:00
Chris Lu	1fcd083db3	printout data size	2020-08-18 09:09:29 -07:00
Chris Lu	3e5339337a	minor	2020-08-18 08:50:14 -07:00
Chris Lu	85001cbec7	properly report io.EOF	2020-08-18 08:18:54 -07:00
Chris Lu	0ca45a5cbc	adjust logs	2020-08-18 00:34:15 -07:00
Chris Lu	cd43738245	fix reading when filling zeros	2020-08-18 00:32:01 -07:00
Chris Lu	ecb3ce46be	adjust error logs	2020-08-18 00:31:42 -07:00
Chris Lu	30fe424469	add one more test case	2020-08-17 22:47:27 -07:00
Chris Lu	1b68ba953b	fix for out of range reads	2020-08-17 22:46:32 -07:00
Chris Lu	56fbd2c211	fix reading	2020-08-17 21:17:32 -07:00
Chris Lu	09e126bae5	refactoring: use interface	2020-08-17 20:20:08 -07:00
Chris Lu	be4d42b8e2	rename	2020-08-17 20:15:53 -07:00
Chris Lu	0625e63648	count 0 as part of the reads	2020-08-17 20:14:40 -07:00
Chris Lu	0be4b6e7f0	logs	2020-08-17 16:05:40 -07:00
Chris Lu	97e54a80d4	rename variables	2020-08-17 16:05:13 -07:00
Chris Lu	abdaf9958d	possibly read more	2020-08-17 16:04:56 -07:00
Chris Lu	9d46c7bc78	rename	2020-08-17 11:12:10 -07:00
Chris Lu	4ccfdaeb4d	prevent nil	2020-08-17 10:07:34 -07:00
Chris Lu	24c8e6bcb4	minor optimization	2020-08-17 10:03:34 -07:00
Chris Lu	f5837b7000	report error first	2020-08-16 23:49:10 -07:00
Chris Lu	2ac27616bc	fix possible out of range bytes ... avoid buff out of range resp.Data = buff[:totalRead]	2020-08-16 23:47:34 -07:00
Chris Lu	ff200398bb	1.90	2020-08-16 21:18:50 -07:00
Chris Lu	4ceeba9e70	streaming reads	2020-08-16 21:07:46 -07:00
Chris Lu	e72953dff7	logs	2020-08-16 21:06:03 -07:00
Chris Lu	627b081b67	adjust logs	2020-08-16 16:32:22 -07:00
Chris Lu	22e5132b3a	adjust log level	2020-08-16 16:25:11 -07:00
Chris Lu	ee0f92a6be	reduce memory allocation	2020-08-16 16:24:40 -07:00
Chris Lu	2ba817afac	read randomly written data	2020-08-16 15:16:46 -07:00
Chris Lu	6111b265e7	fix compilation	2020-08-16 01:38:16 -07:00
Chris Lu	4a77f0820a	clean up logs	2020-08-16 01:37:50 -07:00
Chris Lu	8c9e6eaacd	fix tests	2020-08-16 01:35:52 -07:00
Chris Lu	b71df82292	1.89	2020-08-16 00:57:28 -07:00
Chris Lu	1d9ea30b72	fix ViewFromVisibleIntervals	2020-08-16 00:49:26 -07:00
Chris Lu	aec7f32b02	fix reader_at	2020-08-16 00:49:08 -07:00
Chris Lu	5d80fc2ec7	adjust logs	2020-08-15 21:09:31 -07:00
Chris Lu	e0bfd3161a	update metadata only if changed	2020-08-15 19:56:24 -07:00
Chris Lu	003d48da21	adjust logs	2020-08-15 19:55:28 -07:00
Chris Lu	6ee8d952d2	adjust log level	2020-08-15 18:24:35 -07:00
Chris Lu	3e1395b767	adjust log message	2020-08-15 17:06:16 -07:00
Chris Lu	a22ee30596	fix nil	2020-08-15 17:01:42 -07:00
Chris Lu	0d60e67816	ensure meta data changes are updated	2020-08-15 14:15:07 -07:00
Chris Lu	5b43bddf20	proper deletion ordering ... delete central file store first, then delete local cache	2020-08-15 09:33:41 -07:00
Chris Lu	c647deace1	file size support set file length ... use Attr.FileSize and TotalChunkSize to determine file size	2020-08-15 09:32:47 -07:00
Chris Lu	c03bb180eb	fix error reporting	2020-08-14 00:44:02 -07:00
Chris Lu	eb493283dd	deletion error report	2020-08-14 00:23:01 -07:00
Chris Lu	a7f669044e	rename also applies to open file handle	2020-08-14 00:22:49 -07:00
Chris Lu	edfa73782f	adjust log level	2020-08-14 00:22:21 -07:00
Chris Lu	0983060a90	increase default volume file size limit to 1024 ... avoid possible large chunk size set on mount or filer	2020-08-13 09:07:22 -07:00
Chris Lu	090612492c	fix compilation	2020-08-13 00:07:56 -07:00
Chris Lu	d43129d27f	add back handles lock	2020-08-12 23:52:13 -07:00
Chris Lu	f735d579d3	adjust comment	2020-08-12 13:22:19 -07:00
Chris Lu	8824a9755c	remove directory cache	2020-08-12 13:11:04 -07:00
Chris Lu	83cad3da79	add retry file upload	2020-08-11 20:30:11 -07:00
Chris Lu	152a6cbc2b	minor adjustments	2020-08-10 20:42:27 -07:00
Chris Lu	f1b40567e3	Merge pull request #1419 from robin1900/master ... fix bug: two same volumeId in different collections	2020-08-10 20:38:55 -07:00