Commit Graph

8228 Commits

Author SHA1 Message Date
Roman Arutyunyan
6f3c821d1f HTTP/3: refactored request body parser.
The change reduces diff to the default branch for
src/http/ngx_http_request_body.c.

Also, client Content-Length, if present, is now checked against the real body
size sent by client.
2021-01-25 16:16:47 +03:00
Roman Arutyunyan
a7cf99b10d QUIC: fixed stateless reset recognition and send.
Previously, if an unexpected packet was received on an existing QUIC
connection, stateless reset token was neither recognized nor sent.
2021-02-01 14:46:36 +03:00
Roman Arutyunyan
fef3360466 QUIC: refactored packet processing.
- split ngx_quic_process_packet() in two functions with the second one called
  ngx_quic_process_payload() in charge of decrypring and handling the payload
- renamed ngx_quic_payload_handler() to ngx_quic_handle_frames()
- moved error cleanup from ngx_quic_input() to ngx_quic_process_payload()
- moved handling closed connection from ngx_quic_handle_frames() to
  ngx_quic_process_payload()
- minor fixes
2021-01-28 12:35:18 +03:00
Vladimir Homutov
89dda20510 QUIC: stateless retry.
Previously, quic connection object was created when Retry packet was sent.
This is neither necessary nor convenient, and contradicts the idea of retry:
protecting from bad clients and saving server resources.

Now, the connection is not created, token is verified cryptographically
instead of holding it in connection.
2021-01-29 15:53:47 +03:00
Roman Arutyunyan
cd62534300 HTTP/3: call ngx_handle_read_event() from client header handler.
This function should be called at the end of an event handler to prepare the
event for the next handler call.  Particularly, the "active" flag is set or
cleared depending on data availability.

With this call missing in one code path, read handler was not called again
after handling the initial part of the client request, if the request was too
big to fit into a single STREAM frame.

Now ngx_handle_read_event() is called in this code path.  Also, read timer is
restarted.
2021-01-29 19:42:47 +03:00
Sergey Kandaurov
68aa6fec77 README: reflect renaming of several transport parameter directives.
Reported by Kyriakos Zarifis.
2021-01-27 13:09:45 +03:00
Maxim Dounin
2f008a76f8 Year 2021. 2021-01-19 20:35:17 +03:00
Maxim Dounin
2ec8fac2d6 Core: removed post_accept_timeout.
Keeping post_accept_timeout in ngx_listening_t is no longer needed since
we've switched to 1 second timeout for deferred accept in 5541:fdb67cfc957d.

Further, using it in HTTP code can result in client_header_timeout being
used from an incorrect server block, notably if address-specific virtual
servers are used along with a wildcard listening socket, or if we've switched
to a different server block based on SNI in SSL handshake.
2021-01-19 20:32:00 +03:00
Maxim Dounin
d2c0b9a6c7 Removed incorrect optimization of HEAD requests.
The stub status module and ngx_http_send_response() (used by the empty gif
module and the "return" directive) incorrectly assumed that responding
to HEAD requests always results in r->header_only being set.  This is not
true, and results in incorrect behaviour, for example, in the following
configuration:

   location / {
       image_filter size;
       return 200 test;
   }

Fix is to remove this incorrect micro-optimization from both stub status
module and ngx_http_send_response().

Reported by Chris Newton.
2021-01-19 20:21:12 +03:00
Maxim Dounin
e1ca985122 Upstream: fixed zero size buf alerts on extra data (ticket #2117).
After 7675:9afa45068b8f and 7678:bffcc5af1d72 (1.19.1), during non-buffered
simple proxying, responses with extra data might result in zero size buffers
being generated and "zero size buf" alerts in writer.  This bug is similar
to the one with FastCGI proxying fixed in 7689:da8d758aabeb.

In non-buffered mode, normally the filter function is not called if
u->length is already 0, since u->length is checked after each call of
the filter function.  There is a case when this can happen though: if
the response length is 0, and there are pre-read response body data left
after reading response headers.  As such, a check for u->length is needed
at the start of non-buffered filter functions, similar to the one
for p->length present in buffered filter functions.

Appropriate checks added to the existing non-buffered copy filters
in the upstream (used by scgi and uwsgi proxying) and proxy modules.
2021-01-12 16:59:31 +03:00
Gena Makhomed
7b6b8b62dd Contrib: vim syntax, update core and 3rd party module directives. 2020-12-29 13:13:57 +02:00
Maxim Dounin
09b41d5eab Version bump. 2021-01-11 22:06:27 +03:00
Roman Arutyunyan
52d0bf620a HTTP/3: removed HTTP/3-specific code.
The ngx_http_set_lingering_close() function is not called for HTTP/3.

The change reduces diff to the default branch.
2020-12-21 17:35:13 +00:00
Roman Arutyunyan
7bac596afb HTTP/3: client header validation.
A header with the name containing null, CR, LF, colon or uppercase characters,
is now considered an error.  A header with the value containing null, CR or LF,
is also considered an error.

Also, header is considered invalid unless its name only contains lowercase
characters, digits, minus and optionally underscore.  Such header can be
optionally ignored.
2021-01-18 13:43:36 +03:00
Roman Arutyunyan
88f6b969e6 HTTP/3: added comment. 2021-01-12 21:08:55 +00:00
Roman Arutyunyan
4e312daa7e HTTP/3: client pseudo-headers restrictions.
- :method, :path and :scheme are expected exactly once and not empty
- :method and :scheme character validation is added
- :authority cannot appear more than once
2021-01-22 15:57:41 +03:00
Roman Arutyunyan
9e489d208f HTTP/3: refactored request parser.
The change reduces diff to the default branch for
src/http/ngx_http_request.c and src/http/ngx_http_parse.c.
2021-01-22 16:34:06 +03:00
Sergey Kandaurov
f3c9e9f961 QUIC: draft-33 salt and retry keys.
Notably, the version negotiation table is updated to reject draft-33/QUICv1
(which requires a new TLS codepoint) unless explicitly asked to built with.
2021-01-11 15:25:48 +03:00
Vladimir Homutov
291eb52899 QUIC: fixed header protection macro name. 2020-12-30 20:47:35 +03:00
Vladimir Homutov
c4f31ccca1 QUIC: ngx_quic_bpf module.
The quic kernel bpf helper inspects packet payload for DCID, extracts key
and routes the packet into socket matching the key.

Due to reuseport feature, each worker owns a personal socket, which is
identified by the same key, used to create DCID.

BPF objects are locked in RAM and are subject to RLIMIT_MEMLOCK.
The "ulimit -l" command may be used to setup proper limits, if maps
cannot be created with EPERM or updated with ETOOLONG.
2020-12-25 15:01:15 +03:00
Maxim Dounin
402f4cda0c release-1.19.6 tag 2020-12-15 17:41:39 +03:00
Maxim Dounin
65ddeebf33 nginx-1.19.6-RELEASE 2020-12-15 17:41:39 +03:00
Maxim Dounin
b3341c06cb Updated OpenSSL used for win32 builds. 2020-12-15 16:49:24 +03:00
Vladimir Homutov
b20b58ca7d Core: added interface to linux bpf() system call.
It contains wrappers for operations with BPF maps and for loading BPF programs.
2020-12-15 15:23:07 +03:00
Vladimir Homutov
fb655007a1 QUIC: ngx_quic_module. 2020-12-25 14:18:51 +03:00
Vladimir Homutov
4952711097 QUIC: moved all quic sources into src/event/quic. 2020-12-25 14:01:28 +03:00
Sergey Kandaurov
45666324af QUIC: removed unused <openssl/aes.h> inclusion.
The low-level API was used in early QUIC development.
2020-12-22 16:41:56 +03:00
Sergey Kandaurov
e00439e55d QUIC: fixed -Wtype-limits with GCC <= 5 (ticket #2104). 2020-12-22 12:04:16 +03:00
Sergey Kandaurov
df8ef280a5 QUIC: fixed logging PATH_CHALLENGE/RESPONSE and build with GCC < 5. 2020-12-22 12:04:15 +03:00
Sergey Kandaurov
a969893656 QUIC: fixed building ALPN callback without debug and http2. 2020-12-22 12:04:15 +03:00
Sergey Kandaurov
71f9b41c7f QUIC: fixed build with OpenSSL < 1.1.1.
The <openssl/kdf.h> header is available since OpenSSL 1.1.0, and HKDF API
used for separate Extract and Expand steps in TLSv1.3 - since OpenSSL 1.1.1.
2020-12-22 12:03:43 +03:00
Sergey Kandaurov
2bc8ee6535 QUIC: converted to SSL_CIPHER_get_protocol_id().
This API is available in BoringSSL for quite some time:
https://boringssl.googlesource.com/boringssl/+/3743aaf
2020-12-21 15:05:43 +03:00
Sergey Kandaurov
405b9be899 HTTP/3: staticize internal parsing functions. 2020-12-16 12:47:41 +00:00
Sergey Kandaurov
c3714a8089 HTTP/3: staticize ngx_http_v3_methods. 2020-12-16 12:47:38 +00:00
Sergey Kandaurov
1a3bf7d593 Merged with the default branch. 2020-12-15 16:55:43 +00:00
Maxim Dounin
b138e26395 Fixed double close of non-regular files in flv and mp4.
With introduction of open_file_cache in 1454:f497ed7682a7, opening a file
with ngx_open_cached_file() automatically adds a cleanup handler to close
the file.  As such, calling ngx_close_file() directly for non-regular files
is no longer needed and will result in duplicate close() call.

In 1454:f497ed7682a7 ngx_close_file() call for non-regular files was removed
in the static module, but wasn't in the flv module.  And the resulting
incorrect code was later copied to the mp4 module.  Fix is to remove the
ngx_close_file() call from both modules.

Reported by Chris Newton.
2020-12-11 13:42:07 +03:00
Maxim Dounin
2e94c81b0b Removed extra allocation for r->uri.
The ngx_http_parse_complex_uri() function cannot make URI longer and does
not null-terminate URI, so there is no need to allocate an extra byte.  This
allocation appears to be a leftover from changes in 461:a88a3e4e158f (0.1.5),
where null-termination of r->uri and many other strings was removed.
2020-12-10 20:09:39 +03:00
Maxim Dounin
ce9971b2b5 Fixed parsing of absolute URIs with empty path (ticket #2079).
When the request line contains request-target in the absolute-URI form,
it can contain path-empty instead of a single slash (see RFC 7230, RFC 3986).
Previously, the ngx_http_parse_request_line() function only accepted empty
path when there was no query string.

With this change, non-empty query is also correctly handled.  That is,
request line "GET http://example.com?foo HTTP/1.1" is accepted and results
in $uri "/" and $args "foo".

Note that $request_uri remains "?foo", similarly to how spaces in URIs
are handled.  Providing "/?foo", similarly to how "/" is provided for
"GET http://example.com HTTP/1.1", requires allocation.
2020-12-10 20:09:30 +03:00
Roman Arutyunyan
240f8a918e QUIC: always calculate rtt for largest acknowledged packet.
Previously, when processing client ACK, rtt could be calculated for a packet
different than the largest if it was missing in the sent chain.  Even though
this is an unlikely situation, rtt based on a different packet could be larger
than needed leading to bigger pto timeout and performance degradation.
2020-12-09 21:26:21 +00:00
Roman Arutyunyan
4fd02c5839 QUIC: send and process ack_delay for Initial and Handshake levels.
Previously, this only worked for Application level because before
quic-transport-30, there were the following constraints:

   Because the receiver doesn't use the ACK Delay for Initial and Handshake
   packets, a sender SHOULD send a value of 0.

   When adjusting an RTT sample using peer-reported acknowledgement delays, an
   endpoint ...  MUST ignore the ACK Delay field of the ACK frame for packets
   sent in the Initial and Handshake packet number space.
2020-12-10 14:54:53 +00:00
Roman Arutyunyan
6f9efd423e QUIC: use client max_ack_delay when computing pto timeout.
Previously, server max_ack_delay was used which is wrong.

Also, level check is simplified.
2020-12-09 16:15:24 +00:00
Roman Arutyunyan
e5c10dce5e QUIC: resend handshake packets along with initial.
To speed up handshake, resend both initial and handshake packets if there's
at least one unacknowledged initial packet.
2020-12-08 17:10:22 +00:00
Roman Arutyunyan
fc3f04b111 QUIC: set the temporary flag for input frame buffers.
Missing flag prevented frame data from being copied as the buffer was not
considered a memory buffer.
2020-12-08 14:44:41 +00:00
Ruslan Ermilov
e62a5132ca SSL: fixed SSL shutdown on lingering close.
Ensure c->recv is properly reset to ngx_recv if SSL_shutdown()
blocks on writing.

The bug had appeared in 554c6ae25ffc.
2020-12-08 01:43:36 +03:00
Roman Arutyunyan
c9cbd2f8e7 QUIC: coalesce output packets into a single UDP datagram.
Now initial output packet is not padded anymore if followed by a handshake
packet.  If the datagram is still not big enough to satisfy minimum size
requirements, handshake packet is padded.
2020-12-07 15:09:08 +00:00
Roman Arutyunyan
ec99ccee36 QUIC: introduced QUIC buffers.
Buffers are used to hold frame data.  They have a fixed size and are reused
after being freed.
2020-12-01 19:11:01 +00:00
Vladimir Homutov
ed203729ad QUIC: fixed handling of clients connected to wildcard address.
The patch replaces c->send() occurences with c->send_chain(), because the
latter accounts for the local address, which may be different if the wildcard
listener is used.

Previously, server sent response to client using address different from
one client connected to.
2020-12-07 14:06:00 +03:00
Sergey Kandaurov
1d748f1ca3 QUIC: disabling bidirectional SSL shutdown earlier.
Notably, this fixes an issue with Chrome that can emit a "certificate_unknown"
alert during the SSL handshake where c->ssl->no_wait_shutdown is not yet set.
2020-12-06 14:24:38 +00:00
Vladimir Homutov
90ec7ef6db QUIC: fixed missing quic flag on listener in the stream module. 2020-12-04 15:19:03 +03:00
Roman Arutyunyan
4b440cbf97 HTTP/3: introduced ngx_http_v3_filter.
The filter is responsible for creating HTTP/3 response header and body.

The change removes differences to the default branch for
ngx_http_chunked_filter_module and ngx_http_header_filter_module.
2020-11-27 17:46:21 +00:00