Commit Graph

7794 Commits

Author SHA1 Message Date
Maxim Dounin
2187586207 Xslt: fixed potential buffer overflow with null character.
Due to shortcomings of the ccv->zero flag implementation in complex value
interface, length of the resulting string from ngx_http_complex_value()
might either not include terminating null character or include it,
so the only safe way to work with the result is to use it as a
null-terminated string.

Reported by Patrick Wollgast.
2019-07-18 18:27:54 +03:00
Maxim Dounin
ad42d70fed SSI: avoid potential buffer overflow.
When "-" follows a parameter of maximum length, a single byte buffer
overflow happens, since the error branch does not check parameter length.
Fix is to avoid saving "-" to the parameter key, and instead use an error
message with "-" explicitly written.  The message is mostly identical to
one used in similar cases in the preequal state.

Reported by Patrick Wollgast.
2019-07-18 18:27:53 +03:00
Maxim Dounin
20c8c4fe35 Upstream: fixed EOF handling in unbuffered and upgraded modes.
With level-triggered event methods it is important to specify
the NGX_CLOSE_EVENT flag to ngx_handle_read_event(), otherwise
the event won't be removed, resulting in CPU hog.

Reported by Patrick Wollgast.
2019-07-18 18:27:52 +03:00
Maxim Dounin
36dfa020f2 HTTP/2: return error on output on closed stream.
Without this, an (incorrect) output on a closed stream could result in
a socket leak.
2019-07-18 18:27:50 +03:00
Maxim Dounin
551640703a Core: fixed segfault with too large bucket sizes (ticket #1806).
To save memory hash code uses u_short to store resulting bucket sizes,
so maximum bucket size is limited to 65536 minus ngx_cacheline_size (larger
values will be aligned to 65536 which will overflow u_short).  However,
there were no checks to enforce this, and using larger bucket sizes
resulted in overflows and segmentation faults.

Appropriate safety checks to enforce this added to ngx_hash_init().
2019-07-18 18:27:44 +03:00
Maxim Dounin
676d1a0e94 Perl: removed unused variable, forgotten in 975d7ab37b39. 2019-07-17 17:00:57 +03:00
Ilya Leoshkevich
cfa1316368 Gzip: use zlib to write header and trailer.
When nginx is used with zlib patched with [1], which provides
integration with the future IBM Z hardware deflate acceleration, it ends
up computing CRC32 twice: one time in hardware, which always does this,
and one time in software by explicitly calling crc32().

crc32() calls were added in changesets 133:b27548f540ad ("nginx-0.0.1-
2003-09-24-23:51:12 import") and 134:d57c6835225c ("nginx-0.0.1-
2003-09-26-09:45:21 import") as part of gzip wrapping feature - back
then zlib did not support it.

However, since then gzip wrapping was implemented in zlib v1.2.0.4,
and it's already being used by nginx for log compression.

This patch replaces hand-written gzip wrapping with the one provided by
zlib. It simplifies the code, and makes it avoid computing CRC32 twice
when using hardware acceleration.

[1] https://github.com/madler/zlib/pull/410
2019-07-12 12:43:08 +02:00
Maxim Dounin
29fea7d9ec Perl: named locations in $r->internal_redirect(). 2019-07-12 15:39:28 +03:00
Maxim Dounin
8df08b02b8 Perl: expect escaped URIs in $r->internal_redirect().
Similarly to the change in 5491:74bfa803a5aa (1.5.9), we should accept
properly escaped URIs and unescape them as needed, else it is not possible
to handle URIs with question marks.
2019-07-12 15:39:26 +03:00
Maxim Dounin
9e883a2e48 Perl: additional ctx->header_sent checks.
As we now have ctx->header_sent flag, it is further used to prevent
duplicate $r->send_http_header() calls, prevent output before sending
header, and $r->internal_redirect() after sending header.

Further, $r->send_http_header() protected from calls after
$r->internal_redirect().
2019-07-12 15:39:25 +03:00
Maxim Dounin
78b39bd631 Perl: avoid returning 500 if header was already sent.
Returning NGX_HTTP_INTERNAL_SERVER_ERROR if a perl code died after
sending header will lead to a "header already sent" alert.  To avoid
it, we now check if header was already sent, and return NGX_ERROR
instead if it was.
2019-07-12 15:39:25 +03:00
Maxim Dounin
12d6b3b4a1 Perl: avoid redirects on errors.
Previously, redirects scheduled with $r->internal_redirect() were followed
even if the code then died.  Now these are ignored and nginx will return
an error instead.
2019-07-12 15:38:27 +03:00
Maxim Dounin
cae2e68908 Perl: disabled unrelated calls from variable handlers.
Variable handlers are not expected to send anything to the client, cannot
sleep or read body, and are not expected to modify the request.  Added
appropriate protection to prevent accidental foot shooting.
2019-07-12 15:35:31 +03:00
Maxim Dounin
1988783169 Perl: protection against duplicate $r->sleep() calls.
Duplicate $r->sleep() and/or $r->has_request_body() calls result
in undefined behaviour (in practice, connection leaks were observed).
To prevent this, croak() added in appropriate places.
2019-07-12 15:34:37 +03:00
Maxim Dounin
9d266efbc0 Perl: handling of allocation errors.
Previously, allocation errors in nginx.xs were more or less ignored,
potentially resulting in incorrect code execution in specific low-memory
conditions.  This is changed to use ctx->error bit and croak(), similarly
to how output errors are now handled.

Note that this is mostly a cosmetic change, as Perl itself exits on memory
allocation errors, and hence nginx with Perl is hardly usable in low-memory
conditions.
2019-07-12 13:56:23 +03:00
Maxim Dounin
4a0771f9a6 Perl: propagate errors.
When an error happens, the ctx->error bit is now set, and croak()
is called to terminate further processing.  The ctx->error bit is
checked in ngx_http_perl_call_handler() to cancel further processing,
and is also checked in various output functions - to make sure these won't
be called if croak() was handled by an eval{} in perl code.

In particular, this ensures that output chain won't be called after
errors, as filters might not expect this to happen.  This fixes some
segmentation faults under low memory conditions.  Also this stops
request processing after filter finalization or request body reading
errors.

For cases where an HTTP error status can be additionally returned (for
example, 416 (Requested Range Not Satisfiable) from the range filter),
the ctx->status field is also added.
2019-07-12 13:56:21 +03:00
Maxim Dounin
eae5e4dd01 Perl: reworked perl module to pass ctx instead of request.
This ensures that correct ctx is always available, including after
filter finalization.  In particular, this fixes a segmentation fault
with the following configuration:

    location / {
        image_filter test;

        perl 'sub {
            my $r = shift;
            $r->send_http_header();
            $r->print("foo\n");
            $r->print("bar\n");
        }';
    }

This also seems to be the only way to correctly handle filter finalization
in various complex cases, for example, when embedded perl is used both
in the original handler and in an error page called after filter
finalization.
2019-07-12 11:29:22 +03:00
Maxim Dounin
60e7480533 Perl: removed unneeded NGX_DONE test.
The NGX_DONE test in ngx_http_perl_handle_request() was introduced
in 1702:86bb52e28ce0, which also modified ngx_http_perl_call_handler()
to return NGX_DONE with c->destroyed.  The latter part was then
removed in 3050:f54b02dbb12b, so NGX_DONE test is no longer needed.
2019-07-11 23:20:08 +03:00
Gena Makhomed
1e673c72dd Contrib: vim syntax, update core and 3rd party module directives. 2019-06-30 10:39:01 +03:00
Maxim Dounin
18e063129a Typo. 2019-07-09 16:03:25 +03:00
Maxim Dounin
66e95b50e7 Version bump. 2019-07-09 16:01:32 +03:00
Maxim Dounin
4722452e9c release-1.17.1 tag 2019-06-25 15:19:45 +03:00
Maxim Dounin
c4613796af nginx-1.17.1-RELEASE 2019-06-25 15:19:45 +03:00
Maxim Dounin
32adc8534c Updated OpenSSL used for win32 builds. 2019-06-25 04:47:43 +03:00
Maxim Dounin
d9887ee2ae Perl: disabled not_modified filter (ticket #1786).
Embedded perl does not set any request fields needed for conditional
requests processing.  Further, filter finalization in the not_modified
filter can cause segmentation faults due to cleared ctx as in
ticket #1786.

Before 5fb1e57c758a (1.7.3) the not_modified filter was implicitly disabled
for perl responses, as r->headers_out.last_modified_time was -1.  This
change restores this behaviour by using the explicit r->disable_not_modified
flag.

Note that this patch doesn't try to address perl module robustness against
filter finalization and other errors returned from filter chains.  It should
be eventually reworked to handle errors instead of ignoring them.
2019-06-17 19:48:56 +03:00
Roman Arutyunyan
dda58fc63b Limit req: limit_req_dry_run directive.
A new directive limit_req_dry_run allows enabling the dry run mode.  In this
mode requests are neither rejected nor delayed, but reject/delay status is
logged as usual.
2019-06-05 19:55:27 +03:00
Roman Arutyunyan
16ebfa999b Upstream: background cache update before cache send (ticket #1782).
In case of filter finalization, essential request fields like r->uri,
r->args etc could be changed, which affected the cache update subrequest.
Also, after filter finalization r->cache could be set to NULL, leading to
null pointer dereference in ngx_http_upstream_cache_background_update().
The fix is to create background cache update subrequest before sending the
cached response.

Since initial introduction in 1aeaae6e9446 (1.11.10) background cache update
subrequest was created after sending the cached response because otherwise it
blocked the parent request output.  In 9552758a786e (1.13.1) background
subrequests were introduced to eliminate the delay before sending the final
part of the cached response.  This also made it possible to create the
background cache update subrequest before sending the response.

Note that creating the subrequest earlier does not change the fact that in case
of filter finalization the background cache update subrequest will likely not
have enough time to successfully update the cache entry.  Filter finalization
leads to the main request termination as soon the current iteration of request
processing is complete.
2019-06-03 20:33:26 +03:00
Niklas Keller
c18658e9fe Upstream hash: fall back to round-robin if hash key is empty. 2019-05-23 16:49:22 +03:00
Roman Arutyunyan
e0768d199d Version bump. 2019-05-27 19:47:50 +03:00
Maxim Dounin
b007118fe1 release-1.17.0 tag 2019-05-21 17:23:57 +03:00
Maxim Dounin
8a779e2067 nginx-1.17.0-RELEASE 2019-05-21 17:23:57 +03:00
Sergey Kandaurov
ed0b19cdd4 Cache: reset c->body_start when reading a variant on Vary mismatch.
Previously, a variant not present in shared memory and stored on disk using a
secondary key was read using c->body_start from a variant stored with a main
key.  This could result in critical errors "cache file .. has too long header".
2017-08-04 19:37:37 +03:00
Sergey Kandaurov
b1274232db HTTP/2: rejecting invalid stream identifiers with PROTOCOL_ERROR.
Prodded by Xu Yang.
2020-09-02 23:13:36 +03:00
Roman Arutyunyan
554916301c Cache: ignore stale-if-error for 4xx and 5xx codes.
Previously the stale-if-error extension of the Cache-Control upstream header
triggered the return of a stale response for all error conditions that can be
specified in the proxy_cache_use_stale directive.  The list of these errors
includes both network/timeout/format errors, as well as some HTTP codes like
503, 504, 403, 429 etc.  The latter prevented a cache entry from being updated
by a response with any of these HTTP codes during the stale-if-error period.

Now stale-if-error only works for network/timeout/format errors and ignores
the upstream HTTP code.  The return of a stale response for certain HTTP codes
is still possible using the proxy_cache_use_stale directive.

This change also applies to the stale-while-revalidate extension of the
Cache-Control header, which triggers stale-if-error if it is missing.

Reported at
http://mailman.nginx.org/pipermail/nginx/2020-July/059723.html.
2020-07-29 13:28:04 +03:00
Roman Arutyunyan
d961012799 Version bump. 2020-08-14 12:45:52 +03:00
Sergey Kandaurov
c17bc31d41 SSL: removed OpenSSL 0.9.7 compatibility. 2016-04-11 15:46:36 +03:00
Maxim Dounin
2015a54821 Range filter: fixed duplicate last buffers.
In ngx_http_range_singlepart_body() special buffers where passed
unmodified, including ones after the end of the range.  As such,
if the last buffer of a response was sent separately as a special
buffer, two buffers with b->last_buf set were present in the response.

In particular, this might result in a duplicate final chunk when using
chunked transfer encoding (normally range filter and chunked transfer
encoding are not used together, but this may happen if there are trailers
in the response).  This also likely to cause problems in HTTP/2.

Fix is to skip all special buffers after we've sent the last part of
the range requested.  These special buffers are not meaningful anyway,
since we set b->last_buf in the buffer with the last part of the range,
and everything is expected to be flushed due to it.

Additionally, ngx_http_next_body_filter() is now called even
if no buffers are to be passed to it.  This ensures that various
write events are properly propagated through the filter chain.  In
particular, this fixes test failures observed with the above change
and aio enabled.
2019-05-13 22:44:49 +03:00
Maxim Dounin
ce0f35317e Range filter: fixed loss of incoming chain links.
Filters are not allowed to change incoming chain links, and should allocate
their own links if any modifications are needed.  Nevertheless
ngx_http_range_singlepart_body() modified incoming chain links in some
cases, notably at the end of the requested range.

No problems caused by this are currently known, mostly because of
limited number of possible modifications and the position of the range
body filter in the filter chain.  Though this behaviour is clearly incorrect
and tests demonstrate that it can at least cause some proxy buffers being
lost when using proxy_force_ranges, leading to less effective handling
of responses.

Fix is to always allocate new chain links in ngx_http_range_singlepart_body().
Links are explicitly freed to ensure constant memory usage with long-lived
requests.
2019-05-13 22:44:02 +03:00
Roman Arutyunyan
a39380a41e Unconditional compilation of the postpone filter.
Postpone filter is an essential part of subrequest functionality.  In absence
of it a subrequest response body is sent to the client out of order with
respect to the main request header and body, as well as other subrequests.
For in-memory subrequests the response is also sent to the client instead of
being stored in memory.

Currently the postpone filter is automatically enabled if one of the following
standard modules which are known to create subrequests is enabled: ssi, slice,
addition.  However a third-party module that creates subrequests can still be
built without the postpone filter or be dynamically loaded in nginx built
without it.
2019-05-08 19:22:13 +03:00
Ruslan Ermilov
27b3d3dcca Variables support in proxy_upload_rate and proxy_download_rate. 2019-04-24 16:38:56 +03:00
Ruslan Ermilov
0e2653877e Variables support in limit_rate and limit_rate_after (ticket #293). 2019-04-24 16:38:54 +03:00
Ruslan Ermilov
2ace7fc3e6 Added ngx_http_set_complex_value_size_slot().
If a complex value is expected to be of type size_t, and the compiled
value is constant, the constant size_t value is remembered at compile
time.

The value is accessed through ngx_http_complex_value_size() which
either returns the remembered constant or evaluates the expression
and parses it as size_t.
2019-04-24 16:38:51 +03:00
Vladimir Homutov
2eb2a93d8a Core: enabled "include" in any context (ticket #1615). 2019-04-09 11:40:20 +03:00
Vladimir Homutov
68b147535a Version bump. 2019-04-24 13:41:29 +03:00
Maxim Dounin
d2cf390fc0 release-1.15.12 tag 2019-04-16 17:54:59 +03:00
Maxim Dounin
baa377ce76 nginx-1.15.12-RELEASE 2019-04-16 17:54:58 +03:00
Maxim Dounin
4ca32d6eb3 Updated PCRE used for win32 builds. 2019-04-16 16:32:44 +03:00
Maxim Dounin
f09eae2a75 Fixed incorrect length handling in ngx_utf8_length().
Previously, ngx_utf8_decode() was called from ngx_utf8_length() with
incorrect length, potentially resulting in out-of-bounds read when
handling invalid UTF-8 strings.

In practice out-of-bounds reads are not possible though, as autoindex, the
only user of ngx_utf8_length(), provides null-terminated strings, and
ngx_utf8_decode() anyway returns an errors when it sees a null in the
middle of an UTF-8 sequence.

Reported by Yunbin Liu.
2019-04-15 20:14:07 +03:00
Maxim Dounin
5784889fb9 OCSP stapling: fixed segfault with dynamic certificate loading.
If OCSP stapling was enabled with dynamic certificate loading, with some
OpenSSL versions (1.0.2o and older, 1.1.0h and older; fixed in 1.0.2p,
1.1.0i, 1.1.1) a segmentation fault might happen.

The reason is that during an abbreviated handshake the certificate
callback is not called, but the certificate status callback was called
(https://github.com/openssl/openssl/issues/1662), leading to NULL being
returned from SSL_get_certificate().

Fix is to explicitly check SSL_get_certificate() result.
2019-04-15 19:13:09 +03:00
Maxim Dounin
aaa1a57060 Version bump. 2019-04-15 19:13:06 +03:00