Commit Graph

3191 Commits

Author SHA1 Message Date
Sergey Kandaurov
c622ff3b44 Fixed logging with variable field width. 2016-03-31 02:33:53 +03:00
Sergey Kandaurov
2e25f340b8 Fixed logging in close error handling. 2016-03-31 02:33:50 +03:00
Ruslan Ermilov
7ad57da598 Style. 2016-03-30 11:52:16 +03:00
Maxim Dounin
4618c03f4c Upstream: proxy_next_upstream non_idempotent.
By default, requests with non-idempotent methods (POST, LOCK, PATCH)
are no longer retried in case of errors if a request was already sent
to a backend.  Previous behaviour can be restored by using
"proxy_next_upstream ... non_idempotent".
2016-03-28 19:50:19 +03:00
Maxim Dounin
5a76856dc2 Upstream: cached connections now tested against next_upstream.
Much like normal connections, cached connections are now tested against
u->conf->next_upstream, and u->state->status is now always set.

This allows to disable additional tries even with upstream keepalive
by using "proxy_next_upstream off".
2016-03-28 19:49:52 +03:00
Roman Arutyunyan
3d791c46f5 Sub filter: fixed allocation alignment. 2016-03-28 13:35:25 +03:00
Dmitry Volyntsev
f625ad7ce8 Resolver: added support for SRV records. 2016-03-23 17:44:36 +03:00
Maxim Dounin
890ba54497 Backed out server_tokens changes.
Backed out changesets: cf3e75cfa951, 6b72414dfb4f, 602dc42035fe, e5076b96fd01.
2016-03-22 16:58:38 +03:00
Dmitry Volyntsev
24e38323ee Cache: fixed slots accounting error introduced in c9d680b00744. 2016-03-23 14:16:31 +03:00
Ruslan Ermilov
0b768c1435 Reconsidered server_tokens with an empty value.
An empty value will be treated as "off".
2016-03-22 15:52:28 +03:00
Valentin Bartenev
ae5e76ea06 HTTP/2: improved debugging of sending control frames. 2016-03-21 17:04:21 +03:00
Dmitry Volyntsev
b18ba1e9f6 Cache: added watermark to reduce IO load when keys_zone is full.
When a keys_zone is full then each next request to the cache is
penalized.  That is, the cache has to evict older files to get a
slot from the keys_zone synchronously.  The patch introduces new
behavior in this scenario.  Manager will try to maintain available
free slots in the keys_zone by cleaning old files in the background.
2016-03-18 15:08:21 +03:00
Dmitry Volyntsev
861e75dc20 Cache: report error if slab allocator fails during cache loading. 2016-03-18 14:27:30 +03:00
Maxim Dounin
348f705c00 Threads: writing via threads pools in event pipe.
The "aio_write" directive is introduced, which enables use of aio
for writing.  Currently it is meaningful only with "aio threads".

Note that aio operations can be done by both event pipe and output
chain, so proper mapping between r->aio and p->aio is provided when
calling ngx_event_pipe() and in output filter.

In collaboration with Valentin Bartenev.
2016-03-18 06:44:49 +03:00
Maxim Dounin
ac78b26324 Threads: task pointer stored in ngx_file_t.
This simplifies the interface of the ngx_thread_read() function.

Additionally, most of the thread operations now explicitly set
file->thread_task, file->thread_handler and file->thread_ctx,
to facilitate use of thread operations in other places.

(Potential problems remain with sendfile in threads though - it uses
file->thread_handler as set in ngx_output_chain(), and it should not
be overwritten to an incompatible one.)

In collaboration with Valentin Bartenev.
2016-03-18 06:43:52 +03:00
Ruslan Ermilov
07fc7dc719 Style. 2016-03-16 14:44:41 +03:00
Ruslan Ermilov
8f53f6f626 Win32: fixed build after cf3e75cfa951. 2016-03-15 15:15:30 +03:00
Ruslan Ermilov
3d9ce2d601 Win32: fixed build after cf3e75cfa951. 2016-03-15 15:10:41 +03:00
Ruslan Ermilov
86927c6588 Added variables support to server_tokens.
It can now be set to "off" conditionally, e.g. using the map
directive.

An empty value will disable the emission of the Server: header
and the signature in error messages generated by nginx.

Any other value is treated as "on", meaning that full nginx
version is emitted in the Server: header and error messages
generated by nginx.
2016-03-15 13:36:19 +03:00
Justin Li
573ec98d24 Upstream: avoid closing client connection in edge case.
If proxy_cache is enabled, and proxy_no_cache tests true, it was previously
possible for the client connection to be closed after a 304. The fix is to
recheck r->header_only after the final cacheability is determined, and end the
request if no longer cacheable.

Example configuration:

proxy_cache foo;
proxy_cache_bypass 1;
proxy_no_cache 1;

If a client sends If-None-Match, and the upstream server returns 200 with a
matching ETag, no body should be returned to the client. At the start of
ngx_http_upstream_send_response proxy_no_cache is not yet tested, thus cacheable
is still 1 and downstream_error is set.

However, by the time the downstream_error check is done in process_request,
proxy_no_cache has been tested and cacheable is set to 0. The client connection
is then closed, regardless of keepalive.
2016-03-08 22:31:55 -05:00
Maxim Dounin
2aa6d7fd65 Upstream: fixed "zero size buf" alerts with cache (ticket #918).
If caching was used, "zero size buf in output" alerts might appear
in logs if a client prematurely closed connection.  Alerts appeared
in the following situation:

- writing to client returned an error, so event pipe
  drained all busy buffers leaving body output filters
  in an invalid state;

- when upstream response was fully received,
  ngx_http_upstream_finalize_request() tried to flush
  all pending data.

Fix is to avoid flushing body if p->downstream_error is set.
2016-03-10 21:58:03 +03:00
Maxim Dounin
00fcc6a987 Copy filter: fixed sendfile aio handlers to set ctx->aio.
Sendfile handlers (aio preload and thread handler) are called within
ctx->output_filter() in ngx_output_chain(), and hence ctx->aio cannot
be set directly in ngx_output_chain().  Meanwhile, it must be set to
make sure loop within ngx_output_chain() will be properly terminated.

There are no known cases that trigger the problem, though in theory
something like aio + sub filter (something that needs body in memory,
and can also free some memory buffers) + sendfile can result in
"task already active" and "second aio post" alerts.

The fix is to set ctx->aio in ngx_http_copy_aio_sendfile_preload()
and ngx_http_copy_thread_handler().

For consistency, ctx->aio is no longer set explicitly in
ngx_output_chain_copy_buf(), as it's now done in
ngx_http_copy_thread_handler().
2016-03-03 21:14:19 +03:00
Valentin Bartenev
bc6fcb672c Request body: moved handling of the last part in the save filter.
No functional changes.
2016-03-01 15:18:07 +03:00
Valentin Bartenev
f72bcf8285 HTTP/2: implemented per request timeouts (closes #626).
Previously, there were only three timeouts used globally for the whole HTTP/2
connection:

 1. Idle timeout for inactivity when there are no streams in processing
    (the "http2_idle_timeout" directive);

 2. Receive timeout for incomplete frames when there are no streams in
    processing (the "http2_recv_timeout" directive);

 3. Send timeout when there are frames waiting in the output queue
    (the "send_timeout" directive on a server level).

Reaching one of these timeouts leads to HTTP/2 connection close.

This left a number of scenarios when a connection can get stuck without any
processing and timeouts:

 1. A client has sent the headers block partially so nginx starts processing
    a new stream but cannot continue without the rest of HEADERS and/or
    CONTINUATION frames;

 2. When nginx waits for the request body;

 3. All streams are stuck on exhausted connection or stream windows.

The first idea that was rejected was to detect when the whole connection
gets stuck because of these situations and set the global receive timeout.
The disadvantage of such approach would be inconsistent behaviour in some
typical use cases.  For example, if a user never replies to the browser's
question about where to save the downloaded file, the stream will be
eventually closed by a timeout.  On the other hand, this will not happen
if there's some activity in other concurrent streams.

Now almost all the request timeouts work like in HTTP/1.x connections, so
the "client_header_timeout", "client_body_timeout", and "send_timeout" are
respected.  These timeouts close the request.

The global timeouts work as before.

Previously, the c->write->delayed flag was abused to avoid setting timeouts on
stream events.  Now, the "active" and "ready" flags are manipulated instead to
control the processing of individual streams.
2016-02-24 15:58:07 +03:00
Valentin Bartenev
4e6a490fa7 HTTP/2: always use temporary pool for processing headers.
This is required for implementing per request timeouts.

Previously, the temporary pool was used only during skipping of
headers and the request pool was used otherwise.  That required
switching of pools if the request was closed while parsing.

It wasn't a problem since the request could be closed only after
the validation of the fully parsed header.  With the per request
timeouts, the request can be closed at any moment, and switching
of pools in the middle of parsing header name or value becomes a
problem.

To overcome this, the temporary pool is now always created and
used.  Special checks are added to keep it when either the stream
is being processed or until header block is fully parsed.
2016-02-24 16:05:47 +03:00
Valentin Bartenev
8b40f1eaec HTTP/2: cleaned up state while closing stream.
Without this the state might keep pointing to already closed stream.
2016-02-24 16:05:46 +03:00
Valentin Bartenev
1d294eea3e Fixed buffer over-read while logging invalid request headers.
Since 667aaf61a778 (1.1.17) the ngx_http_parse_header_line() function can return
NGX_HTTP_PARSE_INVALID_HEADER when a header contains NUL character.  In this
case the r->header_end pointer isn't properly initialized, but the log message
in ngx_http_process_request_headers() hasn't been adjusted.  It used the pointer
in size calculation, which might result in up to 2k buffer over-read.

Found with afl-fuzz.
2016-02-24 16:01:23 +03:00
Ruslan Ermilov
73f815e489 Headers filter: fixed "add_header ... '' always".
The "always" parameter was ignored if the header value was empty.
2016-02-18 15:49:11 +03:00
Valentin Bartenev
b5d7d3f024 HTTP/2: added debug logging of response headers.
Because of HPACK compression it's hard to see what headers are actually
sent by the server.
2016-02-16 17:49:14 +03:00
Valentin Bartenev
fcfe483292 HTTP/2: use local pointer instead of r->connection.
No functional changes.
2016-02-16 17:49:14 +03:00
Valentin Bartenev
822fc91b09 HTTP/2: fixed undefined behavior in ngx_http_v2_huff_encode().
When the "pending" value is zero, the "buf" will be right shifted
by the width of its type, which results in undefined behavior.

Found by Coverity (CID 1352150).
2016-02-12 16:36:20 +03:00
Valentin Bartenev
531e6fbfd6 HTTP/2: implemented HPACK Huffman encoding for response headers.
This reduces the size of headers by over 30% on average.

Based on the patch by Vlad Krasnov:
http://mailman.nginx.org/pipermail/nginx-devel/2015-December/007682.html
2016-02-11 15:35:36 +03:00
Maxim Dounin
0f67d6355c Dynamic modules: changed ngx_modules to cycle->modules. 2016-02-04 18:30:21 +03:00
Maxim Dounin
7988334570 Dynamic modules: moved module-related stuff to separate files. 2016-02-04 18:30:21 +03:00
Valentin Bartenev
9add42c71e HTTP/2: fixed possible buffer overrun (ticket #893).
Due to greater priority of the unary plus operator over the ternary operator
the expression didn't work as expected.  That might result in one byte less
allocation than needed for the HEADERS frame buffer.
2016-02-04 18:01:04 +03:00
Valentin Bartenev
cb173ff672 HTTP/2: fixed padding handling in HEADERS frame with CONTINUATION. 2016-02-02 16:33:55 +03:00
Valentin Bartenev
8050277acf HTTP/2: fixed request length accounting.
Now it includes not only the received body size,
but the size of headers block as well.
2016-02-02 16:33:55 +03:00
Valentin Bartenev
405f4f99b4 HTTP/2: fixed excessive memory allocation for pool cleanup. 2016-02-02 16:33:55 +03:00
Valentin Bartenev
3351fbe481 HTTP/2: removed unused field from ngx_http_v2_stream_t. 2016-02-02 16:33:55 +03:00
Valentin Bartenev
8a5df8c720 Request body: removed surplus assigment, no functional changes.
Setting rb->bufs to NULL is surplus after ngx_http_write_request_body()
has returned NGX_OK.
2016-01-12 19:19:07 +03:00
Maxim Dounin
af647a3da2 Upstream: fixed changing method on X-Accel-Redirect.
Previously, only r->method was changed, resulting in handling of a request
as GET within nginx itself, but not in requests to proxied servers.

See http://mailman.nginx.org/pipermail/nginx/2015-December/049518.html.
2016-01-11 19:08:12 +03:00
Maxim Dounin
fda7d021ca Upstream: don't keep connections on early responses (ticket #669). 2015-12-17 16:39:15 +03:00
Maxim Dounin
9973e36944 Fixed PROXY protocol on IPv6 sockets (ticket #858). 2015-12-17 16:39:02 +03:00
Ruslan Ermilov
5e7535df72 Fixed fastcgi_pass with UNIX socket and variables (ticket #855).
This was broken in a93345ee8f52 (1.9.8).
2015-12-09 16:26:59 +03:00
Roman Arutyunyan
56fda3265b Slice filter: terminate first slice with last_in_chain flag.
This flag makes sub filter flush buffered data and optimizes allocation in copy
filter.
2015-12-08 17:39:56 +03:00
Roman Arutyunyan
5c1f76f0e9 Slice filter: never run subrequests when main request is buffered.
With main request buffered, it's possible, that a slice subrequest will send
output before it.  For example, while main request is waiting for aio read to
complete, a slice subrequest can start an aio operation as well.  The order
in which aio callbacks are called is undetermined.
2015-12-08 17:39:56 +03:00
Sergey Kandaurov
79fcf261d0 SSL: fixed possible segfault on renegotiation (ticket #845).
Skip SSL_CTX_set_tlsext_servername_callback in case of renegotiation.
Do nothing in SNI callback as in this case it will be supplied with
request in c->data which isn't expected and doesn't work this way.

This was broken by b40af2fd1c16 (1.9.6) with OpenSSL master branch and LibreSSL.
2015-12-08 16:59:43 +03:00
Roman Arutyunyan
8ba626ccd7 Slice filter.
Splits a request into subrequests, each providing a specific range of response.
The variable "$slice_range" must be used to set subrequest range and proper
cache key.  The directive "slice" sets slice size.

The following example splits requests into 1-megabyte cacheable subrequests.

server {
    listen 8000;

    location / {
        slice 1m;

        proxy_cache cache;
        proxy_cache_key $uri$is_args$args$slice_range;
        proxy_set_header Range $slice_range;
        proxy_cache_valid 200 206 1h;
        proxy_pass http://127.0.0.1:9000;
    }
}
2015-12-07 16:30:48 +03:00
Roman Arutyunyan
1b478c50de Upstream: fill r->headers_out.content_range from upstream response. 2015-12-07 16:30:47 +03:00
Maxim Dounin
6994593478 Style: NGX_PTR_SIZE replaced with sizeof(void *).
The NGX_PTR_SIZE macro is only needed in preprocessor directives where
it's not possible to use sizeof().
2015-12-03 20:06:45 +03:00
Maxim Dounin
af4e89b9a1 Style. 2015-12-02 01:06:54 +03:00
Ruslan Ermilov
362fb6c1d4 Stop emulating a space character after r->method_name.
This is an API change.

The proxy module was modified to not depend on this in 44122bddd9a1.
No known third-party modules seem to depend on this.
2015-11-30 12:54:01 +03:00
Ruslan Ermilov
05a38c7077 Proxy: improved code readability.
Do not assume that space character follows the method name, just pass it
explicitly.

The fuss around it has already proved to be unsafe, see bbdb172f0927 and
http://mailman.nginx.org/pipermail/nginx-ru/2013-January/049692.html for
details.
2015-11-06 15:21:51 +03:00
Ruslan Ermilov
f0f7b437b9 Reduced the number of GET method constants. 2015-11-30 12:04:35 +03:00
Valentin Bartenev
a89548a4e1 Increased the default "connection_pool_size" on 64-bit platforms.
The previous default of 256 bytes isn't enough and results in two
allocations on each accepted connection, which is suboptimal.
2015-11-30 16:27:33 +03:00
Ruslan Ermilov
5ea66922d9 Style: unified request method checks. 2015-11-06 15:22:43 +03:00
Ruslan Ermilov
72b9a317d2 Upstream: fixed "no port" detection in evaluated upstreams.
If an upstream with variables evaluated to address without a port,
then instead of a "no port in upstream" error an attempt was made
to connect() which failed with EADDRNOTAVAIL.
2015-11-21 10:44:07 +03:00
Ruslan Ermilov
1ce1610763 Realip: the $realip_remote_addr variable. 2015-11-16 16:02:02 +03:00
Valentin Bartenev
a52bbefd84 HTTP/2: reused HEADERS and CONTINUATION frames buffers. 2015-11-13 20:10:50 +03:00
Valentin Bartenev
548e31f608 HTTP/2: fixed handling of output HEADERS frames.
The HEADERS frame is always represented by more than one buffer since
b930e598a199, but the handling code hasn't been adjusted.

Only the first buffer of HEADERS frame was checked and if it had been
sent while others had not, the rest of the frame was dropped, resulting
in broken connection.

Before b930e598a199, the problem could only be seen in case of HEADERS
frame with CONTINUATION.
2015-11-13 20:10:50 +03:00
Valentin Bartenev
8323cd693b HTTP/2: fixed invalid headers handling (ticket #831).
The r->invalid_header flag wasn't reset once an invalid header appeared in a
request, resulting in all subsequent headers in the request were also marked
as invalid.
2015-11-13 20:10:50 +03:00
Roman Arutyunyan
89a049be89 Upstream: proxy_cache_convert_head directive.
The directive toggles conversion of HEAD to GET for cacheable proxy requests.
When disabled, $request_method must be added to cache key for consistency.
By default, HEAD is converted to GET as before.
2015-11-11 15:47:30 +03:00
Valentin Bartenev
93aef089b4 SSL: only select HTTP/2 using NPN if "http2" is enabled.
OpenSSL doesn't check if the negotiated protocol has been announced.
As a result, the client might force using HTTP/2 even if it wasn't
enabled in configuration.
2015-11-05 15:01:09 +03:00
Valentin Bartenev
b22c0e0846 HTTP/2: backed out 16905ecbb49e (ticket #822).
It caused inconsistency between setting "in_closed" flag and the moment when
the last DATA frame was actually read.  As a result, the body buffer might not
be initialized properly in ngx_http_v2_init_request_body(), which led to a
segmentation fault in ngx_http_v2_state_read_data().  Also it might cause
start processing of incomplete body.

This issue could be triggered when the processing of a request was delayed,
e.g. in the limit_req or auth_request modules.
2015-11-05 15:01:01 +03:00
Valentin Bartenev
090c471032 HTTP/2: changed behavior of the "http2_max_field_size" directive.
Now it limits only the maximum length of literal string (either raw or
compressed) in HPACK request header fields.  It's easier to understand
and to describe in the documentation.
2015-10-27 23:16:35 +03:00
Valentin Bartenev
edf30961e6 HTTP/2: fixed spelling. 2015-10-27 23:16:35 +03:00
Valentin Bartenev
f36f51725a HTTP/2: simplified checking the END_STREAM flag.
No functional changes.
2015-10-26 17:46:28 +03:00
Valentin Bartenev
d418b92cdc HTTP/2: improved the ngx_http_v2_integer_octets(v) macro.
Previously, it didn't work well for 0, 127, and 128, returning less than needed.
2015-10-26 17:46:16 +03:00
Valentin Bartenev
3565680d94 HTTP/2: fixed the NGX_HTTP_V2_MAX_FIELD macro. 2015-10-26 17:46:13 +03:00
Valentin Bartenev
9150c7df1f HTTP/2: fixed splitting of response headers on CONTINUATION frames.
Previous code has been based on assumption that the header block can only be
splitted at the borders of individual headers.  That wasn't the case and might
result in emitting frames bigger than the frame size limit.

The current approach is to split header blocks by the frame size limit.
2015-09-28 02:32:44 +03:00
Valentin Bartenev
60e41c0fb1 HTTP/2: introduced NGX_HTTP_V2_ENCODE_* macros.
No functional changes.
2015-10-26 17:43:08 +03:00
Valentin Bartenev
69e78850fe HTTP/2: simplified producing of the Last-Modified header. 2015-10-26 17:43:08 +03:00
Valentin Bartenev
70c29f05f0 HTTP/2: fixed header block size calculation. 2015-10-26 17:43:08 +03:00
Piotr Sikora
f93aa4d953 HTTP/2: fix handling of connection errors.
Previously, nginx worker would crash because of a double free
if client disconnected or timed out before sending all headers.

Found with afl-fuzz.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2015-10-01 20:25:55 -07:00
Piotr Sikora
de4f0eb26b HTTP/2: fix indirect reprioritization.
Previously, streams that were indirectly reprioritized (either because of
a new exclusive dependency on their parent or because of removal of their
parent from the dependency tree), didn't have their pointer to the parent
node updated.

This broke detection of circular dependencies and, as a result, nginx
worker would crash due to stack overflow whenever such dependency was
introduced.

Found with afl-fuzz.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2015-10-01 20:25:55 -07:00
Piotr Sikora
71a6b600c2 HTTP/2: reject self-dependent streams.
Per RFC7540, a stream cannot depend on itself.

Previously, this requirement was enforced on PRIORITY frames, but not on
HEADERS frames and due to the implementation details nginx worker would
crash (stack overflow) while opening self-dependent stream.

Found with afl-fuzz.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2015-10-01 20:25:55 -07:00
Valentin Bartenev
c0eb2f0759 HTTP/2: improved error handling while parsing integers.
The case when an integer is out of frame bounds should be checked first
as a more accurate error.
2015-10-20 21:28:38 +03:00
Ruslan Ermilov
c830cf28ae HTTP/2: improved HPACK integer parsing code readability.
No functional changes.
2015-10-20 21:26:38 +03:00
Maxim Dounin
cc74c63d39 Fixed variables prefix comparison.
Variable names are not null-terminated, so using ngx_strncmp() without
extra length checks is wrong.

Reported by Markus Linnala,
http://mailman.nginx.org/pipermail/nginx-devel/2015-August/007211.html.
2015-10-19 21:28:17 +03:00
Maxim Dounin
b9e0b9df44 Fixed handling of empty root.
Previously, a configuration with "root" set to an empty string tried test
a character at clcf->root.data[-1].  Additional test added to make sure
this won't happen.

Reported by Markus Linnala, see
http://mailman.nginx.org/pipermail/nginx-devel/2015-August/007210.html.
2015-10-19 21:28:02 +03:00
Valentin Bartenev
7d2af13f3a HTTP/2: fixed parsing of literal header fields without indexing. 2015-10-19 16:16:47 +03:00
Valentin Bartenev
20669d1949 HTTP/2: fixed $server_protocol value (ticket #800). 2015-09-28 20:02:05 +03:00
Maxim Dounin
49b17956f9 Style. 2015-09-22 17:09:50 +03:00
Valentin Bartenev
a0d7df93a0 Increased the default number of output buffers.
Since an output buffer can only be used for either reading or sending, small
amounts of data left from the previous operation (due to some limits) must be
sent before nginx will be able to read further into the buffer.  Using only
one output buffer can result in suboptimal behavior that manifests itself in
forming and sending too small chunks of data.  This is particularly painful
with SPDY (or HTTP/2) where each such chunk needs to be prefixed with some
header.

The default flow-control window in HTTP/2 is 64k minus one bytes.  With one
32k output buffer this results is one byte left after exhausting the window.
With two 32k buffers the data will be read into the second free buffer before
sending, thus the minimum output is increased to 32k + 1 bytes which is much
better.
2015-09-15 17:49:15 +03:00
Valentin Bartenev
61e1f16d90 HTTP/2: fixed header block parsing with CONTINUATION frames (#792).
It appears that the CONTINUATION frames don't need to be aligned to bounds of
individual headers.
2015-09-22 01:40:04 +03:00
Valentin Bartenev
a27d0bd00d HTTP/2: fixed HPACK header field parsing. 2015-09-22 01:40:04 +03:00
Roman Arutyunyan
f64ff24451 Sub filter: fixed initialization in http{} level (ticket #791).
If sub_filter directive was only specified at http{} level, sub filter
internal data remained uninitialized.  That would lead to a crash in runtime.
2015-09-21 23:08:34 +03:00
Valentin Bartenev
ee37ff613f The HTTP/2 implementation (RFC 7240, 7241).
The SPDY support is removed, as it's incompatible with the new module.
2015-09-11 20:13:06 +03:00
Maxim Dounin
7930a6c442 Fixed segfault with incorrect location nesting.
A configuration with a named location inside a zero-length prefix
or regex location used to trigger a segmentation fault, as
ngx_http_core_location() failed to properly detect if a nested location
was created.  Example configuration to reproduce the problem:

    location "" {
        location @foo {}
    }

Fix is to not rely on a parent location name length, but rather check
command type we are currently parsing.

Identical fix is also applied to ngx_http_rewrite_if(), which used to
incorrectly assume the "if" directive is on server{} level in such
locations.

Reported by Markus Linnala.
Found with afl-fuzz.
2015-09-11 17:04:04 +03:00
Maxim Dounin
ce05841eef Cache: check the whole cache key in addition to hashes.
This prevents a potential attack that discloses cached data if an attacker
will be able to craft a hash collision between some cache key the attacker
is allowed to access and another cache key with protected data.

See http://mailman.nginx.org/pipermail/nginx-devel/2015-September/007288.html.

Thanks to Gena Makhomed and Sergey Brester.
2015-09-11 17:03:56 +03:00
Roman Arutyunyan
51f714c85d Upstream: fixed cache send error handling.
The value of NGX_ERROR, returned from filter handlers, was treated as a generic
upstream error and changed to NGX_HTTP_INTERNAL_SERVER_ERROR before calling
ngx_http_finalize_request().  This resulted in "header already sent" alert
if header was already sent in filter handlers.

The problem appeared in 54e9b83d00f0 (1.7.5).
2015-09-03 15:09:21 +03:00
Valentin Bartenev
5e0b936a07 Decreased the NGX_HTTP_MAX_SUBREQUESTS limit.
There is no much sense in such a big value since its semantics
has been changed in 06e850859a26 to limit recursive subrequests.
2015-08-31 23:26:33 +03:00
Valentin Bartenev
4ec67cf86f Added protection against r->main->count overflow by subrequests.
This overflow has become possible after the change in 06e850859a26,
since concurrent subrequests are not limited now and each of them is
counted in r->main->count.
2015-08-31 23:25:16 +03:00
Valentin Bartenev
0d3b15729c Limit recursive subrequests instead of simultaneous. 2015-08-23 21:03:32 +03:00
Maxim Dounin
2e004237c5 Perl: prototyping behavior explicitly specified.
When prototyping behavior is not explicitly specified, xsubpp emits
a message to stderr asking to do so (see ticket #608).
2015-08-18 16:26:18 +03:00
Maxim Dounin
a0aea61b50 Perl: fixed warning about "sep" may be used uninitialized. 2015-08-18 16:26:05 +03:00
Maxim Dounin
84b0ad63fb Win32: MSVC 2015 compatibility.
Resolved warnings about declarations that hide previous local declarations.
Warnings about WSASocketA() being deprecated resolved by explicit use of
WSASocketW() instead of WSASocket().  When compiling without IPv6 support,
WinSock deprecated warnings are disabled to allow use of gethostbyname().
2015-08-17 18:09:17 +03:00
Dmitry Volyntsev
1576cd10aa Sub filter: support of variables in the strings to replace. 2015-08-17 17:42:02 +03:00
Dmitry Volyntsev
934b49923c Sub filter: support of multiple strings to replace. 2015-08-17 17:42:02 +03:00
Maxim Dounin
05982ffdf5 Fixed wrong URI after try_files in nested location (ticket #97).
The following configuration with alias, nested location and try_files
resulted in wrong file being used.  Request "/foo/test.gif" tried to
use "/tmp//foo/test.gif" instead of "/tmp/test.gif":

    location /foo/ {
        alias /tmp/;
        location ~ gif {
            try_files $uri =405;
        }
    }

Additionally, rev. c985d90a8d1f introduced a regression if
the "/tmp//foo/test.gif" file was found (ticket #768).  Resulting URI
was set to "gif?/foo/test.gif", as the code used clcf->name of current
location ("location ~ gif") instead of parent one ("location /foo/").

Fix is to use r->uri instead of clcf->name in all cases in the
ngx_http_core_try_files_phase() function.  It is expected to be
already matched and identical to the clcf->name of the right
location.
2015-08-16 10:51:34 +03:00
Maxim Dounin
4fec72dc8c Fixed segfault with try_files introduced by c985d90a8d1f.
If alias was used in a location given by a regular expression,
nginx used to do wrong thing in try_files if a location name (i.e.,
regular expression) was an exact prefix of URI.  The following
configuration triggered a segmentation fault on a request to "/mail":

    location ~ /mail {
        alias /path/to/directory;
        try_files $uri =404;
    }

Reported by Per Hansson.
2015-08-16 10:51:16 +03:00
Valentin Bartenev
50ff8b3c3a Core: idle connections now closed only once on exiting.
Iterating through all connections takes a lot of CPU time, especially
with large number of worker connections configured.  As a result
nginx processes used to consume CPU time during graceful shutdown.
To mitigate this we now only do a full scan for idle connections when
shutdown signal is received.

Transitions of connections to idle ones are now expected to be
avoided if the ngx_exiting flag is set.  The upstream keepalive module
was modified to follow this.
2015-08-11 16:28:55 +03:00
Ruslan Ermilov
6dc03bf1a4 Fixed strict aliasing warnings with old GCC versions. 2015-07-16 14:20:48 +03:00
Ruslan Ermilov
1e94ab0db0 Upstream: fixed shared upstreams on win32. 2015-06-16 00:43:00 +03:00
Vladimir Homutov
b7a8f2a08f Disabled duplicate http, mail, and stream blocks.
Such configurations have very limited use, introduce various problems and
are not officially supported.
2015-06-16 23:28:38 +03:00
Roman Arutyunyan
4404c3fd6d Core: renamed ngx_proxy_protocol_parse to ngx_proxy_protocol_read.
The new name is consistent with the ngx_proxy_protocol_write function.
2015-06-16 13:45:19 +03:00
Maxim Dounin
9984f3053f Moved ngx_http_parse_time() to core, renamed accordingly.
The function is now called ngx_parse_http_time(), and can be used by
any code to parse HTTP-style date and time.  In particular, it will be
used for OCSP stapling.

For compatibility, a macro to map ngx_http_parse_time() to the new name
provided for a while.
2015-06-11 20:42:31 +03:00
Maxim Dounin
dd41bfd47a Removed unused ngx_http_get_time() declaration. 2015-06-11 20:42:24 +03:00
Roman Arutyunyan
f5d79ffb82 Upstream keepalive: reduced diffs to the plus version of nginx. 2015-06-04 23:46:44 +03:00
Maxim Dounin
724f0f028e Disabled SSLv3 by default (ticket #653). 2015-05-25 17:58:20 +03:00
Ruslan Ermilov
941a6995d7 Upstream: report to error_log when max_fails is reached.
This can be useful to understand why "no live upstreams" happens,
in particular.
2015-05-20 22:44:00 +03:00
Maxim Dounin
f7f1607bf2 The "reuseport" option of the "listen" directive.
When configured, an individual listen socket on a given address is
created for each worker process.  This allows to reduce in-kernel lock
contention on configurations with high accept rates, resulting in better
performance.  As of now it works on Linux and DragonFly BSD.

Note that on Linux incoming connection requests are currently tied up
to a specific listen socket, and if some sockets are closed, connection
requests will be reset, see https://lwn.net/Articles/542629/.  With
nginx, this may happen if the number of worker processes is reduced.
There is no such problem on DragonFly BSD.

Based on previous work by Sepherosa Ziehau and Yingqi Lu.
2015-05-20 15:51:56 +03:00
Maxim Dounin
d5c34785bc Simplified ngx_http_init_listening().
There is no need to set "i" to 0, as it's expected to be 0 assuming
the bindings are properly sorted, and we already rely on this when
explicitly set hport->naddrs to 1.  Remaining conditional code is
replaced with identical "hport->naddrs = i + 1".

Identical modifications are done in the mail and stream modules,
in the ngx_mail_optimize_servers() and ngx_stream_optimize_servers()
functions, respectively.

No functional changes.
2015-05-20 15:51:28 +03:00
Sergey Kandaurov
3670726bd3 Upstream hash: consistency across little/big endianness. 2015-05-18 16:05:44 +03:00
Ruslan Ermilov
4b9b16d27d Upstream: $upstream_connect_time.
The variable keeps time spent on establishing a connection with
the upstream server.
2015-05-16 01:32:27 +03:00
Ruslan Ermilov
b070e2d713 Upstream: times to obtain header/response are stored as ngx_msec_t. 2015-05-16 01:31:04 +03:00
Ruslan Ermilov
ea2f4d9175 Removed deprecated HTTP directives. 2015-04-29 13:52:49 +03:00
Vladimir Homutov
df555ffce6 Core: the ngx_set_connection_log() macro.
The http and stream versions of this macro were identical.
2015-04-25 22:44:02 +03:00
Roman Arutyunyan
d61f21c5e8 Merge proxy_protocol setting of listen directives.
It's now enough to specify proxy_protocol option in one listen directive to
enable it in all servers listening on the same address/port.  Previously,
the setting from the first directive was always used.
2015-04-24 10:54:06 +03:00
Ruslan Ermilov
33b8e5bc06 Removed the obsolete rtsig module. 2015-04-23 14:17:40 +03:00
Ruslan Ermilov
c1882d9f3f Removed the obsolete aio module. 2015-04-22 18:57:32 +03:00
Ruslan Ermilov
25ddb49dbe Upstream: allow multiple upstreams to use the same shared zone. 2015-04-22 18:37:34 +03:00
Martin Mlynář
958d4a0b1f Memcached: enabled ranges. 2015-04-21 21:04:36 +02:00
Ruslan Ermilov
673941f81d Upstream: simplified ip_hash and hash peer selection code.
Now that peers are stored as a list, the weighted and unweighted
cases became nearly identical.
2015-04-21 19:09:04 +03:00
Sergey Kandaurov
1ff5515e70 Limit req: removed check for unknown limit_req_zone.
With 48b3d5ddfb03, it's possible to specify limit_req_zone after limit_req.
2015-04-20 16:54:41 +03:00
Ruslan Ermilov
a2dac51398 Upstream: fixed unlocked access to peer->conns. 2015-04-19 22:41:09 +03:00
Ruslan Ermilov
7c44e39220 Upstream: get rid of questionable micro-optimization in ip_hash.
If a peer was initially skipped due to max_fails, there's no reason
not to try it again if enough time has passed, and the next_upstream
logic is in action.

This also reduces diffs with NGINX Plus.
2015-04-16 20:09:11 +03:00
Ruslan Ermilov
cf31347ee8 Upstream: the "zone" directive.
Upstreams with the "zone" directive are kept in shared memory,
with a consistent view of all worker processes.
2015-04-14 19:01:25 +03:00
Ruslan Ermilov
b0b7b5a356 Upstreams: locking. 2015-04-14 19:01:23 +03:00
Ruslan Ermilov
4d53631022 Upstream: store peers as a linked list.
This is an API change.
2015-04-10 14:48:36 +03:00
Ruslan Ermilov
18fa775b10 Upstream: track the number of active connections to upstreams.
This also simplifies the implementation of the least_conn module.
2015-04-10 13:16:23 +03:00
Ruslan Ermilov
8c82b75127 Upstream: abbreviated SSL handshake may interact badly with Nagle. 2015-04-07 00:07:04 +03:00
Valentin Bartenev
3f72e18908 Request body: always flush buffers if request buffering is off.
This fixes unbuffered proxying to SSL backends, since it prevents
ngx_ssl_send_chain() from accumulation of request body in the SSL
buffer.
2015-04-06 19:20:36 +03:00
Valentin Bartenev
355a793b89 Cache: added support for reading of the header in thread pools. 2015-04-01 03:49:17 +03:00
Sergey Kandaurov
99bf1b56ec Fixed invalid access to complex value defined as an empty string.
Found by Valgrind.
2015-03-31 17:45:50 +03:00
Valentin Bartenev
0e853129ac SPDY: always push pending data.
This helps to avoid suboptimal behavior when a client waits for a control
frame or more data to increase window size, but the frames have been delayed
in the socket buffer.

The delays can be caused by bad interaction between Nagle's algorithm on
nginx side and delayed ACK on the client side or by TCP_CORK/TCP_NOPUSH
if SPDY was working without SSL and sendfile() was used.

The pushing code is now very similar to ngx_http_set_keepalive().
2015-03-23 21:04:13 +03:00
Valentin Bartenev
44586bf5ee SPDY: fixed error handling in ngx_http_spdy_send_output_queue(). 2015-03-23 20:47:46 +03:00
Maxim Dounin
5e96e84ffb Proxy: fixed proxy_request_buffering and chunked with preread body.
If any preread body bytes were sent in the first chain, chunk size was
incorrectly added before the whole chain, including header, resulting in
an invalid request sent to upstream.  Fixed to properly add chunk size
after the header.
2015-03-26 02:31:30 +03:00
Maxim Dounin
7ec559df5d Upstream: uwsgi_request_buffering, scgi_request_buffering. 2015-03-23 21:09:19 +03:00
Maxim Dounin
f4714d26f0 FastCGI: fastcgi_request_buffering. 2015-03-23 21:09:19 +03:00
Maxim Dounin
756997b696 Proxy: proxy_request_buffering chunked support. 2015-03-23 21:09:19 +03:00
Maxim Dounin
2743bb68ee Request body: unbuffered reading.
The r->request_body_no_buffering flag was introduced.  It instructs
client request body reading code to avoid reading the whole body, and
to call post_handler early instead.  The caller should use the
ngx_http_read_unbuffered_request_body() function to read remaining
parts of the body.

Upstream module is now able to use this mode, if configured with
the proxy_request_buffering directive.
2015-03-23 21:09:19 +03:00
Maxim Dounin
d02d2cff9b Request body: filters support. 2015-03-23 21:09:19 +03:00
Maxim Dounin
be99a8801e Request body: moved request body writing to save filter. 2015-03-23 21:09:19 +03:00
Maxim Dounin
d21e644d80 Request body: free chain links in ngx_http_write_request_body(). 2015-03-23 21:09:12 +03:00
Maxim Dounin
3560af8999 Format specifier fixed for file size of buffers. 2015-03-23 19:28:54 +03:00
Maxim Dounin
58eb93f791 Proxy: fixed proxy_set_body with proxy_cache.
If the last header evaluation resulted in an empty header, the e.skip flag
was set and was not reset when we've switched to evaluation of body_values.
This incorrectly resulted in body values being skipped instead of producing
some correct body as set by proxy_set_body.  Fix is to properly reset
the e.skip flag.

As the problem only appeared if the last potentially non-empty header
happened to be empty, it only manifested itself if proxy_set_body was used
with proxy_cache.
2015-03-23 19:28:54 +03:00
Ruslan Ermilov
07de3f538b Removed stub implementation of win32 mutexes. 2015-03-23 13:52:47 +03:00
Maxim Dounin
a20c6d202e SSL: avoid SSL_CTX_set_tmp_rsa_callback() call with LibreSSL.
LibreSSL removed support for export ciphers and a call to
SSL_CTX_set_tmp_rsa_callback() results in an error left in the error
queue.  This caused alerts "ignoring stale global SSL error (...called
a function you should not call) while SSL handshaking" on a first connection
in each worker process.
2015-03-23 02:42:34 +03:00
Ruslan Ermilov
babd64e344 Removed busy locks. 2015-03-20 06:45:32 +03:00