Commit Graph

100 Commits

Author SHA1 Message Date
Piotr Sikora
ca1a5057e2 Upstream: allow recovery from "429 Too Many Requests" response.
This change adds "http_429" parameter to "proxy_next_upstream" for
retrying rate-limited requests, and to "proxy_cache_use_stale" for
serving stale cached responses after being rate-limited.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2017-03-24 02:48:03 -07:00
Roman Arutyunyan
3e4339eacd Upstream: proxy_cache_background_update and friends.
The directives enable cache updates in subrequests.
2017-02-10 15:13:41 +03:00
Dmitry Volyntsev
6d9023f7b4 Cache: proxy_cache_max_range_offset and friends.
It configures a threshold in bytes, above which client range
requests are not cached.  In such a case the client's Range
header is passed directly to a proxied server.
2016-11-02 20:05:21 +03:00
Ruslan Ermilov
3fae83a91c Upstream: added the ngx_http_upstream_resolved_t.name field.
This fixes inconsistency in what is stored in the "host" field.
Normally it would contain the "host" part of the parsed URL
(e.g., proxy_pass with variables), but for the case of an
implicit upstream specified with literal address it contained
the text representation of the socket address (that is, host
including port for IP).

Now the "host" field always contains the "host" part of the URL,
while the text representation of the socket address is stored
in the newly added "name" field.

The ngx_http_upstream_create_round_robin_peer() function was
modified accordingly in a way to be compatible with the code
that does not know about the new "name" field.

The "stream" code was similarly modified except for not adding
compatibility in ngx_stream_upstream_create_round_robin_peer().

This change is also a prerequisite for the next change.
2016-10-31 18:33:33 +03:00
Ruslan Ermilov
4e1720b0a2 Upstream: removed unnecessary condition in proxy_eval() and friends.
The first condition added in d3454e719bbb should have just replaced
the second one.
2016-10-31 18:33:31 +03:00
Maxim Dounin
948a18ed54 Upstream: handling of proxy_set_header at http level.
When headers are set at the "http" level and not redefined in
a server block, we now preserve conf->headers into the "http"
section configuration to inherit it to all servers.

The same applies to conf->headers_cache, though it may not be effective
if no servers use cache at the "server" level as conf->headers_cache
is only initialized if cache is enabled on a given level.

Similar changes made in fastcgi/scgi/uwsgi to preserve conf->params
and conf->params_cache.
2016-10-14 19:48:26 +03:00
Tim Taubert
4f578bfcab SSL: ngx_ssl_ciphers() to set list of ciphers.
This patch moves various OpenSSL-specific function calls into the
OpenSSL module and introduces ngx_ssl_ciphers() to make nginx more
crypto-library-agnostic.
2016-06-15 21:05:30 +01:00
Roman Arutyunyan
be79f5cb16 Upstream: the "transparent" parameter of proxy_bind and friends.
This parameter lets binding the proxy connection to a non-local address.
Upstream will see the connection as coming from that address.
When used with $remote_addr, upstream will accept the connection from real
client address.

Example:

    proxy_bind $remote_addr transparent;
2015-12-18 19:05:27 +03:00
Maxim Dounin
4618c03f4c Upstream: proxy_next_upstream non_idempotent.
By default, requests with non-idempotent methods (POST, LOCK, PATCH)
are no longer retried in case of errors if a request was already sent
to a backend.  Previous behaviour can be restored by using
"proxy_next_upstream ... non_idempotent".
2016-03-28 19:50:19 +03:00
Ruslan Ermilov
72b9a317d2 Upstream: fixed "no port" detection in evaluated upstreams.
If an upstream with variables evaluated to address without a port,
then instead of a "no port in upstream" error an attempt was made
to connect() which failed with EADDRNOTAVAIL.
2015-11-21 10:44:07 +03:00
Maxim Dounin
724f0f028e Disabled SSLv3 by default (ticket #653). 2015-05-25 17:58:20 +03:00
Maxim Dounin
7ec559df5d Upstream: uwsgi_request_buffering, scgi_request_buffering. 2015-03-23 21:09:19 +03:00
Valentin Bartenev
7817df480e Upstream: added variables support to proxy_cache and friends. 2014-12-22 12:59:09 +03:00
Valentin Bartenev
c40a72761c Upstream: refactored proxy_cache and friends.
The configuration handling code has changed to look similar to the proxy_store
directive and friends.  This simplifies adding variable support in the following
patch.

No functional changes.
2014-12-22 12:59:02 +03:00
Valentin Bartenev
66dcbab7a1 Upstream: mutually exclusive inheritance of "cache" and "store".
Currently, storing and caching mechanisms cannot work together, and a
configuration error is thrown when the proxy_store and proxy_cache
directives (as well as their friends) are configured on the same level.

But configurations like in the example below were allowed and could result
in critical errors in the error log:

    proxy_store on;

    location / {
        proxy_cache one;
    }

Only proxy_store worked in this case.

For more predictable and errorless behavior these directives now prevent
each other from being inherited from the previous level.
2014-12-22 12:58:59 +03:00
Valentin Bartenev
492844dbb1 Upstream: simplified proxy_store and friends configuration code.
This changes internal API related to handling of the "store"
flag in ngx_http_upstream_conf_t.  Previously, a non-null value
of "store_lengths" was enough to enable store functionality with
custom path.  Now, the "store" flag is also required to be set.

No functional changes.
2014-12-22 12:58:56 +03:00
Valentin Bartenev
fb7d63250e Upstream: fixed inheritance of proxy_store and friends.
The proxy_store, fastcgi_store, scgi_store and uwsgi_store were inherited
incorrectly if a directive with variables was defined, and then redefined
to the "on" value, i.e. in configurations like:

    proxy_store /data/www$upstream_http_x_store;

    location / {
        proxy_store on;
    }
2014-10-09 20:30:11 +04:00
Maxim Dounin
8d7c6491fe Upstream: fixed unexpected inheritance into limit_except blocks.
The proxy_pass directive and other handlers are not expected to be inherited
into nested locations, but there is a special code to inherit upstream
handlers into limit_except blocks, as well as a configuration into if{}
blocks.  This caused incorrect behaviour in configurations with nested
locations and limit_except blocks, like this:

    location / {
        proxy_pass http://u;

        location /inner/ {
            # no proxy_pass here

            limit_except GET {
                # nothing
            }
        }
    }

In such a configuration the limit_except block inside "location /inner/"
unexpectedly used proxy_pass defined in "location /", while it shouldn't.
Fix is to avoid inheritance of conf->upstream.upstream (and
conf->proxy_lengths) into locations which don't have noname flag.
2014-12-09 18:22:31 +03:00
Maxim Dounin
c863e1b3b5 Upstream: inheritance of proxy_pass and friends (ticket #645).
Instead of independant inheritance of conf->upstream.upstream (proxy_pass
without variables) and conf->proxy_lengths (proxy_pass with variables)
we now test them both and inherit only if neither is set.  Additionally,
SSL context is also inherited only in this case now.

Based on the patch by Alexey Radkov.
2014-12-09 18:21:55 +03:00
Roman Arutyunyan
8898c97dbd Upstream: different header lists for cached and uncached requests.
The upstream modules remove and alter a number of client headers
before sending the request to upstream.  This set of headers is
smaller or even empty when cache is disabled.

It's still possible that a request in a cache-enabled location is
uncached, for example, if cache entry counter is below min_uses.
In this case it's better to alter a smaller set of headers and
pass more client headers to backend unchanged.  One of the benefits
is enabling server-side byte ranges in such requests.
2014-11-19 17:33:23 +03:00
Roman Arutyunyan
bcf6b11a34 Upstream: moved header lists to separate structures.
No functional changes.
2014-11-19 17:33:22 +03:00
Roman Arutyunyan
62dfdf1814 Upstream: moved header initializations to separate functions.
No functional changes.
2014-11-19 17:33:21 +03:00
Roman Arutyunyan
2628cc11d4 Cache: proxy_cache_lock_age and friends.
Once this age is reached, the cache lock is discarded and another
request can acquire the lock.  Requests which failed to acquire
the lock are not allowed to cache the response.
2014-11-18 20:41:12 +03:00
Piotr Sikora
a9d9b09103 Upstream: add "proxy_ssl_certificate" and friends.
Signed-off-by: Piotr Sikora <piotr@cloudflare.com>
2014-10-30 04:30:41 -07:00
Roman Arutyunyan
18e4d042b0 Upstream: proxy_limit_rate and friends.
The directives limit the upstream read rate.  For example,
"proxy_limit_rate 42" limits proxy upstream read rate to
42 bytes per second.
2014-10-28 12:29:59 +03:00
Roman Arutyunyan
2244a5e9cf Upstream: proxy_force_ranges and friends.
The directives enable byte ranges for both cached and uncached
responses regardless of backend headers.
2014-10-14 18:32:01 +04:00
Roman Arutyunyan
02ce6c415f Upstream: limited next_upstream time and tries (ticket #544).
The new directives {proxy,fastcgi,scgi,uwsgi,memcached}_next_upstream_tries
and {proxy,fastcgi,scgi,uwsgi,memcached}_next_upstream_timeout limit
the number of upstreams tried and the maximum time spent for these tries
when searching for a valid upstream.
2014-09-12 18:50:47 +04:00
Maxim Dounin
3a235bf52e Added warning about unset cache keys.
In fastcgi, scgi and uwsgi modules there are no default cache keys, and
using a cache without a cache key set is likely meaningless.
2014-09-11 20:08:52 +04:00
Maxim Dounin
d7d26feba1 Style. 2014-09-11 20:08:45 +04:00
Piotr Sikora
2cfcef5b42 Style: remove whitespace between function name and parentheses.
Signed-off-by: Piotr Sikora <piotr@cloudflare.com>
2014-07-08 03:03:14 -07:00
Maxim Dounin
46ac5c760c Upstream: cache revalidation using If-None-Match. 2014-06-26 02:35:01 +04:00
Maxim Dounin
cae1bd3831 Upstream: uwsgi_ssl_name, uwsgi_ssl_verify, and so on.
Just a merge of proxy_ssl_name, proxy_ssl_verify commits into uwsgi module,
code is identical.
2014-04-18 20:13:32 +04:00
Maxim Dounin
93eb94d622 Upstream: plugged potential memory leak on reload.
The SSL_CTX_set_cipher_list() may fail if there are no valid ciphers
specified in proxy_ssl_ciphers / uwsgi_ssl_ciphers, resulting in
SSL context leak.

In theory, ngx_pool_cleanup_add() may fail too, but this case is
intentionally left out for now as it's almost impossible and proper fix
will require changes to http ssl and mail ssl code as well.
2014-04-18 20:13:24 +04:00
Piotr Sikora
60d508ceb9 Upstream: fix $upstream_status variable.
Previously, upstream's status code was overwritten with
cached response's status code when STALE or REVALIDATED
response was sent to the client.

Signed-off-by: Piotr Sikora <piotr@cloudflare.com>
2014-02-11 21:54:42 -08:00
Valentin Bartenev
2b1156d101 Style: removed surplus semicolons. 2013-12-27 18:47:42 +04:00
Maxim Dounin
0c585adfd4 Trailing whitespace fix. 2013-12-12 20:28:48 +04:00
Piotr Sikora
19f475276d Fixed incorrect ngx_str_set() usage, broken in c82b2e020b9f.
Found by Coverity Scan CID 1135525.

Signed-off-by: Piotr Sikora <piotr@cloudflare.com>
2013-12-06 16:00:49 -08:00
Maxim Dounin
67b0d16546 Fixed build without SSL, broken by c82b2e020b9f. 2013-12-04 23:13:13 +04:00
Maxim Dounin
24e1db93a8 SSL support in the uwsgi module.
Based on patch by Roberto De Ioris.
2013-12-04 23:01:27 +04:00
Maxim Dounin
3dec2b131c Upstream: skip empty cache headers.
Notably this fixes HTTP_IF_MODIFIED_SINCE which was always sent with
cache enabled in fastcgi/scgi/uwsgi after 43ccaf8e8728.
2013-11-29 17:23:38 +04:00
Maxim Dounin
1ac2693a33 Upstream: cache revalidation with conditional requests.
The following new directives are introduced: proxy_cache_revalidate,
fastcgi_cache_revalidate, scgi_cache_revalidate, uwsgi_cache_revalidate.
Default is off.  When set to on, they enable cache revalidation using
conditional requests with If-Modified-Since for expired cache items.

As of now, no attempts are made to merge headers given in a 304 response
during cache revalidation with headers previously stored in a cache item.
Headers in a 304 response are only used to calculate new validity time
of a cache item.
2013-11-18 20:48:22 +04:00
Maxim Dounin
989a71377b Upstream: subrequest_in_memory support for SCGI and uwsgi enabled.
This was missed in 9d59a8eda373 when non-buffered support was added to SCGI
and uwsgi.
2013-09-27 16:50:34 +04:00
Maxim Dounin
1a983a0c05 Upstream: http_403 support in proxy_next_upstream (and friends).
The parameter is mostly identical to http_404, and is expected to
be used in similar situations.  The 403 code might be returned by
a backend instead of 404 on initial sync of new directories with rsync.

See here for feature request and additional details:
http://mailman.nginx.org/pipermail/nginx-ru/2013-April/050920.html
2013-05-27 16:54:09 +04:00
Maxim Dounin
56bc5f250b Trailing whitespace fix. 2013-02-23 11:50:42 +00:00
Maxim Dounin
208fc03e7c Connection upgrade support in uwsgi and scgi modules.
Prodded by Roberto De Ioris.
2013-02-20 16:41:05 +00:00
Ruslan Ermilov
5d143ca864 Fixed and improved the "*_bind" directives of proxying modules.
The "proxy_bind", "fastcgi_bind", "uwsgi_bind", "scgi_bind" and
"memcached_bind" directives are now inherited; inherited value
can be reset by the "off" parameter.  Duplicate directives are
now detected.  Parameter value can now contain variables.
2013-01-16 09:42:57 +00:00
Ruslan Ermilov
73fb7e878f Allow the complex value to be defined as an empty string.
This makes conversion from strings to complex values possible
without the loss of functionality.
2012-12-06 23:03:53 +00:00
Maxim Dounin
b34f84d8de Added r->state reset on fastcgi/scgi/uwsgi request start.
Failing to do so results in problems if 400 or 414 requests are
redirected to fastcgi/scgi/uwsgi upstream, as well as after invalid
headers got from upstream.  This was already fixed for proxy in r3478,
but fastcgi (the only affected protocol at that time) was missed.

Reported by Matthieu Tourne.
2012-05-11 13:09:24 +00:00
Ruslan Ermilov
43d2b1c045 Fixed grammar in error messages. 2012-04-12 19:35:41 +00:00
Maxim Dounin
eb526b7d7d Fixed incorrect ngx_cpystrn() usage in ngx_http_*_process_header().
This resulted in a disclosure of previously freed memory if upstream
server returned specially crafted response, potentially exposing
sensitive information.

Reported by Matthew Daley.
2012-03-15 11:27:12 +00:00