Commit Graph

1652 Commits

Author SHA1 Message Date
Dmitry Volyntsev
b072a6957c Cache: cache manager limits.
The new parameters "manager_files", "manager_sleep"
and "manager_threshold" were added to proxy_cache_path
and friends.

Note that ngx_path_manager_pt was changed to return ngx_msec_t
instead of time_t (API change).
2016-10-05 14:22:30 +03:00
Ruslan Ermilov
f9430de485 Core: use c->log while closing connection.
c->pool is not destroyed here since c52408583801.
2016-10-05 13:57:43 +03:00
Ruslan Ermilov
2e12afb955 Modules compatibility: removed two NGX_HAVE_DEFERRED_ACCEPT checks.
Removed (NGX_HAVE_DEFERRED_ACCEPT && defined TCP_DEFER_ACCEPT)
from the signature accordingly.
2016-10-03 15:58:30 +03:00
Maxim Dounin
58b85017a4 Modules compatibility: removed unneeded NGX_HAVE_REUSEPORT checks.
Removed NGX_HAVE_REUSEPORT from the signature accordingly.
2016-10-03 15:58:29 +03:00
Maxim Dounin
0a961a0917 Modules compatibility: removed unneeded IPV6_V6ONLY checks.
The IPV6_V6ONLY macro is now checked only while parsing appropriate flag
and when using the macro.

The ipv6only field in listen structures is always initialized to 1,
even if not supported on a given platform.  This is expected to prevent
a module compiled without IPV6_V6ONLY from accidentally creating dual
sockets if loaded into main binary with proper IPV6_V6ONLY support.
2016-10-03 15:58:25 +03:00
Maxim Dounin
a7f80ec354 Modules compatibility: http2.
HTTP/2-specific fields in structures are now available unconditionally.
Removed NGX_HTTP_V2 from the signature accordingly.
2016-10-03 15:58:22 +03:00
Maxim Dounin
2641022229 Modules compatibility: degradation fields now always present.
There is no need to save these two bits as they are wasted anyway.
Removed NGX_HTTP_DEGRADATION from the signature accordingly.
2016-10-03 15:58:19 +03:00
Ruslan Ermilov
c40de74682 Introduced the NGX_COMPAT macro.
When enabled, some structures are padded to be size compatible
with their NGINX Plus versions.
2016-09-29 15:28:24 +03:00
Dmitry Volyntsev
833d03a0a2 Resolver: introduced state field in ngx_resolver_srv_name_t.
It keeps the actual state value of a DNS SRV subrequest
and can be used to report a more detailed log for
failed SRV records.
2016-09-29 19:40:47 +03:00
Maxim Dounin
27c8065e7b Modules compatibility: cache purge fields. 2016-09-29 18:06:01 +03:00
Valentin Bartenev
89f82c1155 Fixed log levels of configuration parsing errors.
All the errors that prevent loading configuration must be printed on the "emerg"
log level.  Previously, nginx might silently fail to load configuration in some
cases as the default log level is "error".
2016-09-20 15:07:16 +03:00
Ruslan Ermilov
1fd83ac0c8 Removed influence of some options on structures. 2016-09-20 12:30:52 +03:00
Vladimir Homutov
afa771140b Version bump. 2016-09-15 14:56:26 +03:00
Dmitry Volyntsev
a613df5b3f Core: introduced ngx_cidr_match() function. 2016-09-07 13:56:53 +03:00
Piotr Sikora
14b1b6e10a Thread pools: create threads in detached state.
This prevents theoretical resource leak, since those threads are never joined.

Found with ThreadSanitizer.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2016-08-15 05:52:04 -07:00
Vladimir Homutov
386e3e80bb Version bump. 2016-07-26 19:07:18 +03:00
Sergey Kandaurov
6299f5e914 Avoid left-shifting integers into the sign bit, which is undefined.
Found with UndefinedBehaviorSanitizer.
2016-07-07 21:02:28 +03:00
Ruslan Ermilov
6810b9f148 Use NGX_MAX_PATH_LEVEL where appropriate.
The macro was unused since 0.7.44.
2016-07-06 13:22:29 +03:00
Ruslan Ermilov
5c4ca2ac80 Version bump. 2016-07-06 13:10:06 +03:00
Maxim Dounin
c60b61a290 Internal md5 and sha1 implementations are now always used.
This reduces the number of moving parts in ABI compatibility checks.
Additionally, it also allows to use OpenSSL in FIPS mode while still
using md5 for non-security tasks.
2016-06-30 18:57:39 +03:00
Ruslan Ermilov
fb6c764921 Removed unused flag accept_context_updated from ngx_event_t.
Also, removed practically unused flag accept_context_updated from
ngx_connection_t.
2016-06-29 14:30:00 +03:00
Roman Arutyunyan
9810fd06cb Fixed build on MSVC. 2016-06-20 15:11:50 +03:00
Roman Arutyunyan
5b201ac31f Introduced ngx_inet_get_port() and ngx_inet_set_port() functions. 2016-06-20 11:50:39 +03:00
Maxim Dounin
ab8504b937 An internal SHA1 implementation. 2016-06-09 16:55:38 +03:00
Sergey Kandaurov
ba212375a7 Version bump. 2016-06-07 12:26:34 +03:00
Valentin Bartenev
99d2ede934 Version bump. 2016-05-24 21:54:32 +03:00
Maxim Dounin
1a1470b865 Fixed build on MSVC. 2016-05-24 17:43:58 +03:00
Dmitry Volyntsev
06176bce91 Realip: port support in X-Real-IP and X-Forwarded-For.
Now, the module extracts optional port which may accompany an
IP address.  This custom extension is introduced, among other
things, in order to facilitate logging of original client ports.
Addresses with ports are expected to be in the RFC 3986 format,
that is, with IPv6 addresses in square brackets.  E.g.,
"X-Real-IP: [2001:0db8::1]:12345" sets client port ($remote_port)
to 12345.
2016-05-23 18:44:23 +03:00
Dmitry Volyntsev
f56cf3ddc7 Added the $proxy_protocol_port variable. 2016-05-23 18:44:21 +03:00
Ruslan Ermilov
fd064d3b88 Introduced the ngx_sockaddr_t type.
It's properly aligned and can hold any supported sockaddr.
2016-05-23 16:37:20 +03:00
Ruslan Ermilov
61fbcd1cad Belatedly changed the ngx_create_listening() prototype.
The function is called only with "struct sockaddr *" since 0.7.58.
2016-05-20 17:02:04 +03:00
Ruslan Ermilov
41d512c069 Removed a surplus condition from ngx_parse_inet6_url().
No functional changes, since the condition was always true.
2016-05-11 17:55:30 +03:00
Valentin Bartenev
66be8c6608 Core: fixed port handling in ngx_parse_inet6_url().
This fixes buffer over-read when no port is specified in cases
similar to 5df5d7d771f6, and catches missing port separator.
2016-05-11 17:55:20 +03:00
Ruslan Ermilov
2a83e5fa6d Removed unused flag unexpected_eof from ngx_connection_t. 2016-04-28 16:30:19 +03:00
Ruslan Ermilov
438b58f91f Removed some bitrot.
Removed NGX_CONF_MULTI unused since 1.3.4.
Removed ngx_url_t.one_addr unused since 1.3.10.
2016-04-26 18:26:43 +03:00
Ruslan Ermilov
9aca6d4d57 Version bump. 2016-04-26 18:26:18 +03:00
Maxim Dounin
eeb72127cf Thread pools: memory barriers in task completion notifications.
The ngx_thread_pool_done object isn't volatile, and at least some
compilers assume that it is permitted to reorder modifications of
volatile and non-volatile objects.  Added appropriate ngx_memory_barrier()
calls to make sure all modifications will happen before the lock is released.

Reported by Mindaugas Rasiukevicius,
http://mailman.nginx.org/pipermail/nginx-devel/2016-April/008160.html.
2016-04-19 17:18:28 +03:00
Ruslan Ermilov
400d42437c Fixed NGX_CONF_TAKE1/NGX_CONF_FLAG misuse (as in e444e8f6538b). 2016-04-12 19:01:56 +03:00
Ruslan Ermilov
37a3a2b2e8 Removed redundant "u" format specifier.
It is implied for "x" and "X".
2016-04-08 15:03:38 +03:00
Ruslan Ermilov
2f9f3453f2 Version bump. 2016-04-07 19:09:42 +03:00
Maxim Dounin
7cee215f15 Core: removed incorrect GCC 2.7 check.
It was broken since introduction (__GNU__ instead of __GNUC__) and did
nothing.  Moreover, GCC 2.7 is happy with the normal version of the code.

Reported by Joel Cunningham,
http://mailman.nginx.org/pipermail/nginx-devel/2016-March/007964.html.
2016-04-01 13:17:12 +03:00
Maxim Dounin
2e251b1c34 SSL: SSLeay_version() is deprecated in OpenSSL 1.1.0.
SSLeay_version() and SSLeay() are no longer available if OPENSSL_API_COMPAT
is set to 0x10100000L.  Switched to using OpenSSL_version() instead.

Additionally, we now compare version strings instead of version numbers,
and this correctly works for LibreSSL as well.
2016-03-31 23:38:38 +03:00
Ruslan Ermilov
5afd74bed6 Removed the prototype mysql module. 2016-03-31 22:33:26 +03:00
Ruslan Ermilov
5ad6e01b46 Fixed ngx_os_signal_process() prototype. 2016-03-31 22:00:33 +03:00
Sergey Kandaurov
6a716c6123 Fixed format specifiers in ngx_sprintf(). 2016-03-31 02:34:00 +03:00
Sergey Kandaurov
00ef9ff5f0 Fixed logging. 2016-03-31 02:33:57 +03:00
Sergey Kandaurov
c622ff3b44 Fixed logging with variable field width. 2016-03-31 02:33:53 +03:00
Sergey Kandaurov
2e25f340b8 Fixed logging in close error handling. 2016-03-31 02:33:50 +03:00
Ruslan Ermilov
7ad57da598 Style. 2016-03-30 11:52:16 +03:00
Ruslan Ermilov
dfbc51a8cc Version bump. 2016-03-30 11:52:07 +03:00
Maxim Dounin
6871615528 Win32: additional error code NGX_EEXIST_FILE (ticket #910).
On Windows there are two possible error codes which correspond to
the EEXIST error code: ERROR_FILE_EXISTS used by CreateFile(CREATE_NEW),
and ERROR_ALREADY_EXISTS used by CreateDirectory().

MoveFile() seems to use both: ERROR_ALREADY_EXISTS when moving within
one filesystem, and ERROR_FILE_EXISTS when copying a file to a different
drive.
2016-03-29 09:51:46 +03:00
Piotr Sikora
c3aed0a233 Core: allow strings without null-termination in ngx_parse_url().
This fixes buffer over-read while using variables in the "proxy_pass",
"fastcgi_pass", "scgi_pass", and "uwsgi_pass" directives, where result
of string evaluation isn't null-terminated.

Found with MemorySanitizer.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2016-02-26 17:30:27 -08:00
Roman Arutyunyan
030a1f959c Fixed socket inheritance on reload and binary upgrade.
On nginx reload or binary upgrade, an attempt is made to inherit listen sockets
from the previous configuration.  Previously, no check for socket type was made
and the inherited socket could have the wrong type.  On binary upgrade, socket
type was not detected at all.  Wrong socket type could lead to errors on that
socket due to different logic and unsupported syscalls.  For example, a UDP
socket, inherited as TCP, lead to the following error after arrival of a
datagram: "accept() failed (102: Operation not supported on socket)".
2016-03-25 14:10:38 +03:00
Dmitry Volyntsev
2e6e246794 Win32: fixed build after 384154fc634f. 2016-03-23 18:34:05 +03:00
Dmitry Volyntsev
f625ad7ce8 Resolver: added support for SRV records. 2016-03-23 17:44:36 +03:00
Dmitry Volyntsev
2b2fe92aaa Resolver: do not enable resolve timer if provided timeout is zero. 2016-03-23 17:44:36 +03:00
Dmitry Volyntsev
98f222c6c1 Resolver: introduced valid field in resolver responses.
It hints the amount of time a response could be considered
as valid.
2016-03-23 17:44:36 +03:00
Valentin Bartenev
0f4315f998 Core: introduced the NGX_DEBUG_PALLOC macro.
It allows to turn off accumulation of small pool allocations into a big
preallocated chunk of memory.  This is useful for debugging memory access
with sanitizer, since such accumulation can cover buffer overruns from
being detected.
2016-03-23 17:44:04 +03:00
Valentin Bartenev
9d08bda415 Core: use ngx_palloc_small() to allocate ngx_pool_large_t.
This structure cannot be allocated as a large block anyway, otherwise that will
result in infinite recursion, since each large allocation requires to allocate
another ngx_pool_large_t.

The room for the structure is guaranteed by the NGX_MIN_POOL_SIZE constant.
2016-03-23 17:44:04 +03:00
Valentin Bartenev
21dfe90a4e Core: introduced the ngx_palloc_small() function.
It deduplicates some code for allocations from memory pool.
No functional changes.
2016-03-23 17:44:04 +03:00
Valentin Bartenev
b94eb1dfe6 Core: moved logging before freeing large blocks of pool.
This fixes use-after-free memory access with enabled debug log
when pool->log is allocated as a large block.
2016-03-23 17:44:04 +03:00
Piotr Sikora
26f669f109 Fix build with -Wmissing-prototypes.
Broken in 5eb4d7541107 (1.9.6), fix somehow missed in 3600bbfb43e3.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2016-03-18 16:38:08 -07:00
Maxim Dounin
10c8c8d6a4 Threads: offloading of temp files writing to thread pools.
The ngx_thread_write_chain_to_file() function introduced, which
uses ngx_file_t thread_handler, thread_ctx and thread_task fields.
The task context structure (ngx_thread_file_ctx_t) is the same for
both reading and writing, and can be safely shared as long as
operations are serialized.

The task->handler field is now always set (and not only when task is
allocated), as the same task can be used with different handlers.

The thread_write flag is introduced in the ngx_temp_file_t structure
to explicitly enable use of ngx_thread_write_chain_to_file() in
ngx_write_chain_to_temp_file() when supported by caller.

In collaboration with Valentin Bartenev.
2016-03-18 06:44:03 +03:00
Maxim Dounin
ac78b26324 Threads: task pointer stored in ngx_file_t.
This simplifies the interface of the ngx_thread_read() function.

Additionally, most of the thread operations now explicitly set
file->thread_task, file->thread_handler and file->thread_ctx,
to facilitate use of thread operations in other places.

(Potential problems remain with sendfile in threads though - it uses
file->thread_handler as set in ngx_output_chain(), and it should not
be overwritten to an incompatible one.)

In collaboration with Valentin Bartenev.
2016-03-18 06:43:52 +03:00
Roman Arutyunyan
2ce791f2cd Stream: UDP proxy. 2016-01-20 19:52:12 +03:00
Sergey Kandaurov
db72c6be20 Fixed compilation with -Wmissing-prototypes. 2016-03-14 19:23:23 +03:00
Maxim Dounin
00fcc6a987 Copy filter: fixed sendfile aio handlers to set ctx->aio.
Sendfile handlers (aio preload and thread handler) are called within
ctx->output_filter() in ngx_output_chain(), and hence ctx->aio cannot
be set directly in ngx_output_chain().  Meanwhile, it must be set to
make sure loop within ngx_output_chain() will be properly terminated.

There are no known cases that trigger the problem, though in theory
something like aio + sub filter (something that needs body in memory,
and can also free some memory buffers) + sendfile can result in
"task already active" and "second aio post" alerts.

The fix is to set ctx->aio in ngx_http_copy_aio_sendfile_preload()
and ngx_http_copy_thread_handler().

For consistency, ctx->aio is no longer set explicitly in
ngx_output_chain_copy_buf(), as it's now done in
ngx_http_copy_thread_handler().
2016-03-03 21:14:19 +03:00
Ruslan Ermilov
ea9a2cf2ba Dynamic modules: removed unnecessary initialization.
It became unnecessary after 85dea406e18f.
2016-02-25 16:29:51 +03:00
Ruslan Ermilov
ceccebbb31 Version bump. 2016-02-25 16:28:42 +03:00
Vladimir Homutov
faa96e82d2 Core: added support for more than 64 CPUs in worker_cpu_affinity. 2016-02-18 13:58:49 +03:00
Maxim Dounin
79654d53e3 Core: improved logging on invalid NGINX variable (ticket #900). 2016-02-15 17:41:52 +03:00
Ruslan Ermilov
3b9b07377b Dynamic modules: fixed a version mismatch message (ticket #898).
Based on a patch by Takashi Takizawa.
2016-02-11 18:46:46 +03:00
Piotr Sikora
2fe20c9b34 Core: ngx_module_t compatibility with C++.
Changes to NGX_MODULE_V1 and ngx_module_t in 85dea406e18f (1.9.11)
broke all modules written in C++, because ISO C++11 does not allow
conversion from string literal to char *.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2016-02-09 16:31:26 -08:00
Maxim Dounin
839c1997d9 Version bump. 2016-02-10 06:23:34 +03:00
Maxim Dounin
97f59dda09 Dynamic modules.
The auto/module script is extended to understand ngx_module_link=DYNAMIC.
When set, it links the module as a shared object rather than statically
into nginx binary.  The module can later be loaded using the "load_module"
directive.

New auto/module parameter ngx_module_order allows to define module loading
order in complex cases.  By default the order is set based on ngx_module_type.

3rd party modules can be compiled dynamically using the --add-dynamic-module
configure option, which will preset ngx_module_link to "DYNAMIC" before
calling the module config script.

Win32 support is rudimentary, and only works when using MinGW gcc (which
is able to handle exports/imports automatically).

In collaboration with Ruslan Ermilov.
2016-02-04 20:25:29 +03:00
Maxim Dounin
86cc342a26 Dynamic modules: dlopen() support. 2016-02-04 18:30:21 +03:00
Maxim Dounin
0f67d6355c Dynamic modules: changed ngx_modules to cycle->modules. 2016-02-04 18:30:21 +03:00
Maxim Dounin
7988334570 Dynamic modules: moved module-related stuff to separate files. 2016-02-04 18:30:21 +03:00
Ruslan Ermilov
c6ed65ce7c Resolver: style.
Use the original query name in error and debug messages when
processing PTR responses.
2015-12-17 17:30:22 +03:00
Ruslan Ermilov
bf269cbc55 Resolver: improved PTR response processing.
The previous code only parsed the first answer, without checking its
type, and required a compressed RR name.

The new code checks the RR type, supports responses with multiple
answers, and doesn't require the RR name to be compressed.

This has a side effect in limited support of CNAME.  If a response
includes both CNAME and PTR RRs, like when recursion is enabled on
the server, PTR RR is handled.

Full CNAME support in PTR response is not implemented in this change.
2015-12-17 17:21:16 +03:00
Ruslan Ermilov
06806aa001 Resolver: style.
Renamed argument in ngx_resolver_process_a() for consistency.
2015-12-17 17:21:12 +03:00
Roman Arutyunyan
cbee15bb6c Resolver: free TCP buffers on resolver cleanup. 2016-02-02 13:42:55 +03:00
Ruslan Ermilov
625481f6f1 Resolver: fixed possible resource leak introduced in 5a16d40c63de.
Found by Coverity (CID 1351175).
2016-02-02 11:35:19 +03:00
Roman Arutyunyan
aa8fa81e42 Resolver: TCP support.
Resend DNS query over TCP once UDP response came truncated.
2016-01-28 15:28:20 +03:00
Roman Arutyunyan
c8c44f7478 Resolver: per-request DNS server balancer.
Previously, a global server balancer was used to assign the next DNS server to
send a query to.  That could lead to a non-uniform distribution of servers per
request.  A request could be assigned to the same dead server several times in a
row and wait longer for a valid server or even time out without being processed.

Now each query is sent to all servers sequentially in a circle until a
response is received or timeout expires.  Initial server for each request is
still globally balanced.
2016-01-28 15:28:20 +03:00
Roman Arutyunyan
1945fff7bf Resolver: renamed UDP-specific structures, fields and variables.
They will be used for TCP connections as well.
2016-01-28 15:28:20 +03:00
Roman Arutyunyan
52d3601155 Resolver: removed unused field from ngx_resolver_ctx_t. 2016-01-28 15:28:19 +03:00
Roman Arutyunyan
c541b82c21 Version bump. 2016-01-28 15:25:58 +03:00
Ruslan Ermilov
fe89d99796 Resolver: limited CNAME recursion.
Previously, the recursion was only limited for cached responses.
2016-01-26 16:47:14 +03:00
Roman Arutyunyan
a3d42258d9 Resolver: fixed use-after-free memory accesses with CNAME.
When several requests were waiting for a response, then after getting
a CNAME response only the last request's context had the name updated.
Contexts of other requests had the wrong name.  This name was used by
ngx_resolve_name_done() to find the node to remove the request context
from.  When the name was wrong, the request could not be properly
cancelled, its context was freed but stayed linked to the node's waiting
list.  This happened e.g. when the first request was aborted or timed
out before the resolving completed.  When it completed, this triggered
a use-after-free memory access by calling ctx->handler of already freed
request context.  The bug manifests itself by
"could not cancel <name> resolving" alerts in error_log.

When a request was responded with a CNAME, the request context kept
the pointer to the original node's rn->u.cname.  If the original node
expired before the resolving timed out or completed with an error,
this would trigger a use-after-free memory access via ctx->name in
ctx->handler().

The fix is to keep ctx->name unmodified.  The name from context
is no longer used by ngx_resolve_name_done().  Instead, we now keep
the pointer to resolver node to which this request is linked.
Keeping the original name intact also improves logging.
2016-01-26 16:46:59 +03:00
Roman Arutyunyan
b1a110e3a4 Resolver: changed the ngx_resolver_create_*_query() arguments.
No functional changes.

This is needed by the following change.
2016-01-26 16:46:48 +03:00
Ruslan Ermilov
4016e6b1da Resolver: fixed CNAME processing for several requests.
When several requests were waiting for a response, then after getting
a CNAME response only the last request was properly processed, while
others were left waiting.
2016-01-26 16:46:38 +03:00
Ruslan Ermilov
4b581a7c21 Resolver: fixed crashes in timeout handler.
If one or more requests were waiting for a response, then after
getting a CNAME response, the timeout event on the first request
remained active, pointing to the wrong node with an empty
rn->waiting list, and that could cause either null pointer
dereference or use-after-free memory access if this timeout
expired.

If several requests were waiting for a response, and the first
request terminated (e.g., due to client closing a connection),
other requests were left without a timeout and could potentially
wait indefinitely.

This is fixed by introducing per-request independent timeouts.
This change also reverts 954867a2f0a6 and 5004210e8c78.
2016-01-26 16:46:31 +03:00
Roman Arutyunyan
c44fd4e837 Resolver: fixed possible segmentation fault on DNS format error. 2016-01-26 16:46:18 +03:00
Maxim Dounin
d0bf684ab6 Core: worker_cpu_affinity auto.
If enabled, workers are bound to available CPUs, each worker to once CPU
in order.  If there are more workers than available CPUs, remaining are
bound in a loop, starting again from the first available CPU.

The optional mask parameter defines which CPUs are available for automatic
binding.

In collaboration with Vladimir Homutov.
2016-01-11 19:23:17 +03:00
Maxim Dounin
852a565be3 Version bump. 2015-12-17 16:38:51 +03:00
Ruslan Ermilov
fac1f23e5d Version bump. 2015-12-09 14:41:16 +03:00
Maxim Dounin
6994593478 Style: NGX_PTR_SIZE replaced with sizeof(void *).
The NGX_PTR_SIZE macro is only needed in preprocessor directives where
it's not possible to use sizeof().
2015-12-03 20:06:45 +03:00
Ruslan Ermilov
bfe1987b41 Core: enabled "include" inside http upstreams (ticket #635).
The directive already works inside stream upstream blocks.
2015-11-23 12:40:19 +03:00
Valentin Bartenev
3aef20d257 Version bump. 2015-11-17 19:01:41 +03:00