Commit Graph

1529 Commits

Author SHA1 Message Date
Maxim Dounin
9984f3053f Moved ngx_http_parse_time() to core, renamed accordingly.
The function is now called ngx_parse_http_time(), and can be used by
any code to parse HTTP-style date and time.  In particular, it will be
used for OCSP stapling.

For compatibility, a macro to map ngx_http_parse_time() to the new name
provided for a while.
2015-06-11 20:42:31 +03:00
Valentin Bartenev
66f6ba5c95 Fixed excessive memory usage while parsing configuration.
The b->pos points to the next symbol here.

Reported by ilexshen.
2015-06-01 21:08:56 +03:00
Ruslan Ermilov
087c31e8b2 Fixed bullying style of comments. 2015-05-29 09:26:33 +03:00
Ruslan Ermilov
3de18022d8 Version bump. 2015-05-29 09:26:27 +03:00
Maxim Dounin
f7f1607bf2 The "reuseport" option of the "listen" directive.
When configured, an individual listen socket on a given address is
created for each worker process.  This allows to reduce in-kernel lock
contention on configurations with high accept rates, resulting in better
performance.  As of now it works on Linux and DragonFly BSD.

Note that on Linux incoming connection requests are currently tied up
to a specific listen socket, and if some sockets are closed, connection
requests will be reset, see https://lwn.net/Articles/542629/.  With
nginx, this may happen if the number of worker processes is reduced.
There is no such problem on DragonFly BSD.

Based on previous work by Sepherosa Ziehau and Yingqi Lu.
2015-05-20 15:51:56 +03:00
Valentin Bartenev
9275dc19e6 Core: properly initialized written bytes counter in memory log. 2015-05-19 19:27:07 +03:00
Valentin Bartenev
89ad9ea309 Fixed overflow detection in ngx_inet_addr().
Overflow detection of the last octet might not work.

Reported by Sergey Polovko.
2015-04-28 18:55:03 +03:00
Valentin Bartenev
8568baef1f Version bump. 2015-04-28 18:54:48 +03:00
Maxim Dounin
2ec8cfcd7d Win32: shared memory base addresses and remapping.
Two mechanisms are implemented to make it possible to store pointers
in shared memory on Windows, in particular on Windows Vista and later
versions with ASLR:

- The ngx_shm_remap() function added to allow remapping of a shared memory
  zone to the address originally used for it in the master process.  While
  important, it doesn't solve the problem by itself as in many cases it's
  not possible to use the address because of conflicts with other
  allocations.

- We now create mappings at the same address in all processes by starting
  mappings at predefined addresses normally unused by newborn processes.

These two mechanisms combined allow to use shared memory on Windows
almost without problems, including reloads.

Based on the patch by Sergey Brester:
http://mailman.nginx.org/pipermail/nginx-devel/2015-April/006836.html
2015-04-27 18:25:42 +03:00
Maxim Dounin
153dbf3877 Win32: fixed shm.handle loss on reload. 2015-04-27 03:44:30 +03:00
Vladimir Homutov
df555ffce6 Core: the ngx_set_connection_log() macro.
The http and stream versions of this macro were identical.
2015-04-25 22:44:02 +03:00
Ruslan Ermilov
3180f84b60 Removed the "worker_rlimit_sigpending" directive.
It was only needed by the just removed rtsig module.
2015-04-23 14:26:11 +03:00
Ruslan Ermilov
33b8e5bc06 Removed the obsolete rtsig module. 2015-04-23 14:17:40 +03:00
Ruslan Ermilov
c1882d9f3f Removed the obsolete aio module. 2015-04-22 18:57:32 +03:00
Sergey Kandaurov
06ff117986 Core: allow shared memory size to be declared after a reference.
For example, this fixes the case when "proxy_cache_path" is specified
after "proxy_cache" that references it.
2015-04-20 16:53:04 +03:00
Ruslan Ermilov
c799c82faa Stream: port from NGINX+. 2015-04-20 13:05:11 +03:00
Ruslan Ermilov
64c5df4b62 Removed "worker_threads" and "thread_stack_size" directives. 2015-03-26 14:15:17 +03:00
Ruslan Ermilov
324e389e17 Removed unused thread-local-storage code. 2015-03-26 14:15:15 +03:00
Ruslan Ermilov
2b0d2f0583 Removed ngx_threaded and related code. 2015-03-26 14:15:13 +03:00
Sergey Kandaurov
d72fe76bb1 Core: ensure that ngx_config.h is always included first.
This fixes compilation of various 3rd party modules when nginx is
configured with threads.
2015-04-16 18:18:37 +03:00
Maxim Dounin
4091eb749b Version bump. 2015-04-16 17:26:19 +03:00
Ruslan Ermilov
cf31347ee8 Upstream: the "zone" directive.
Upstreams with the "zone" directive are kept in shared memory,
with a consistent view of all worker processes.
2015-04-14 19:01:25 +03:00
Ruslan Ermilov
3fc23e2542 Core: read/write locks. 2015-03-21 14:05:08 +03:00
Ruslan Ermilov
b25f0ddcf1 Fixed build, broken by 8b7f062a3fe6.
Casting a "const char *" to "char *" doesn't work on older gcc versions.
2015-04-16 15:05:40 +03:00
Vladimir Homutov
fd4da89228 Core: added OpenSSL version information to "nginx -V" output. 2015-04-16 12:17:41 +03:00
Vladimir Homutov
513f184098 Version bump. 2015-04-16 12:13:51 +03:00
Ruslan Ermilov
edb0c6ba64 Core: guard against spinlock usage without atomic ops.
The new thread pools code uses spinlocks.
2015-03-26 14:15:06 +03:00
Sergey Kandaurov
38a97edf3a Core: limited levels of subdirectory hierarchy used for temp files.
Similar to ngx_http_file_cache_set_slot(), the last component of file->name
with a fixed length of 10 bytes, as generated in ngx_create_temp_path(), is
used as a source for the names of intermediate subdirectories with each one
taking its own part.  Ensure that the sum of specified levels with slashes
fits into the length (ticket #731).
2015-04-07 01:32:08 +03:00
Sergey Kandaurov
256082c4f8 Core: removed excessive initialization in ngx_conf_set_path_slot().
Level hierarchy is pre-zeroed in ngx_pcalloc() of the surrounding ngx_path_t.
2015-04-07 01:32:07 +03:00
Sergey Kandaurov
eed7d47930 Core: fixed error handling on ngx_conf_full_name() failure. 2015-04-07 01:32:05 +03:00
Maxim Dounin
67986d81de Version bump. 2015-03-26 17:36:39 +03:00
Maxim Dounin
4df7aa6f92 Output chain: free chain links in ngx_chain_writer(). 2015-03-23 21:09:05 +03:00
Maxim Dounin
9131eb11c9 Output chain: zero size buf alerts in ngx_chain_writer().
Now we log a "zero size buf in chain writer" alert if we encounter a zero
sized buffer in ngx_chain_writer(), and skip the buffer.
2015-03-23 20:56:58 +03:00
Valentin Bartenev
e87a565aab Thread pools: implemented graceful exiting of threads. 2015-03-23 17:51:21 +03:00
Valentin Bartenev
510c9cdca8 Thread pools: removed unused pointer to memory pool.
No functional changes.
2015-03-23 17:51:21 +03:00
Valentin Bartenev
cb43696e0c Thread pools: keep waiting tasks mutex in ngx_thread_pool_t.
It's not needed for completed tasks queue since the previous change.

No functional changes.
2015-03-23 17:51:21 +03:00
Valentin Bartenev
f109a68b4f Thread pools: replaced completed tasks queue mutex with spinlock. 2015-03-23 17:51:21 +03:00
Ruslan Ermilov
07de3f538b Removed stub implementation of win32 mutexes. 2015-03-23 13:52:47 +03:00
Valentin Bartenev
cc569b9252 Core: added cyclic memory buffer support for error_log.
Example of usage:

  error_log memory:16m debug;

This allows to configure debug logging with minimum impact on performance.
It's especially useful when rare crashes are experienced under high load.

The log can be extracted from a coredump using the following gdb script:

  set $log = ngx_cycle->log

  while $log->writer != ngx_log_memory_writer
      set $log = $log->next
  end

  set $buf = (ngx_log_memory_buf_t *) $log->wdata
  dump binary memory debug_log.txt $buf->start $buf->end
2015-03-19 19:29:43 +03:00
Ruslan Ermilov
f8d10849ad Removed ngx_connection_t.lock. 2015-03-20 06:43:19 +03:00
Ruslan Ermilov
c34368715f Thread pools: silence warning on process exit.
Work around pthread_cond_destroy() and pthread_mutex_destroy() returning
EBUSY.  A proper solution would be to ensure all threads are terminated.
2015-03-19 23:20:18 +03:00
Ruslan Ermilov
afe1fcffaa Thread pools: fixed the waiting tasks accounting.
Behave like POSIX semaphores.  If N worker threads are waiting for tasks,
at least that number of tasks should be allowed to be put into the queue.
2015-03-19 13:00:48 +03:00
Ruslan Ermilov
20d07074e3 Thread pools: keep waiting tasks counter in ngx_thread_pool_t.
It's not needed for completed tasks queue.

No functional changes.
2015-03-19 23:19:35 +03:00
Valentin Bartenev
7ed1a9681b Added support for offloading Linux sendfile() in thread pools. 2015-03-14 17:37:30 +03:00
Valentin Bartenev
a7ad493aa6 Added support for offloading read() in thread pools. 2015-03-14 17:37:25 +03:00
Valentin Bartenev
305fc021db Thread pools implementation. 2015-03-14 17:37:07 +03:00
Ruslan Ermilov
83ba5ed2ec Renamed NGX_THREADS to NGX_OLD_THREADS because of deprecation.
It's mostly dead code and the original idea of worker threads has been rejected.
2015-03-04 18:26:25 +03:00
Ruslan Ermilov
a43f1bcf6e Overflow detection in ngx_inet_addr(). 2015-03-17 00:26:22 +03:00
Ruslan Ermilov
3ad3f4ae0a Core: overflow detection in ngx_parse_time() (ticket #732). 2015-03-17 00:26:20 +03:00
Ruslan Ermilov
7943fd374a Refactored ngx_parse_time().
No functional changes.
2015-03-17 00:26:18 +03:00
Ruslan Ermilov
8167ce8537 Core: overflow detection in number parsing functions. 2015-03-17 00:26:15 +03:00
Ruslan Ermilov
309928b941 Core: expose maximum values of time_t and ngx_int_t.
These are needed to detect overflows.
2015-03-17 00:24:34 +03:00
Valentin Bartenev
a9a01e0938 Log: use ngx_cpymem() in a couple of places, no functional changes. 2015-03-04 19:20:30 +03:00
Maxim Dounin
7dd799357d Core: fixed potential buffer overrun when initializing hash.
Initial size as calculated from the number of elements may be bigger
than max_size.  If this happens, make sure to set size to max_size.

Reported by Chris West.
2015-02-24 18:37:14 +03:00
Roman Arutyunyan
6337c1d185 Core: make ngx_connection_local_sockaddr() always assign address.
Previously, this function checked for connection local address existence
and returned error if it was missing.  Now a new address is assigned in this
case making it possible to call this function not only for accepted connections.
2015-02-17 14:26:44 +03:00
Valentin Bartenev
0a3d458805 Unbreak building on FreeBSD without file AIO.
It appeared that the NGX_HAVE_AIO_SENDFILE macro was defined regardless of
the "--with-file-aio" configure option and the NGX_HAVE_FILE_AIO macro.

Now they are related.

Additionally, fixed one macro.
2015-02-11 20:00:07 +03:00
Valentin Bartenev
2b8d6ad805 Refactored sendfile() AIO preload.
This reduces layering violation and simplifies the logic of AIO preread, since
it's now triggered by the send chain function itself without falling back to
the copy filter.  The context of AIO operation is now stored per file buffer,
which makes it possible to properly handle cases when multiple buffers come
from different locations, each with its own configuration.
2015-02-11 17:52:15 +03:00
Ruslan Ermilov
917c47a918 Version bump. 2015-02-11 15:51:03 +03:00
Sergey Kandaurov
66c25a7d77 Core: fixed build on Tru64 UNIX.
There was a typo in NGX_EACCES.

Reported by Goetz T. Fischer.
2015-02-10 01:51:08 +03:00
Roman Arutyunyan
2bae4205ab Core: fixed a race resulting in extra sem_post()'s.
The mtx->wait counter was not decremented if we were able to obtain the lock
right after incrementing it.  This resulted in unneeded sem_post() calls,
eventually leading to EOVERFLOW errors being logged, "sem_post() failed
while wake shmtx (75: Value too large for defined data type)".

To close the race, mtx->wait is now decremented if we obtain the lock right
after incrementing it in ngx_shmtx_lock().  The result can become -1 if a
concurrent ngx_shmtx_unlock() decrements mtx->wait before the added code does.
However, that only leads to one extra iteration in the next call of
ngx_shmtx_lock().
2015-02-04 16:22:43 +03:00
Roman Arutyunyan
cae1b5d63a Core: reverted prefix-based temp files (a9138c35120d).
The use_temp_path http cache feature is now implemented using a separate temp
hierarchy in cache directory.  Prefix-based temp files are no longer needed.
2015-02-02 21:28:09 +03:00
Roman Arutyunyan
54e14a387f Core: supported directory skipping in ngx_walk_tree().
If pre_tree_handler() returns NGX_DECLINED, the directory is ignored.
2015-02-02 19:38:32 +03:00
Valentin Bartenev
314d0d40e7 Fixed AIO handling in the output chain.
The ctx->aio flag must be taken into account in the short path too.
2015-01-28 21:33:06 +03:00
Igor Sysoev
415c8ec4b4 A bounds check of %N format on Windows.
Thanks to Joe Bialek, Adam Zabrocki and Microsoft Vulnerability Research.
2015-01-27 15:38:15 +03:00
Maxim Dounin
a1f330a0a2 Core: added disk_full_time checks to error log. 2015-01-13 19:51:37 +03:00
Valentin Bartenev
a54e37edda Core: added prefix-based temporary files.
Now, if the "path" parameter is NULL, ngx_create_temp_file() will use
file->name as a predefined file path prefix.
2014-12-26 16:22:54 +03:00
Valentin Bartenev
c07bebdcd1 Unified handling of ngx_create_temp_file() return value.
The original check for NGX_AGAIN was surplus, since the function returns
only NGX_OK or NGX_ERROR.  Now it looks similar to other places.

No functional changes.
2014-12-26 16:22:50 +03:00
Maxim Dounin
fe4b25c731 Version bump. 2014-12-24 18:53:38 +03:00
Valentin Bartenev
5453f0afe6 Autoindex: implemented JSON output format. 2014-12-12 20:25:35 +03:00
Maxim Dounin
811281ec36 Version bump. 2014-12-04 16:37:31 +03:00
Maxim Dounin
d228fa5093 Typo. 2014-11-28 16:57:23 +03:00
Vladimir Homutov
742b5dd045 Syslog: allowed underscore symbol in tag (ticket #667). 2014-11-20 20:02:21 +03:00
Ruslan Ermilov
ef563debf1 Resolver: fixed debug event logging.
In 954867a2f0a6, we switched to using resolver node as the timer event data.
This broke debug event logging.

Replaced now unused ngx_resolver_ctx_t.ident with ngx_resolver_node_t.ident
so that ngx_event_ident() extracts something sensible when accessing
ngx_resolver_node_t as ngx_connection_t.
2014-11-20 15:24:42 +03:00
Ruslan Ermilov
d4b7b74686 Resolver: fixed use-after-free memory access.
In 954867a2f0a6, we switched to using resolver node as the
timer event data, so make sure we do not free resolver node
memory until the corresponding timer is deleted.
2014-11-20 15:24:40 +03:00
Valentin Bartenev
d85d459bb2 Moved the code for coalescing file buffers to a separate function. 2014-08-13 15:11:45 +04:00
Valentin Bartenev
41ed973456 Renamed ngx_handle_sent_chain() to ngx_chain_update_sent().
No functional changes.
2014-11-19 21:16:19 +03:00
Valentin Bartenev
246152c864 Events: removed broken thread support from event timers.
It's mostly dead code.  And the idea of thread support for this task has
been deprecated.
2014-08-25 13:34:39 +04:00
Valentin Bartenev
78dd8b5eda Version bump. 2014-10-28 21:58:49 +03:00
Yichun Zhang
52e4dc2f74 Core: fixed buffer overrun when hash max_size reached. 2014-10-02 12:00:17 -07:00
Valentin Bartenev
70995077ad Version bump. 2014-10-02 22:36:27 +04:00
Vladimir Homutov
27fa3123f9 Syslog: improved error handling of unix domain sockets.
If a syslog daemon is restarted and the unix socket is used, further logging
might stop to work.  In case of send error, socket is closed, forcing
a reconnection at the next logging attempt.
2014-08-26 14:56:54 +04:00
Vladimir Homutov
d79cbf15e6 Syslog: enabled logging of send errors.
The ngx_cycle->log is used when sending the message.  This allows to log syslog
send errors in another log.

Logging to syslog after its cleanup handler has been executed was prohibited.
Previously, this was possible from ngx_destroy_pool(), which resulted in error
messages caused by attempts to write into the closed socket.

The "processing" flag is renamed to "busy" to better match its semantics.
2014-09-01 17:55:07 +04:00
Valentin Bartenev
4aac91049f Moved the code for adjusting sent buffers in a separate function. 2014-08-13 15:11:45 +04:00
Valentin Bartenev
681f74a9ba Version bump. 2014-09-17 12:04:47 +04:00
Maxim Dounin
c5bee22d16 Core: ngx_regex_compile() error handling fixes.
Now we actually return NGX_ERROR on errors, and provide an error
string for memory allocation errors.

Reported by Markus Linnala.
2014-09-08 21:35:53 +04:00
Valentin Bartenev
37d24e7e3b Events: processing of posted events changed from LIFO to FIFO.
In theory, this can provide a bit better distribution of latencies.

Also it simplifies the code, since ngx_queue_t is now used instead
of custom implementation.
2014-09-01 18:20:18 +04:00
Valentin Bartenev
2a81e05566 Events: removed broken thread support from posted events.
It's mostly dead code.  And the idea of thread support for this task has
been deprecated.
2014-09-01 18:20:03 +04:00
Ruslan Ermilov
da46eed537 Resolver: notify all waiting requests on timeout.
If a "resolver_timeout" occurs, only the first waiting request
was notified.  Other requests may hang forever.
2014-08-19 15:43:26 +04:00
Maxim Dounin
4b5876c922 Version bump. 2014-08-05 22:50:25 +04:00
Tatsuhiko Kubo
244a6a28b1 Core: improved ngx_pstrdup() error handling. 2014-08-01 20:04:14 +09:00
Piotr Sikora
3ba8f2f73d Core: use uppercase hexadecimal digits for percent-encoding.
RFC3986 says that, for consistency, URI producers and normalizers
should use uppercase hexadecimal digits for all percent-encodings.

This is also what modern web browsers and other tools use.

Using lowercase hexadecimal digits makes it harder to interact with
those tools in case when use of the percent-encoded URI is required,
for example when $request_uri is part of the cache key.

Signed-off-by: Piotr Sikora <piotr@cloudflare.com>
2014-06-26 23:39:23 -07:00
Vladimir Homutov
c54899ddfd Core: fixed default log initialization.
The ngx_log_insert() function may invalidate pointer passed to it,
so make sure to don't use it after the ngx_log_insert() call.
2014-07-24 16:25:07 +04:00
Ruslan Ermilov
7e52432a05 Resolver: fixed resend on malformed responses.
DNS request resend on malformed responses was broken in 98876ce2a7fd (1.5.8).

Reported by Pramod Korathota.
2014-07-16 10:21:28 +04:00
Jianjun Zheng
584692027b Core: bugfix for the ngx_slab_max_size case. 2014-06-18 16:39:59 +08:00
Tatsuhiko Kubo
ef2b59699a Style: use ngx_str_null(). 2014-07-09 23:22:14 +09:00
Tatsuhiko Kubo
bb4edb5cef Style: use ngx_strlen() instead of strlen(). 2014-07-09 23:20:40 +09:00
Maxim Dounin
098c6bd81f Version bump. 2014-07-09 03:15:00 +04:00
Maxim Dounin
9d4de05be4 Core: removed meaningless check from ngx_palloc_block().
The check became meaningless after refactoring in 2a92804f4109.
With the loop currently in place, "current" can't be NULL, hence
the check can be dropped.

Additionally, the local variable "current" was removed to
simplify code, and pool->current now used directly instead.

Found by Coverity (CID 714236).
2014-06-26 03:34:19 +04:00
Maxim Dounin
1a5cdafa82 Core: plugged socket leak during configuration test.
This isn't really important as configuration testing shortly ends with
a process termination which will free all sockets, though Coverity
complains.

Prodded by Coverity (CID 400872).
2014-06-26 03:34:05 +04:00
Ruslan Ermilov
05d717b35d Core: added ngx_slab_calloc() and ngx_slab_calloc_locked().
These functions return zeroed memory, analogous to ngx_pcalloc().
2014-06-04 15:09:19 +04:00