Clearing cache based on free space left on a file system is
expected to allow better disk utilization in some cases, notably
when disk space might be also used for something other than nginx
cache (including nginx own temporary files) and while loading
cache (when cache size might be inaccurate for a while, effectively
disabling max_size cache clearing).
Based on a patch by Adam Bambuch.
NFS on Linux is known to report wsize as a block size (in both f_bsize
and f_frsize, both in statfs() and statvfs()). On the other hand,
typical file system block sizes on Linux (ext2/ext3/ext4, XFS) are limited
to pagesize. (With FAT, block sizes can be at least up to 512k in
extreme cases, but this doesn't really matter, see below.)
To avoid too aggressive cache clearing on NFS volumes on Linux, block
sizes larger than pagesize are now ignored.
Note that it is safe to ignore large block sizes. Since 3899:e7cd13b7f759
(1.0.1) cache size is calculated based on fstat() st_blocks, and rounding
to file system block size is preserved mostly for Windows.
Note well that on other OSes valid block sizes seen are at least up
to 65536. In particular, UFS on FreeBSD is known to work well with block
and fragment sizes set to 65536.
A special last buffer with cl->buf->pos set to NULL can be present in
a chain when writing request body if chunked encoding was used. This
resulted in a NULL pointer dereference if it happened to be the only
buffer left after a do...while loop iteration in ngx_write_chain_to_file().
The problem originally appeared in nginx 1.3.9 with chunked encoding
support. Additionally, rev. 3832b608dc8d (nginx 1.9.13) changed the
minimum number of buffers to trigger this from IOV_MAX (typically 1024)
to NGX_IOVS_PREALLOCATE (typically 64).
Fix is to skip such buffers in ngx_chain_to_iovec(), much like it is
done in other places.
The ngx_thread_write_chain_to_file() function introduced, which
uses ngx_file_t thread_handler, thread_ctx and thread_task fields.
The task context structure (ngx_thread_file_ctx_t) is the same for
both reading and writing, and can be safely shared as long as
operations are serialized.
The task->handler field is now always set (and not only when task is
allocated), as the same task can be used with different handlers.
The thread_write flag is introduced in the ngx_temp_file_t structure
to explicitly enable use of ngx_thread_write_chain_to_file() in
ngx_write_chain_to_temp_file() when supported by caller.
In collaboration with Valentin Bartenev.
This simplifies the interface of the ngx_thread_read() function.
Additionally, most of the thread operations now explicitly set
file->thread_task, file->thread_handler and file->thread_ctx,
to facilitate use of thread operations in other places.
(Potential problems remain with sendfile in threads though - it uses
file->thread_handler as set in ngx_output_chain(), and it should not
be overwritten to an incompatible one.)
In collaboration with Valentin Bartenev.
This fixes suboptimal behavior caused by surplus lseek() for sequential writes
on systems without pwrite(). A consecutive read after write might result in an
error on systems without pread() and pwrite().
Fortunately, at the moment there are no widely used systems without these
syscalls.
This will result in alphabetical sorting of included files if
the "include" directive with wildcards is used.
Note that the behaviour is now different from that on Windows, where
alphabetical sorting is not guaranteed for FindFirsFile()/FindNextFile()
(used to be alphabetical on NTFS, but not on FAT).
Approved by Igor Sysoev, prodded by many.
Catched by dav_chunked.t on Solaris. In released versions this might
potentially result in corruption of complex protocol responses if they
were written to disk and there were more distinct buffers than IOV_MAX
in a single write.
The only thing we could potentially do here in case of error
returned is to complain to error log, but we don't have log
structure available here due to interface limitations.
Prodded by Coverity.
Previously result of last iteration's writev() was returned. This was
unnoticed as return value was only used if chain contained only one or
two buffers.
this fixes slash after link to a directory in ngx_http_autoindex_module;
*) use cached dirent.d_type as hint on all systems
the issues has been introduced in r2235
therefore fallback to stat() if dirent.d_type is not set,
this fixes slash after directory name in ngx_http_autoindex_module;
the issue has been introduced in r2235
*) Change in internal API: the HTTP modules initialization was moved
from the init module phase to the HTTP postconfiguration phase.
*) Change: now the request body is not read beforehand for the
ngx_http_perl_module: it's required to start the reading using the
$r->has_request_body method.
*) Feature: the ngx_http_perl_module supports the DECLINED return code.
*) Feature: the ngx_http_dav_module supports the incoming "Date" header
line for the PUT method.
*) Feature: the "ssi" directive is available inside the "if" block.
*) Bugfix: a segmentation fault occurred if there was an "index"
directive with variables and the first index name was without
variables; the bug had appeared in 0.1.29.
*) Feature: the ngx_http_dav_module.
*) Change: the ngx_http_perl_module optimizations.
Thanks to Sergey Skvortsov.
*) Feature: the ngx_http_perl_module supports the $r->request_body_file
method.
*) Feature: the "client_body_in_file_only" directive.
*) Workaround: now on disk overflow nginx tries to write access logs
once a second only.
Thanks to Anton Yuzhaninov and Maxim Dounin.
*) Bugfix: now the "limit_rate" directive more precisely limits rate if
rate is more than 100 Kbyte/s.
Thanks to ForJest.
*) Bugfix: now the IMAP/POP3 proxy escapes the "\r" and "\n" symbols in
login and password to pass authorization server.
Thanks to Maxim Dounin.
*) Change: the "variables_hash_max_size" and
"variables_hash_bucket_size" directives.
*) Feature: the $body_bytes_sent variable can be used not only in the
"log_format" directive.
*) Feature: the $ssl_protocol and $ssl_cipher variables.
*) Feature: the cache line size detection for widespread CPUs at start
time.
*) Feature: now the "accept_mutex" directive is supported using
fcntl(2) on platforms different from i386, amd64, sparc64, and ppc.
*) Feature: the "lock_file" directive and the --with-lock-path=PATH
autoconfiguration directive.
*) Bugfix: if the HTTPS protocol was used in the "proxy_pass" directive
then the requests with the body was not transferred.
*) Change: the "valid_referers" directive and the "$invalid_referer"
variable were moved to the new ngx_http_referer_module from the
ngx_http_rewrite_module.
*) Change: the "$apache_bytes_sent" variable name was changed to
"$body_bytes_sent".
*) Feature: the "$sent_http_..." variables.
*) Feature: the "if" directive supports the "=" and "!=" operations.
*) Feature: the "proxy_pass" directive supports the HTTPS protocol.
*) Feature: the "proxy_set_body" directive.
*) Feature: the "post_action" directive.
*) Feature: the ngx_http_empty_gif_module.
*) Feature: the "worker_cpu_affinity" directive for Linux.
*) Bugfix: the "rewrite" directive did not unescape URI part in
redirect, now it is unescaped except the %00-%25 and %7F-%FF
characters.
*) Bugfix: nginx could not be built by the icc 9.0 compiler.
*) Bugfix: if the SSI was enabled for zero size static file, then the
chunked response was encoded incorrectly.
*) Feature: the Sun Studio 10 C compiler support.
*) Feature: the "proxy_upstream_max_fails",
"proxy_upstream_fail_timeout", "fastcgi_upstream_max_fails", and
"fastcgi_upstream_fail_timeout" directives.
*) Bugfix: nginx did run on Linux parisc.
*) Feature: nginx now does not start under FreeBSD if the sysctl
kern.ipc.somaxconn value is too big.
*) Bugfix: if a request was internally redirected by the
ngx_http_index_module module to the ngx_http_proxy_module or
ngx_http_fastcgi_module modules, then the index file was not closed
after request completion.
*) Feature: the "proxy_pass" can be used in location with regular
expression.
*) Feature: the ngx_http_rewrite_filter_module module supports the
condition like "if ($HTTP_USER_AGENT ~ MSIE)".
*) Bugfix: nginx started too slow if the large number of addresses and
text values were used in the "geo" directive.
*) Change: a variable name must be declared as "$name" in the "geo"
directive. The previous variant without "$" is still supported, but
will be removed soon.
*) Feature: the "%{VARIABLE}v" logging parameter.
*) Feature: the "set $name value" directive.
*) Bugfix: gcc 4.0 compatibility.
*) Feature: the --with-openssl-opt=OPTIONS autoconfiguration directive.
