Input filter might free a buffer if there is no data in it, and in case
of first buffer (used for cache header and request header, aka p->buf_to_file)
this resulted in cache corruption. Buffer memory was reused to read upstream
response before headers were written to disk.
Fix is to avoid moving pointers in ngx_event_pipe_add_free_buf() to a buffer
start if we were asked to free a buffer used by p->buf_to_file.
This fixes occasional cache file corruption, usually resulted
in "cache file ... has md5 collision" alerts.
Reported by Anatoli Marinov.
This parameter allows to don't require certificate to be signed by
a trusted CA, e.g. if CA certificate isn't known in advance, like in
WebID protocol.
Note that it doesn't add any security unless the certificate is actually
checked to be trusted by some external means (e.g. by a backend).
Patch by Mike Kazantsev, Eric O'Connor.
With the "ssl_stapling_verify" commit build with old OpenSSL libraries
was broken due to incorrect prototype of the ngx_ssl_stapling() function.
One incorrect use of ngx_log_debug() instead of ngx_log_debug2() slipped in
and broke win32 build.
OCSP response verification is now switched off by default to simplify
configuration, and the ssl_stapling_verify allows to switch it on.
Note that for stapling OCSP response verification isn't something required
as it will be done by a client anyway. But doing verification on a server
allows to mitigate some attack vectors, most notably stop an attacker from
presenting some specially crafted data to all site clients.
This is expected to simplify configuration in a common case when OCSP
response is signed by a certificate already present in ssl_certificate
chain. This case won't need any extra trusted certificates.
This will result in better error message in case of incorrect response
from OCSP responder:
... OCSP responder sent invalid "Content-Type" header: "text/plain"
while requesting certificate status, responder: ...
vs.
... d2i_OCSP_RESPONSE() failed (SSL:
error:0D07209B:asn1 encoding routines:ASN1_get_object:too long
error:0D068066:asn1 encoding routines:ASN1_CHECK_TLEN:bad object header
error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error)
while requesting certificate status, responder: ...
This includes the ssl_stapling_responder directive (defaults to OCSP
responder set in certificate's AIA extension).
OCSP response for a given certificate is requested once we get at least
one connection with certificate_status extension in ClientHello, and
certificate status won't be sent in the connection in question. This due
to limitations in the OpenSSL API (certificate status callback is blocking).
Note: SSL_CTX_use_certificate_chain_file() was reimplemented as it doesn't
allow to access the certificate loaded via SSL_CTX.
Very basic version without any OCSP responder query code, assuming valid
DER-encoded OCSP response is present in a ssl_stapling_file configured.
Such file might be produced with openssl like this:
openssl ocsp -issuer root.crt -cert domain.crt -respout domain.staple \
-url http://ocsp.example.com
The directive allows to specify additional trusted Certificate Authority
certificates to be used during certificate verification. In contrast to
ssl_client_certificate DNs of these cerificates aren't sent to a client
during handshake.
Trusted certificates are loaded regardless of the fact whether client
certificates verification is enabled as the same certificates will be
used for OCSP stapling, during construction of an OCSP request and for
verification of an OCSP response.
The same applies to a CRL (which is now always loaded).
The SSL_COMP_get_compression_methods() is only available as an API
function in OpenSSL 0.9.8+, require it explicitly to unbreak build
with OpenSSL 0.9.7.
Previous code used sk_SSL_COMP_delete(ssl_comp_methods, i) while iterating
stack from 0 to n, resulting in removal of only even compression methods.
In real life this change is a nop, as there is only one compression method
which is enabled by default in OpenSSL.
We don't have strong reason to inform about any errors
reported by close() call here, and there are no other things
to do with its return value.
Prodded by Coverity.
In case of EMFILE/ENFILE returned from accept() we disable accept events,
and (in case of no accept mutex used) arm timer to re-enable them later.
With accept mutex we just drop it, and rely on normal accept mutex handling
to re-enable accept events once it's acquired again.
As we now handle errors in question, logging level was changed to "crit"
(instead of "alert" used for unknown errors).
Note: the code might call ngx_enable_accept_events() multiple times if
there are many listen sockets. The ngx_enable_accept_events() function was
modified to check if connection is already active (via c->read->active) and
skip it then, thus making multiple calls safe.
Instead of checking if there is events{} section present in configuration
in init_module handler we now do the same in init_conf handler. This
allows master process to detect incorrect configuration early and
reject it.
Integer overflow is undefined behaviour in C and this indeed caused
problems on Solaris/SPARC (at least in some cases). Fix is to
subtract unsigned integers instead, and then cast result to a signed
one, which is implementation-defined behaviour and used to work.
Strictly speaking, we should compare (unsigned) result with the maximum
value of the corresponding signed integer type instead, this will be
defined behaviour. This will require much more changes though, and
considered to be overkill for now.
Previous code incorrectly assumed that nodes with identical keys are linked
together. This might not be true after tree rebalance.
Patch by Lanshun Zhou.
With previous code raw buffer might be lost if p->input_filter() was called
on a buffer without any data and used ngx_event_pipe_add_free_buf() to
return it to the free list. This eventually might cause "all buffers busy"
problem, resulting in segmentation fault due to null pointer dereference in
ngx_event_pipe_write_chain_to_temp_file().
In ngx_event_pipe_add_free_buf() the buffer was added to the list start
due to pos == last, and then "p->free_raw_bufs = cl->next" in
ngx_event_pipe_read_upstream() dropped both chain links to the buffer
from the p->free_raw_bufs list.
Fix is to move "p->free_raw_bufs = cl->next" before calling the
p->input_filter().
Previously if ngx_add_event() failed a connection was freed two times (once
in the ngx_event_connect_peer(), and again by a caller) as pc->connection was
left set. Fix is to always use ngx_close_connection() to close connection
properly and set pc->connection to NULL on errors.
Patch by Piotr Sikora.
It's already called by OPENSSL_config(). Calling it again causes some
openssl engines (notably GOST) to corrupt memory, as they don't expect
to be created more than once.
Support for TLSv1.1 and TLSv1.2 protocols was introduced in OpenSSL 1.0.1
(-beta1 was recently released). This change makes it possible to disable
these protocols and/or enable them without other protocols.
It is currently used from master process on abnormal worker termination to
unlock accept mutex (unlocking of accept mutex was broken in 1.0.2). It is
expected to be used in the future to unlock other mutexes as well.
Shared mutex code was rewritten to make this possible in a safe way, i.e.
with a check if lock was actually held by the exited process. We again use
pid to lock mutex, and use separate atomic variable for a count of processes
waiting in sem_wait().
Stale write event may happen if epoll_wait() reported both read and write
events, and processing of the read event closed descriptor.
Patch by Yichun Zhang (agentzh).
If possible we now just extend already present file buffer in p->out chain
instead of keeping ngx_buf_t for each buffer we've flushed to disk. This
saves about 120 bytes of memory per buffer flushed to disk, and resolves
high CPU usage observed in edge cases (due to coalescing these buffers on
send).
1. In ngx_event_pipe_write_chain_to_temp_file() make sure to fully write
all shadow buffers up to last_shadow. With this change recycled buffers
cannot appear in p->out anymore. This also fixes segmentation faults
observed due to ngx_event_pipe_write_chain_to_temp() not freeing any
raw buffers while still returning NGX_OK.
2. In ngx_event_pipe_write_to_downstream() we now properly check for busy
size as a size of buffers, not a size of data in these buffers. This
fixes situations where all available buffers became busy (including
segmentation faults due to this).
3. The ngx_event_pipe_free_shadow_raw_buf() function is dropped. It's
incorrect and not needed.
If client closed connection in ngx_event_pipe_write_to_downstream(), buffers
in the "out" chain were lost. This caused cpu hog if all available buffers
were in the "out" chain. Fix is to call ngx_chain_update_chains() before
checking return code of output filter to avoid loosing buffers in the "out"
chain.
Note that this situation (all available buffers in the "out" chain) isn't
normal, it should be prevented by busy buffers limit. Though right now it
may happen with complex protocols like fastcgi. This should be addressed
separately.
The default value is 32 AIO simultaneous requests per worker. Previously
they were hardcoded to 1024, and it was too large, since Linux allocated
them early on io_setup(), but not on request itself. So with default value
of /proc/sys/fs/aio-max-nr equal to 65536 only 64 worker processes could
be run simultaneously. 32 AIO requests are enough for modern disks even if
server runs only 1 worker.
syscall(2) uses usual libc convention, it returns -1 on error and
sets errno. Obsolete _syscall(2) returns negative value of error.
Thanks to Hagai Avrahami.
As long as ngx_event_pipe() has more data read from upstream than specified
in p->length it's passed to input filter even if buffer isn't yet full. This
allows to process data with known length without relying on connection close
to signal data end.
By default p->length is set to -1 in upstream module, i.e. end of data is
indicated by connection close. To set it from per-protocol handlers upstream
input_filter_init() now called in buffered mode (as well as in
unbuffered mode).
The ngx_chain_update_chains() needs pool to free chain links used for buffers
with non-matching tags. Providing one helps to reduce memory consumption
for long-lived requests.
Setting read->eof to 0 seems to be just a typo. It appeared in
nginx-0.0.1-2003-10-28-18:45:41 import (r164), while identical code in
ngx_recv.c introduced in the same import do actually set read->eof to 1.
Failure to set read->eof to 1 results in EOF not being generally detectable
from connection flags. On the other hand, kqueue won't report any read
events on such a connection since we use EV_CLEAR. This resulted in read
timeouts if such connection was cached and used for another request.
If connection has unsent alerts, SSL_shutdown() tries to send them even
if SSL_set_shutdown(SSL_RECEIVED_SHUTDOWN|SSL_SENT_SHUTDOWN) was used.
This can be prevented by SSL_set_quiet_shutdown(). SSL_set_shutdown()
is required nevertheless to preserve session.
enabled in any server. The previous r1033 does not help when unused zone
becomes used after reconfiguration, so it is backed out.
The initial thought was to make SSL modules independed from SSL implementation
and to keep OpenSSL code dependance as much as in separate files.
and is shared among all hosts instead of pregenerating for every HTTPS host
on configuraiton phase. This decreases start time for configuration with
large number of HTTPS hosts.
