Friday, February 9, 2018

Nginx: readv() failed (104: Connection reset by peer) while reading upstream

Leave a Comment

I have read every other thread about this error with no luck.

I'm running Nginx 1.12.2 with PHP-FPM 5.6 on RHEL 6

I am getting these errors on any/all pages of my site. It is not related to slow requests. I have witnessed it personally and I immediately get a 500 error page without any waiting.

It affects both POST and GET requests.

I am not running any opcode caches (that I am aware of) I checked all config files, phpinfo() as well as php -v and did not see any mention of opcache or opcode.)

I have turned on catch_workers_output in PHP-FPM. I am not receiving any PHP errors

I have plenty of memory available and there are no errors in the system logs about memory, file descriptors or sockets.

Tried pm_max_requests with both high and low numbers

Not running xdebug plugin

Seems to happen at random times, even in the middle of the night when we aren't very busy at all.

I'm desperate for a solution to this issue which has gone on for months now.

Here's some of the information asked for in the comments below:

Nginx fastcgi.conf (differences from the stock file):

fastcgi_intercept_errors on; fastcgi_param  REDIRECT_STATUS    200; fastcgi_buffer_size   32k; fastcgi_buffers 8 16k; fastcgi_max_temp_file_size 0; fastcgi_keep_conn on; 

Nginx conf specifics:

sendfile        on; tcp_nopush      on; tcp_nodelay     on; server_tokens   off; gzip            off; gzip_static     on; gzip_comp_level 5; gzip_min_length 1024; types_hash_max_size 2048; client_body_timeout   900; client_header_timeout 900; keepalive_timeout     900; send_timeout          900; proxy_connect_timeout  900s; proxy_send_timeout  900s; proxy_read_timeout  900s; fastcgi_send_timeout 900s; fastcgi_read_timeout 900s; 

Nginx location specific config:

client_max_body_size 40M;  location ~ \.php$ {        fastcgi_pass   127.0.0.1:9000;        include        fastcgi.conf; } 

php-fpm config details:

listen = 127.0.0.1:9000 listen.allowed_clients = 127.0.0.1 pm = dynamic pm.max_children = 1000 pm.start_servers = 300 pm.min_spare_servers = 200 pm.max_spare_servers = 350 pm.max_requests = 1000 ; I've tried higher/lower with no difference slowlog = /var/log/php-fpm/www-slow.log catch_workers_output = yes php_admin_value[error_log] = /var/log/php_error_log php_admin_flag[log_errors] = on php_value[session.save_handler] = files php_value[session.save_path] = /var/lib/php/session 

Installed Versions:

php.x86_64                                                               5.6.33-1.el6.remi                                      @remi-php56                        php-bcmath.x86_64                                                        5.6.33-1.el6.remi                                      @remi-php56                        php-cli.x86_64                                                           5.6.33-1.el6.remi                                      @remi-php56                        php-common.x86_64                                                        5.6.33-1.el6.remi                                      @remi-php56                        php-devel.x86_64                                                         5.6.33-1.el6.remi                                      @remi-php56                        php-fedora-autoloader.noarch                                             1.0.0-1.el6.remi                                       @remi                              php-fpm.x86_64                                                           5.6.33-1.el6.remi                                      @remi-php56                        php-gd.x86_64                                                            5.6.33-1.el6.remi                                      @remi-php56                        php-gmp.x86_64                                                           5.6.33-1.el6.remi                                      @remi-php56                        php-imap.x86_64                                                          5.6.33-1.el6.remi                                      @remi-php56                        php-mbstring.x86_64                                                      5.6.33-1.el6.remi                                      @remi-php56                        php-mcrypt.x86_64                                                        5.6.33-1.el6.remi                                      @remi-php56                        php-mysqlnd.x86_64                                                       5.6.33-1.el6.remi                                      @remi-php56                        php-pdo.x86_64                                                           5.6.33-1.el6.remi                                      @remi-php56                        php-pear.noarch                                                          1:1.10.5-4.el6.remi                                    @remi                              php-pecl-igbinary.x86_64                                                 2.0.5-1.el6.remi.5.6                                   @remi-php56                        php-pecl-imagick.x86_64                                                  3.4.3-5.el6.remi.5.6                                   @remi-php56                        php-pecl-jsonc.x86_64                                                    1.3.10-2.el6.remi.5.6                                  @remi-php56                        php-pecl-jsonc-devel.x86_64                                              1.3.10-2.el6.remi.5.6                                  @remi-php56                        php-pecl-memcache.x86_64                                                 3.0.8-3.el6.remi.5.6                                   @remi-php56                        php-pecl-memcached.x86_64                                                2.2.0-3.el6.remi.5.6                                   @remi-php56                        php-pecl-msgpack.x86_64                                                  0.5.7-1.el6.remi.5.6                                   @remi-php56                        php-pecl-ssh2.x86_64                                                     0.13-3.el6.remi.5.6                                    @remi-php56                        php-pecl-zip.x86_64                                                      1.15.2-1.el6.remi.5.6                                  @remi-php56                        php-process.x86_64                                                       5.6.33-1.el6.remi                                      @remi-php56                        php-xml.x86_64                                                           5.6.33-1.el6.remi                                      @remi-php56    nginx version: nginx/1.12.2  Kernel: 2.6.32-696.20.1.el6 

0 Answers

If You Enjoyed This, Take 5 Seconds To Share It

0 comments:

Post a Comment