Why does this nginx server redirect to a wrong domain?

Why does this nginx server redirect to a wrong domain? My nginx webserver has two domains to serve, server1.eu and server2.eu, why do they interfere with each other?

When forcing nginx only to listen to IPv4 and browsing to server1.eu there's a SSL certificate mismatch (only valid for server2.eu) and after forcing accept the certificate there's a 301 redirect to server2.eu

With server2.eu removed from sites-enabled, and server1.eu IPv4 disabled in nginx server setup, it results in "no connection to server" when browsing to server1.eu

With server2.eu removed from sites-enabled, and server1.eu IPv6 disabled from nginx server setup, it works seemingly normal and there is no SSL certificate mismatch and no 301 redirect.

Again restoring server2.eu and disabling server1.eu IPv4 from nginx server setup, results in a SSL certificate mismatch (only valid for server2.eu) and after forcing accept certificate 301 redirect to server2.eu

find /etc/nginx/{conf.d,sites-enabled} gives

/etc/nginx/sites-enabled/server1.eu  /etc/nginx/sites-enabled/server2.eu  

The zone file records:

AAAA    server1.eu    directs to IPv6 address A   server1.eu    directs to IPv4 address AAAA    server2.eu  directs to IPv6 address A server2.eu directs to IPv4 address 

the nginx server configuration:

server {     listen 80;     listen [::]:80;     return 301 https://www.server1.eu; } server {         listen 443 ssl http2;         listen [::]:443 ssl http2;         ssl_certificate /etc/letsencrypt/live/server1.eu/fullchain.pem;         ssl_certificate_key /etc/letsencrypt/live/server1.eu/privkey.pem;         include snippets/ssl-params.conf;          server_name www.server1.eu;         root /var/www/server1.eu/webroot;         index index.php index.html index.htm ;          location / {             try_files $uri $uri/ /index.php?$args;         }          location ~ \.php$ {             include fastcgi.conf;             fastcgi_pass unix:/run/php/php7.2-fpm.sock;         } } 

and

server {     listen 80;     listen [::]:80;     server_name www.server2.eu;     return 301 https://$server_name$request_uri; }  server {         listen 443 ssl http2;         listen [::]:443 ssl http2;     ssl_certificate /etc/letsencrypt/live/server2.eu/fullchain.pem;     ssl_certificate_key /etc/letsencrypt/live/server2.eu/privkey.pem;     include snippets/ssl-params.conf;     server_name www.server2.eu;     root /var/www/server2.eu/webroot;     index index.php index.html index.htm ;          location / {             try_files $uri $uri/ /index.php?$args;         }          location ~ \.php$ {             include fastcgi.conf;             fastcgi_pass unix:/run/php/php7.2-fpm.sock;         } } 

1 Answers

Answers 1

The nginx webserver listening to IPv6 takes precedence over IPv4 and interferes with SNI. Testing with removing servers reveals the default behaviour of nginx.

Remove all servers except server 1, with IPv4 and IPv6 enabled, reload nginx, then activate server 2, with only an IPv4 listener and reload nginx again. Browsing to server 2 will let you end up at server 1. It appears that nginx automatically listens to the first added IPv6. So interchanging the sequence of activation will switch the routing.

Read More

sendfile() failed (32: Broken pipe) while sending request to upstream, request: "POST

I'm having issue when uploading file on production using meteor with nginx + passenger. I'm also using meteor files for uploading files. it worked great in development but i can't upload files in production. i got and error in browser console:

POST http://my-url/ net::ERR_INCOMPLETE_CHUNKED_ENCODING 200 (OK)

i found error in my passenger log file saying Not keep-aliving application session connection because application did not allow it here is the log:

[ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/HttpChunkedBodyParser.h:183 ]: [Client 63] ChunkedBodyParser: parsing new chunk [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/HttpChunkedBodyParser.h:123 ]: [Client 63] ChunkedBodyParser: chunk size determined: 982 bytes [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/HttpChunkedBodyParser.h:162 ]: [Client 63] ChunkedBodyParser: parsing 982 of 982 bytesof remaining chunk data; 0 now remaining [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:1416 ]: [FBC 0x7f71e801b670] Feeding 982 bytes [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:486 ]: [FBC 0x7f71e801b670] pushBuffer() completed: nbuffers = 1, bytesBuffered = 982 [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:554 ]: [FBC 0x7f71e801b670] Reader: reading next [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:586 ]: [FBC 0x7f71e801b670] Reader: found buffer, 982 bytes [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:493 ]: [FBC 0x7f71e801b670] popBuffer() completed: nbuffers = 0,bytesBuffered = 0 [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:594 ]: [FBC 0x7f71e801b670] Reader: feeding buffer, 982 bytes [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:554 ]: [FBC 0x7f71e801b670] Reader: reading next [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:561 ]: [FBC 0x7f71e801b670] Reader: no more buffers. Transitioning to RS_INACTIVE [ D3 2018-09-27 16:53:44.2194 2500/Ta Ser/FileBufferedChannel.h:539 ]: [FBC 0x7f71e801b670] Calling dataFlushedCallback [ D3 2018-09-27 16:53:44.2194 2500/Ta age/Cor/Con/ForwardResponse.cpp:64 ]: [Client 2-63] Event: onAppSourceData [ D3 2018-09-27 16:53:44.2194 2500/Ta age/Cor/Con/ForwardResponse.cpp:206 ]: [Client 2-63] Processing 7 bytes of application data: "\r\n0\r\n\r\n" [ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:248 ]: [Client 63] ChunkedBodyParser: done parsing a chunk [ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:183 ]: [Client 63] ChunkedBodyParser: parsing new chunk [ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:123 ]: [Client 63] ChunkedBodyParser: chunk size determined: 0bytes [ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:162 ]: [Client 63] ChunkedBodyParser: parsing 0 of 0 bytes of remaining chunk data; 0 now remaining [ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:164 ]: [Client 63] ChunkedBodyParser: end chunk detected [ D3 2018-09-27 16:53:44.2195 2500/Ta Ser/HttpChunkedBodyParser.h:267 ]: [Client 63] ChunkedBodyParser: end chunk reached [ D2 2018-09-27 16:53:44.2195 2500/Ta age/Cor/Con/ForwardResponse.cpp:224 ]: [Client 2-63] End of application response body reached [ D2 2018-09-27 16:53:44.2195 2500/Ta age/Cor/Con/ForwardResponse.cpp:1077 ]: [Client 2-63] Not keep-aliving application sessionconnection because application did not allow it [ D3 2018-09-27 16:53:44.2195 2500/Ta age/Cor/App/Socket.h:201 ]: Socket unix:/tmp/passenger.toEIX2t/apps.s/node.1si9u5: connection not checked back into connection pool. There are now 2 connections in total [ D2 2018-09-27 16:53:44.2195 2500/Ta age/Cor/App/Gro/SessionManagement.cpp:150 ]: Session closed for process (pid=2519, group=/var/www/hmn/bundle (production))

• Meteor 1.6.1
• Meteor-Files 1.9.11
• Ubuntu 16.04
• nginx 1.14.0
• passenger 5.3.5
• AWS

this is my nginx config file

    user www-data;     worker_processes  1;      events {         worker_connections  1024;     }       http {     include /etc/nginx/mime.types;     default_type  application/octet-stream;      sendfile        on;     # tcp_nopush     on;     # tcp_nodelay on;     server_tokens off;      keepalive_timeout 65;     types_hash_max_size 2048;      server_names_hash_bucket_size 64;      access_log /var/log/nginx/access.log;     error_log /var/log/nginx/error.log;      gzip  on;     gzip_disable "MSIE [1-6]\.";      gzip_proxied any;     gzip_http_version 1.0;     gzip_min_length 500;     gzip_types    text/plain text/xml text/css                   text/comma-separated-values                   text/javascript                   application/x-javascript                   application/atom+xml;      include /etc/nginx/passenger.conf;     include /etc/nginx/conf.d/*.conf;     include /etc/nginx/sites-enabled/*;     }      // sites-enabled/* file     server {         listen 80 default_server;         listen [::]:80 default_server;         server_name _;         return 301 https://$host$request_uri;     }     server{         listen 443 default_server ssl;         listen [::]:443 default_server ssl;         ssl on;         ssl_certificate    /etc/ssl/mydomain.pem;         ssl_certificate_key    /etc/ssl/mydomain.key;         ssl_dhparam /etc/ssl/dhparam.pem;         server_name xxx.xxx.xxx.xxx;         passenger_enabled on;         passenger_sticky_sessions on;         root /var/www/my_app/bundle/public;         passenger_app_type node;         passenger_startup_file main.js;         passenger_env_var MONGO_URL mongodb://some_shard_urls;         passenger_env_var ROOT_URL https://xxx.xxx.xxx.xxx;         passenger_env_var MONGO_OPLOG_URL: mongodb://some_shard_urls;         keepalive_timeout  1000;         ssl_session_timeout 1d;         ssl_session_cache shared:SSL:50m;         ssl_session_tickets off;         ssl_prefer_server_ciphers on;         ssl_protocols TLSv1 TLSv1.1 TLSv1.2;         ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GC$         ssl_stapling on;         ssl_stapling_verify on;          add_header Strict-Transport-Security "max-age=31536000;";         access_log  /var/log/my_app/access.log;         error_log   /var/log/my_app/error.log;          proxy_send_timeout 60s;          location / {                 proxy_set_header Connection "";                 proxy_http_version 1.1;                 proxy_redirect     off;                 client_max_body_size 100M;                 if ($uri != '/') {                     expires 30d;                 }                 break;         } } 

I'm using aws S3 to hosting the file and meteor files has feature to integrate with s3. here is my sample code:

 const ProductAssets = new FilesCollection({     debug: false,     collectionName: 'collection_name',     allowClientCode: false,     storagePath: './products',     permissions: 0777,     // debug: true,     chunkSize: 'dynamic',     parentDirPermissions: 0777,     onBeforeUpload: function(file) {         if (/png|jpe?g/i.test(file.extension)) {             return true;         }     },     onAfterUpload(fileRef) {         Meteor.call('upload.to.s3', fileRef)     }, });  // upload to s3 method 'upload.to.s3': function(fileRef) {     _.each(fileRef.versions, async (vRef, version) => {         const filePath = 'products/' + fileRef._id + '/' + version + '-' + fileRef._id + '.' + fileRef.extension;         await s3.putObject({             // ServerSideEncryption: 'AES256',             StorageClass: 'STANDARD',             Bucket: bucket,             Key: filePath,             Body: fs.createReadStream(vRef.path),             ContentType: vRef.type         }, (error, data) => {             bound(() => {                 if (error) {                     console.error(error);                 } else {                     // Update FilesCollection with link to the file at AWS                     const upd = { $set: {} };                     upd['$set']['versions.' + version + '.meta.pipePath'] = filePath;                      ProductAssets.collection.update({                         _id: fileRef._id                     }, upd, (updError) => {                         if (updError) {                             console.error(updError);                         } else {                             // Unlink original files from FS after successful upload to AWS:S3                             ProductAssets.unlink(ProductAssets.collection.findOne(fileRef._id), version);                         }                     });                 }             })         });     }) }, 

on side note, files that is being uploaded doesn't saved into database and s3 on production. on development, the files is saved and uploaded to s3 properly

anybody know how to fix it and explain what is wrong? thank you!

0 Answers

Read More

nginx returns 403 forbidden error on file upload

I have a php website, in codeigniter, which allows users to upload files. Upload works fine for most of files. But for few files nginx throws 403 Forbidden error. like

<html> <head><title>403 Forbidden</title></head> <body bgcolor="white"> <center><h1>403 Forbidden</h1></center> </body> </html> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> 

This is one of the sample .rb file that fails to upload

# Sample code from Programing Ruby, page 58 string = <<END_OF_STRING     The body of the string     is the input lines up to     one ending with the same     text that followed the '<<' END_OF_STRING 

If i remove '<<' from this file, upload works. Where is this filtration happening?

I have seen nginx error logs, nothing in there. I have seen codeigniter logs, nothing there. Infact upload request doesn't reach my Codeigniter controller, so must be blocked before reaching there by nginx?

Here is upload code in Javascript

function upload(file, params) {     var formData = new FormData();     formData.append("Filedata", file);     $.each(params, function(key, value) {        formData.append(key, value);      });      var xhr = new XMLHttpRequest();     var action = "/upload/file";      xhr.upload.onprogress = function(e){         // show progress with e.loaded, e.total     };      xhr.onerror = function(e) {          // handle error     };      xhr.open("POST", action, true);     xhr.send(formData); } 

On server side, I have simple code for now.

if( !isset($_FILES['Filedata']) || !file_exists($_FILES['Filedata']['tmp_name']) ) {     die('File not submitted.'); } else {      // Save file code is here } 

1 Answers

Answers 1

I don't have a solution, but probably related to your single quotes. Doesn't .append have it read through that file? Wouldn't it terminate on the single quote?
Essentially going:

if( !isset($_FILES['# Sample code from Programing Ruby, page 58   string = <<END_OF_STRING       The body of the string       is the input lines up to       one ending with the same       text that followed the '   

Read More

nginx proxy forward and stream_socket_get_name

How to make stream_socket_get_name return the real IP address from the remote client?

$ip = stream_socket_get_name($socket, true); 

The above returns something like 127.0.0.1:39872

nginx

server {     listen 8443 ssl;     server_name  websocket.example.com;      ssl_certificate  /var/ini/ssl/public.crt;     ssl_certificate_key  /var/ini/ssl/private.key;      location / {         proxy_redirect  off;          proxy_pass  http://127.0.0.1:9000/;         proxy_http_version  1.1;         proxy_set_header  Upgrade $http_upgrade;         proxy_set_header  Connection "upgrade";         proxy_read_timeout  300;          proxy_set_header  X-Real-IP $remote_addr;         proxy_set_header  X-Forwarded-Proto https;         proxy_set_header  X-Forwarded-For $remote_addr;         proxy_set_header  X-Forwarded-Host $remote_addr;     } } 

1 Answers

Answers 1

The problem is stream_socket_get_name() operates on a file handle. In this case the file handle always connects to your proxy, so you can only get the proxy information. But you see those proxy_set_header directives? Those are how the remote IP is and remote port could be passed in. In your PHP, you'd have to check the values for the headers.

Read More

DNS and nginx server setup problem, slow server and 502 response

I'm setting up a new server with Ubuntu 18.04.1 Nginx 1.14.0 and PHP 7.2.7 Everything works fine except a test page where I set up a lot of broken links to missing images.

It seems to take forever for the server to realise they are missing and respond to the http request. Some missing files give a HTTP status of 404 and some give 502. What causes these delays and 502 errors? Did I do something wrong in the nginx or php configuration?

On my old server I have the exact same page (https://vuyk.eu/portfolio-2) which loads very quick. There must be a difference in server setup that I would like to solve.

Edit: After a suggestion from Dayo I did some tests. It seems to be a DNS problem. When I remove the line "listen [::]:443 ssl http2;" in the nginx server conf file the problem is gone. Still why would this be a problem?

Edit: When accessing IP 2a03:b0c0:0:1010::190:6001 through a browser, there is a certificate mismatch notification. This is strange because the nginx server setup (see contents listed below) leads both IPv4 and IPv6 to the same certificate.

Edit: So the server doesn't recognize the IPv6 address being test.vuyk.eu but accessing IP https://37.139.19.66 immediately shows https://test.vuyk.eu

The zone file records:

AAAA    test.vuyk.eu    directs to 2a03:b0c0:0:1010::190:6001 3600 A   test.vuyk.eu    directs to 37.139.19.66           3600 

Dayo suggested the hosts file might be a problem, here is the contents:

127.0.0.1 localhost ::1 localhost 2a03:b0c0:0:1010::190:6001 localhost 127.0.1.1 vuykhost2.vuyk.eu   # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts 

The nginx server configuration, when I remove the line "listen [::]:443 ssl http2;" everything works fine:

server {         listen 443 ssl http2;         listen [::]:443 ssl http2;         ssl_certificate /etc/letsencrypt/live/test.vuyk.eu/fullchain.pem;         ssl_certificate_key /etc/letsencrypt/live/test.vuyk.eu/privkey.pem;         include snippets/ssl-params.conf;          server_name test.vuyk.eu;         root /var/www/vuyk.eu/webroot;         index index.php index.html index.htm ;          location / {             try_files $uri $uri/ /index.php?$args;         }          location ~ \.php$ {             include fastcgi.conf;             fastcgi_pass unix:/run/php/php7.2-fpm.sock;         } } 

nginx.conf

user www-data; worker_processes auto; pid /run/nginx.pid; include /etc/nginx/modules-enabled/*.conf;  events {     worker_connections 2048;     multi_accept on; }  http {      ##     # Basic Settings     ##      sendfile on;     tcp_nopush on;     tcp_nodelay on;     #   keepalive_timeout 65;     types_hash_max_size 2048;     # server_tokens off;      # server_names_hash_bucket_size 64;     # server_name_in_redirect off;      include /etc/nginx/mime.types;     default_type application/octet-stream;      ##     # SSL Settings     ##      ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE     ssl_prefer_server_ciphers on;      ##     # Logging Settings     ##      access_log /var/log/nginx/access.log;     error_log /var/log/nginx/error.log;      ##     # Gzip Settings     ##      gzip             on;     gzip_comp_level  2;     gzip_min_length  1000;     gzip_proxied     expired no-cache no-store private auth;     gzip_types       text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;      ##     # Virtual Host Configs     ##      include /etc/nginx/conf.d/*.conf;     include /etc/nginx/sites-enabled/*;     client_body_buffer_size 10K;     client_header_buffer_size 1k;     client_max_body_size 100m;     large_client_header_buffers 4 8k;     fastcgi_buffers 16 16k;     fastcgi_buffer_size 32k;     fastcgi_read_timeout 500; #gateway probleem     client_body_timeout 12;     client_header_timeout 12;     keepalive_timeout 25;     send_timeout 10; } 

The php app I use is Joomla 3.8.11 with a custom script to show a custom error page:

header("HTTP/1.0 404 Not Found");  echo file_get_contents('https://test.vuyk.eu/404-page-not-found');  exit; 

After removing file_get_contents there are no errors anymore. However I'm wondering why, as it used to work fine on my old server (see edit above about DNS). Also I need this script to properly show a HTTP status 404 and a custom error page without changing the addressbar.

A part of the nginx error.log:

2018/08/30 16:25:27 [error] 29228#29228: *76 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 2a02:a440:91e3:1:4481:654b:a3e8:9617, server: test.vuyk.eu, request: "GET /images/klanten1/JHoogeveen.gif HTTP/2.0", upstream: "fastcgi://unix:/run/php/php7.2-fpm.sock:", host: "test.vuyk.eu", referrer: "https://test.vuyk.eu/portfolio-2" 

Messages from the php7.2-fpm.log (there are a lot similar lines)

[30-Aug-2018 16:16:08] WARNING: [pool www] server reached pm.max_children setting (15), consider raising it [30-Aug-2018 16:16:27] WARNING: [pool www] child 29026, script '/var/www/vuyk.eu/webroot/index.php' (request: "GET /index.php") execution timed out (22.937711 sec), terminating [30-Aug-2018 16:16:27] WARNING: [pool www] child 29245 exited on signal 15 (SIGTERM) after 20.490546 seconds from start [30-Aug-2018 16:16:27] NOTICE: [pool www] child 29263 started 

The timeline of HTTP requests and replies, that shows the requests on non-existent files and the response of the server, some give a 404 which is good, some give 502 bad gateway (on my old server they're all 404's):

GET https://test.vuyk.eu/portfolio-2 [HTTP/2.0 200 OK 132ms] GET https://test.vuyk.eu/templates/purity_iii/css/bootstrap.css [HTTP/2.0 200 OK 40ms] GET https://test.vuyk.eu/templates/system/css/system.css [HTTP/2.0 200 OK 50ms] GET https://test.vuyk.eu/templates/purity_iii/css/template.css [HTTP/2.0 200 OK 50ms] GET https://test.vuyk.eu/templates/purity_iii/fonts/font-awesome/css/font-awesome.min.css [HTTP/2.0 200 OK 50ms] GET https://test.vuyk.eu/templates/purity_iii/css/layouts/corporate.css [HTTP/2.0 200 OK 50ms] GET https://test.vuyk.eu/media/jui/js/jquery.min.js?48b6d1b3850bca834b403c58682b4579 [HTTP/2.0 200 OK 60ms] GET https://test.vuyk.eu/media/jui/js/jquery-noconflict.js?48b6d1b3850bca834b403c58682b4579 [HTTP/2.0 200 OK 60ms] GET https://test.vuyk.eu/media/jui/js/jquery-migrate.min.js?48b6d1b3850bca834b403c58682b4579 [HTTP/2.0 200 OK 60ms] GET https://test.vuyk.eu/media/system/js/caption.js?48b6d1b3850bca834b403c58682b4579 [HTTP/2.0 200 OK 70ms] GET https://test.vuyk.eu/plugins/system/t3/base-bs3/bootstrap/js/bootstrap.js? 8b6d1b3850bca834b403c58682b4579 [HTTP/2.0 200 OK 80ms] GET https://test.vuyk.eu/plugins/system/t3/base-bs3/js/jquery.tap.min.js [HTTP/2.0 200 OK 80ms] GET https://test.vuyk.eu/plugins/system/t3/base-bs3/js/script.js [HTTP/2.0 200 OK 70ms] GET https://test.vuyk.eu/plugins/system/t3/base-bs3/js/menu.js [HTTP/2.0 200 OK 70ms] GET https://test.vuyk.eu/templates/purity_iii/js/script.js [HTTP/2.0 200 OK 70ms] GET https://test.vuyk.eu/plugins/system/t3/base-bs3/js/nav-collapse.js [HTTP/2.0 200 OK 70ms] GET https://test.vuyk.eu/templates/purity_iii/css/custom-vuyk.css [HTTP/2.0 200 OK 70ms] GET https://test.vuyk.eu/images/klanten1/schipper2.gif [HTTP/2.0 502 Bad Gateway 23988ms] GET https://test.vuyk.eu/images/klanten1/Kuiper.gif [HTTP/2.0 502 Bad Gateway 24038ms] GET https://test.vuyk.eu/images/klanten1/WindMatch.gif [HTTP/2.0 502 Bad Gateway 24008ms] GET https://test.vuyk.eu/images/klanten1/Tuinland.gif [HTTP/2.0 502 Bad Gateway 24018ms] GET https://test.vuyk.eu/images/klanten1/Wezenberg.gif [HTTP/2.0 502 Bad Gateway 24038ms] GET https://test.vuyk.eu/images/klanten1/Morgenster.gif [HTTP/2.0 502 Bad Gateway 23998ms] GET https://test.vuyk.eu/images/klanten1/Harrie-boerhof.gif [HTTP/2.0 502 Bad Gateway 24028ms] GET https://test.vuyk.eu/images/klanten1/Lococensus.gif [HTTP/2.0 502 Bad Gateway 23998ms] GET https://test.vuyk.eu/images/klanten1/JHoogeveen.gif [HTTP/2.0 502 Bad Gateway 23978ms] GET https://test.vuyk.eu/images/klanten1/DeDeur.gif [HTTP/2.0 502 Bad Gateway 23988ms] GET https://test.vuyk.eu/images/klanten1/Runhaar.gif [HTTP/2.0 502 Bad Gateway 23958ms] GET https://test.vuyk.eu/images/klanten1/Schunselaar-schildersbedrijf.gif [HTTP/2.0 502 Bad Gateway 23948ms] GET https://test.vuyk.eu/images/klanten1/Capelle.gif [HTTP/2.0 502 Bad Gateway 23958ms] GET https://test.vuyk.eu/images/klanten1/Distantlake.gif [HTTP/2.0 502 Bad Gateway 24038ms] GET https://test.vuyk.eu/images/klanten1/Eikenaar.gif [HTTP/2.0 502 Bad Gateway 24018ms] GET https://test.vuyk.eu/images/klanten1/FFWD.gif [HTTP/2.0 404 Not Found 26274ms] GET https://test.vuyk.eu/images/klanten1/Veltec.gif [HTTP/2.0 404 Not Found 26791ms] GET https://test.vuyk.eu/images/klanten1/Heutink.gif [HTTP/2.0 404 Not Found 26811ms] GET https://test.vuyk.eu/images/klanten1/Lindeboom.gif [HTTP/2.0 404 Not Found 26777ms] GET https://test.vuyk.eu/images/klanten1/aataxi.gif [HTTP/2.0 404 Not Found 26828ms] GET https://test.vuyk.eu/images/klanten1/Aewind.gif [HTTP/2.0 404 Not Found 26811ms] GET https://test.vuyk.eu/images/klanten1/Praatmaatgroep.gif [HTTP/2.0 404 Not Found 26800ms] GET https://test.vuyk.eu/media/system/css/system.css [HTTP/2.0 200 OK 20ms] JQMIGRATE: Migrate is installed, version 1.4.1 jquery-migrate.min.js:2:542 GET https://test.vuyk.eu/images/logo.gif [HTTP/2.0 200 OK 20ms] GET https://test.vuyk.eu/images/reclame-en-communicatie.gif [HTTP/2.0 200 OK 20ms] GET https://test.vuyk.eu/fonts/opensans-regular-webfont.woff [HTTP/2.0 200 OK 40ms] GET https://test.vuyk.eu/templates/purity_iii/fonts/font-awesome/fonts/fontawesome-webfont.woff2?v=4.7.0 [HTTP/2.0 200 OK 70ms] 

fastcgi.conf

fastcgi_param  PATH_TRANSLATED    $document_root$fastcgi_path_info; fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name; fastcgi_param  PATH_INFO          $fastcgi_path_info; fastcgi_param  QUERY_STRING       $query_string; fastcgi_param  REQUEST_METHOD     $request_method; fastcgi_param  CONTENT_TYPE       $content_type; fastcgi_param  CONTENT_LENGTH     $content_length;  fastcgi_param  SCRIPT_NAME        $fastcgi_script_name; fastcgi_param  REQUEST_URI        $request_uri; fastcgi_param  DOCUMENT_URI       $document_uri; fastcgi_param  DOCUMENT_ROOT      $document_root; fastcgi_param  SERVER_PROTOCOL    $server_protocol; fastcgi_param  REQUEST_SCHEME     $scheme; fastcgi_param  HTTPS              $https if_not_empty;  fastcgi_param  GATEWAY_INTERFACE  CGI/1.1; fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;  fastcgi_param  REMOTE_ADDR        $remote_addr; fastcgi_param  REMOTE_PORT        $remote_port; fastcgi_param  SERVER_ADDR        $server_addr; fastcgi_param  SERVER_PORT        $server_port; fastcgi_param  SERVER_NAME        $server_name;  # PHP only, required if PHP was built with --enable-force-cgi-redirect fastcgi_param  REDIRECT_STATUS    200; 

php.ini

[PHP]  engine = On short_open_tag = Off precision = 14 output_buffering = 4096 zlib.output_compression = Off implicit_flush = Off unserialize_callback_func = serialize_precision = -1 disable_functions = pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wifcontinued,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_get_handler,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,pcntl_async_signals, disable_classes = zend.enable_gc = On expose_php = Off max_execution_time = 30 max_input_time = 60 memory_limit = 128M error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT display_errors = Off display_startup_errors = Off log_errors = On log_errors_max_len = 1024 ignore_repeated_errors = Off ignore_repeated_source = Off report_memleaks = On html_errors = On variables_order = "GPCS" request_order = "GP" register_argc_argv = Off auto_globals_jit = On post_max_size = 28M auto_prepend_file = auto_append_file = default_mimetype = "text/html" default_charset = "UTF-8" doc_root = user_dir = enable_dl = Off cgi.fix_pathinfo=1 file_uploads = On upload_max_filesize = 20M max_file_uploads = 20 allow_url_fopen = On allow_url_include = Off default_socket_timeout = 60 cli_server.color = On date.timezone = "Europe/Amsterdam"  [Pdo_mysql] pdo_mysql.cache_size = 2000 pdo_mysql.default_socket= [mail function] SMTP = localhost smtp_port = 25 mail.add_x_header = Off  [ODBC] odbc.allow_persistent = On odbc.check_persistent = On odbc.max_persistent = -1 odbc.max_links = -1 odbc.defaultlrl = 4096 odbc.defaultbinmode = 1  [Interbase] ibase.allow_persistent = 1 ibase.max_persistent = -1 ibase.max_links = -1 ibase.timestampformat = "%Y-%m-%d %H:%M:%S" ibase.dateformat = "%Y-%m-%d" ibase.timeformat = "%H:%M:%S"  [MySQLi] mysqli.max_persistent = -1 mysqli.allow_persistent = On mysqli.max_links = -1 mysqli.cache_size = 2000 mysqli.default_port = 3306 mysqli.default_socket = mysqli.default_host = mysqli.default_user = mysqli.default_pw = mysqli.reconnect = Off [mysqlnd] mysqlnd.collect_statistics = On mysqlnd.collect_memory_statistics = Off  [PostgreSQL] pgsql.allow_persistent = On pgsql.auto_reset_persistent = Off pgsql.max_persistent = -1 pgsql.max_links = -1 pgsql.ignore_notice = 0 pgsql.log_notice = 0  [bcmath] bcmath.scale = 0  [Session] session.save_handler = files session.use_strict_mode = 0 session.use_cookies = 1 session.use_only_cookies = 1 session.name = PHPSESSID session.auto_start = 0 session.cookie_lifetime = 0 session.cookie_path = / session.cookie_domain = session.cookie_httponly = session.serialize_handler = php session.gc_probability = 0 session.gc_divisor = 1000 session.gc_maxlifetime = 1440 session.referer_check = session.cache_limiter = nocache session.cache_expire = 180 session.use_trans_sid = 0 session.sid_length = 26 session.trans_sid_tags = "a=href,area=href,frame=src,form=" session.sid_bits_per_character = 5  [Assertion] zend.assertions = -1  [mbstring] mbstring.func_overload = 0  [Tidy] tidy.clean_output = Off  [soap] soap.wsdl_cache_enabled=1 soap.wsdl_cache_dir="/tmp" soap.wsdl_cache_ttl=86400 soap.wsdl_cache_limit = 5 [ldap] ldap.max_links = -1 

3 Answers

Answers 1

Heres whats happening.

It says in your error log server reached pm.max_children setting (15), consider raising it

So the max.children limit of 15 means PHP-FPM will stop launching processes once an app has 15 processes running, and any more requests for processes which come in will be queued until one of the previous processes ends.

You are using a php script to generate a 404 page, you then load a page with a load of broken links, your Nginx try files directive ends with a php script:

try_files $uri $uri/ /index.php?$args;

From the Nginx docs that means:

If none of the files were found, an internal redirect to the uri specified in the last parameter is made.

So for every broken link you just added an extra php process to the queue. If you count your 502 errors in the log you'll see there are 15. Because Nginx looks for 15 /index.php?$args which it can't find so tries to display a 404 which guess what? Is generated in php and now everything is broken.

15 processes which cant return 404 because the process limit has been reached and they each need another process to generate a 404 page, so until they time out no more processes for you.

The whole idea of serving a 404 page this way is crazy anyway. It's a static page, you should be serving it from Nginx because web servers are really really good at delivering static content fast, passing it to php, which in turn requests it from your own server again makes absolutely no sense.

Download your custom page to a file:

curl -o /var/www/vuyk.eu/webroot/404.html https://test.vuyk.eu/404-page-not-found 

Now add an error page directive in your Nginx conf:

error_page 404 /404.html; 

and now you have Nginx serving a custom error page without changing the client url and absolutely no load on your server.

Answers 2

It's seems that images you try to load are unavailable and requests are passed to PHP where 404 page is generated. Your custom 404 page is fetching resources via http

echo file_get_contents('https://test.vuyk.eu/404-page-not-found');  

If this fetch is slow your script might execute for very long time which might lead to timeouts. Also this might result in requests piling up and draining server resources.

Try to replace this fetch with something faster, you can try to read/include 404 page data directly from filesystem.

Answers 3

There are two broad possibilities as to why this is slow on the new server.

1. Problems with Webserver / PHP
2. Problems with DNS

To troubleshoot, enter the command line on your server and try to fetch a missing file using wget or cUrl. If you get a response as fast as you expect, then you most likely have an issue with your Webserver / PHP. If it is also slow, then the issue is with the DNS setup on your new server.

In any case, it appears that using file_get_contents for external URLs can lead to funky results. (Yes, The files are on your server but as you have a full url, it is treated like an external url).

So instead of ...

echo file_get_contents('https://test.vuyk.eu/404-page-not-found');  

use

echo file_get_contents('/server/path/to/404-page-not-found'); 

If you can't do this because 404-page-not-found is not a physical file and has to be run through Joomla to be generated, then why not use cUrl instead? This is specifically for 'external' files.

function curlFile($url) {   $ch = curl_init();    curl_setopt($ch, CURLOPT_HEADER, 0);   curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);   curl_setopt($ch, CURLOPT_URL, $url);    $ret = curl_exec($ch);   curl_close($ch);    return $ret; }  echo curlFile('https://test.vuyk.eu/404-page-not-found'); 

Note that if you found a DNS issue, you will need to resolve that notwithstanding.

Read More

URL requested a HTTP redirect, but it could not be followed. - Facebook/Nginx issue

I have used Facebooks sharing debugger to highlight an issue on the website

URL requested a HTTP redirect, but it could not be followed. 

https://developers.facebook.com/tools/debug/sharing/?q=https%3A%2F%2Fwww.badgerbookings.com

This is also stopping it accepting the url in the privacy policy when creating an app.

I have researched and made sure to add all OG meta tags. I also "reduced" down the redirects on my nginx to only support a http > https redirect which to me seems pretty standard.

It still produces the error on both the debugger and the Privacy Policy URL.

My Nginx config:

server_tokens off; #Enables or disables emitting nginx version on error pages and in the “Server” response header field  map $http_upgrade $connection_upgrade {         default upgrade;         '' close; }  server {     listen 80;     server_name _;     return 301 https://www.badgerbookings.com$request_uri; }  server {         server_name www.badgerbookings.com  badgerbookings.com *.badgerbookings.com;          location / {             proxy_pass http://localhost:3000;                 proxy_http_version 1.1;                 proxy_set_header Upgrade $http_upgrade; # allow websockets                 proxy_set_header Connection $connection_upgrade;                 proxy_set_header X-Forwarded-For $remote_addr; # preserve client IP                 proxy_set_header Host $http_host;                 proxy_set_header X-Forward-Proto http;                 proxy_set_header X-Nginx-Proxy true;          }      listen 443 ssl; # managed by Certbot     ssl_certificate /etc/letsencrypt/live/badgerbookings.com-0001/fullchain.pem; # managed by Certbot     ssl_certificate_key /etc/letsencrypt/live/badgerbookings.com-0001/privkey.pem; # managed by Certbot     include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot     ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot  } 

2 Answers

Answers 1

Go to Facebook Developer's policy page.

Scroll down to this bit:

Privacy Policy

a. Provide a publicly available and easily accessible privacy policy that explains what data you are collecting and how you will use that data.

Now run

curl https://badgerbookings.com/terms

Are you looking at an easily accessible privacy policy which is publicly available at that url?

Answers 2

You maybe having IPv6 issues which can be resolved as simple as adding a listen [::]:443 ssl directive in you SSL server block.

If that doesn't fix it, try redirecting with a matching if directive

if ($scheme != "https") {     return 301 https://www.badgerbookings.com$request_uri } 

This is best if you unite both server blocks in one, to avoid more code. Just delete the non-https one and insert port 80 listen directives on the other one as well, with that conditional redirect, this way your code will be even slimmer.

Read More

Apache two apps one domain share language /en - Magento & Wordpress

We have Wordpress in the root / in a physical subfolder /wp and Magento in /products.

We are wanting to make the sites multi-language using sub folders e.g domain.com/en

The problem arises as magento appends the store code (language) after the url so we have

domain.com/en (wordpress) domain.com/products/en (magento) 

Naturally we would like

domain.com/en domain.com/en/products 

Now it's very easy to make it work with some rewrite rule

RewriteRule ^(.*)/products/?(.*)$ /products/$1 [L] 

But still we have an issue as Magento generates the links as /products/en it's possible to start modifying where these links are generated like in

\Magento\Store\Model\Store  

In the _updatePathUseStoreView function, this doesn't seem to handle all links though

In general seems like a bad solution, another idea is to use Apache mod_substitute also seems bad practice, and overhead.

Another option is to have both apps in the root and have some lookup logic to see which url belongs to which app.

Any ideas for a setup that can purely use just Nginx/Apache. That does not compromise on having unique url's or regex'ing content.

This is my .htaccess in the root

<IfModule mod_rewrite.c> RewriteEngine on  RewriteCond %{HTTP_HOST} ^(www.)?domain.com$ RewriteCond %{REQUEST_URI} !^/wp/ RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d #RewriteCond %{REQUEST_URI} !^/(.*)/products RewriteRule ^(.*)$ /wp/$1 RewriteCond %{HTTP_HOST} ^(www.)?domain.com$ RewriteRule ^(/)?$ wp/index.php [L]  RewriteCond %{REQUEST_URI} ^/(.*)/products RewriteRule ^(.*)$ /products/index.php [L]  </IfModule> 

The exact spec I'm trying to achieve is this.

• Wordpress is installed in /wp , Magento in /products
• Language codes via subfolders used on both sites to appear as /en/wordpress-page /en/products/magento-page

Attempt 1 Use base link URL entering /en/products there and keeping the base URL as /products

as the first request is forwarded I had to work the setEnv like so in the root .htaccess

RewriteCond %{REQUEST_URI} ^/(.*)/products RewriteRule ^(.*)$ /products/index.php [E=MAGE_RUN_CODE:%1] [L] 

then in /products/.htaccess

RewriteCond "%{ENV:REDIRECT_MAGE_RUN_CODE}" RewriteRule .* - [E=MAGE_RUN_CODE:%{ENV:REDIRECT_MAGE_RUN_CODE}] [L] 

I checked the code was coming through on index.php by doing

 echo getenv('MAGE_RUN_CODE'); 

In my case the store code is "en" etc.. but the language switcher does not work it hits Magento but gets 404 even thought the store code is definitely coming through.

2 Answers

Answers 1

You only need some configuration from backoffice.

System => Configuration => General => Web => Url options

Add Store Code to Urls No

System => Configuration => General => Web => Unsecure

Base Link URL http://example.com/en/products/

System => Configuration => General => Web => Secure

Base Link URL https://example.com/en/products/

Then, add a rule in htaccess to set the correct store code:

SetEnvIf Host .*example.com/en* MAGE_RUN_CODE=en_store SetEnvIf Host .*example.com/fr* MAGE_RUN_CODE=fr_store

Answers 2

What is the exact spec you're trying to achieve?

Do you have multiple pages like /products, and multiple languages like /en?

I did something similar at BXR.SU — I didn't like the way OpenGrok, my backend, was handling the URLs, so, I would automatically make my nginx fix the URLs on top of OpenGrok, seamlessly fixing the URLs presented to the user, whereas the backend would continue to use the old URLs (e.g., with the /xref/ for most pages, which I don't like, and was set to remove with nginx); this approach appears to be similar to your spec, where you want to do this on the front-end web-server without doing any modifications to the backend.

The approach is briefly described at nginx redirect loop, remove index.php from url, with the idea being that nginx has two types of redirects — internal, where the contents of the $uri variable gets changed (without any visibility to the user), and external, where a 301 Moved (or some such) response is provided to the client (and the user would then see the browser making a request with the new URL).

E.g., you may want to have something like the following:

location /en/ {     # issue an external redirect, unless we're here from an internal one     if ($request_uri ~ "^(/en)(/product)(.*)") {         return 301 $2$1$3; # external redirect     }     proxy_pass …; } location /products/ {     rewrite ^(/products/)(en/)(.*) $2$1$3 last; # internal redirect     … } 

Read More

Application failed to start on shiny web server after a few days

I have a running shiny app on a web server that worked fine until I would say last week. Now, on occasion (I guess every two days) the app stops working with the "Application failed to start" message. When I restart the shiny server, as I did just now, everything runs fine again.

https://butterlab.imb-mainz.de/flydev/

The funny thing is, I have other apps on this server as well, and they are not affected and run fine in parallel, even if this app failed.

I can not find any error message in the log files. And I am wondering: how I could debug this, since the app is now running fine?

Looking forward to any advice.

EDIT:
I checked the shiny-server.log file after the error occurred and I found the following message:

[2018-06-14 14:29:20.080] [WARN] shiny-server - RobustSockJS collision: MqU4rgur76RPgjJIPr [2018-06-15 01:28:18.398] [WARN] shiny-server - Error handling message: Error: Discard position id too big [2018-06-15 02:00:10.358] [INFO] shiny-server - Error getting worker: Error: The application took too long to respond. [2018-06-15 02:00:10.364] [INFO] shiny-server - Error getting worker: Error: The application took too long to respond. 

The last message gets repeated whenever someone accesses the server.

0 Answers

Read More

Nginx pointed to wrong directory with Docker on Windows

I'm setting up a Laravel application with Docker, using a Docker image configuration I found here: https://blog.pusher.com/docker-for-development-laravel-php/

Now, this works fine on my Ubuntu machine (16.04), but on Window (10 Pro) I get a weird error. It first complains about not finding a composer.json file. Then, with each request I make to localhost:8000, I get the following error:

15#15: *1 open() "/var/www/public404" failed (2: No such file or directory), client: 172.17.0.1, server: , request: "GET / HTTP/1.1", host: "localhost:8000" 

I am very new to this, but it seems that nginx points to /var/www/public404 - I have no idea how that "404" got there. I have a feeling it has to do with the line try_files $uri = 404; in the site.conf file, however, I don't really know how that works and I don't want to break it... The weird thing is that this works with Ubuntu, but not on Windows (or maybe that's not weird at all?).

I use docker build . -t my-image to build the image and docker run -p 8000:80 --name="my-container" my-image to run a container using the image.

The EOL of all the config files is set to line feed. Does anybody have any idea how I might fix this?

Dockerfile

FROM nginx:mainline-alpine LABEL maintainer="John Doe <john@doe>"  COPY start.sh /start.sh COPY nginx.conf /etc/nginx/nginx.conf COPY supervisord.conf /etc/supervisord.conf COPY site.conf /etc/nginx/sites-available/default.conf  RUN apk add --update \ php7 \ php7-fpm \ php7-pdo \ php7-pdo_mysql \ php7-mcrypt \ php7-mbstring \ php7-xml \ php7-openssl \ php7-json \ php7-phar \ php7-zip \ php7-dom \ php7-session \ php7-tokenizer \ php7-zlib && \ php7 -r "copy('http://getcomposer.org/installer', 'composer-setup.php');" && \ php7 composer-setup.php --install-dir=/usr/bin --filename=composer && \ php7 -r "unlink('composer-setup.php');" && \ ln -s /etc/php7/php.ini /etc/php7/conf.d/php.ini  RUN apk add --update \ bash \ openssh-client \ supervisor  RUN mkdir -p /etc/nginx && \ mkdir -p /etc/nginx/sites-available && \ mkdir -p /etc/nginx/sites-enabled && \ mkdir -p /run/nginx && \ ln -s /etc/nginx/sites-available/default.conf /etc/nginx/sites-enabled/default.conf && \ mkdir -p /var/log/supervisor && \ rm -Rf /var/www/* && \ chmod 755 /start.sh  RUN sed -i -e "s/;cgi.fix_pathinfo=1/cgi.fix_pathinfo=0/g" \ -e "s/variables_order = \"GPCS\"/variables_order = \"EGPCS\"/g" \ /etc/php7/php.ini && \ sed -i -e "s/;daemonize\s*=\s*yes/daemonize = no/g" \ -e "s/;catch_workers_output\s*=\s*yes/catch_workers_output = yes/g" \ -e "s/user = nobody/user = nginx/g" \ -e "s/group = nobody/group = nginx/g" \ -e "s/;listen.mode = 0660/listen.mode = 0666/g" \ -e "s/;listen.owner = nobody/listen.owner = nginx/g" \ -e "s/;listen.group = nobody/listen.group = nginx/g" \ -e "s/listen = 127.0.0.1:9000/listen = \/var\/run\/php-fpm.sock/g" \ -e "s/^;clear_env = no$/clear_env = no/" \ /etc/php7/php-fpm.d/www.conf  EXPOSE 443 80 WORKDIR /var/www  CMD ["/start.sh"] 

start.sh

#!/bin/bash  # ---------------------------------------------------------------------- # Create the .env file if it does not exist. # ----------------------------------------------------------------------  if [[ ! -f "/var/www/.env" ]] && [[ -f "/var/www/.env.example" ]]; then cp /var/www/.env.example /var/www/.env fi  # ---------------------------------------------------------------------- # Run Composer # ----------------------------------------------------------------------  if [[ ! -d "/var/www/vendor" ]]; then cd /var/www composer update composer dump-autoload -o fi  # ---------------------------------------------------------------------- # Start supervisord # ----------------------------------------------------------------------  exec /usr/bin/supervisord -n -c /etc/supervisord.conf 

site.conf

server { listen 80;  root /var/www/public; index index.php index.html;  location / { try_files $uri $uri/ /index.php?$query_string; }  location ~ /\. { deny all; }  location ~ \.php$ { try_files $uri = 404; fastcgi_split_path_info ^(.+\.php)(/.+)$; fastcgi_pass unix:/var/run/php-fpm.sock; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; } 

nginx.conf

user nginx; worker_processes 1;  error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid;  events { worker_connections 1024; }  http { include /etc/nginx/mime.types; default_type application/octet-stream; access_log off; sendfile on; #tcp_nopush on; keepalive_timeout 65; #gzip on; include /etc/nginx/sites-enabled/*.conf; } 

supervisord.conf

[unix_http_server] file=/dev/shm/supervisor.sock  [supervisord] logfile=/tmp/supervisord.log logfile_maxbytes=50MB logfile_backups=10 loglevel=warn pidfile=/tmp/supervisord.pid nodaemon=false minfds=1024 minprocs=200 user=root  [rpcinterface:supervisor] supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface  [supervisorctl] serverurl=unix:///dev/shm/supervisor.sock  [program:php-fpm7] command = /usr/sbin/php-fpm7 --nodaemonize --fpm-config /etc/php7/php-fpm.d/www.conf autostart=true autorestart=true priority=5 stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr stderr_logfile_maxbytes=0  [program:nginx] command=/usr/sbin/nginx -g "daemon off;" autostart=true autorestart=true priority=10 stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr stderr_logfile_maxbytes=0 

1 Answers

Answers 1

site.conf

server {   listen 80 default_server;   root /var/www/public;    index index.php;   server_name localhost;    location / {     try_files $uri /index.php?$query_string;   }    location ~* \.php$ {     fastcgi_split_path_info ^(.+\.php)(.*)$;     fastcgi_pass 127.0.0.1:9000;     fastcgi_params SCRIPT_FILENAME $document_root$fastcgi_script_name;     include fastcgi_params;   } } 

Read More

Docker + NGINX + SSL Termination

Im trying to set up NGINX within a Docker container so that it will perform SSL termination for traffic going to another container (tcp443 -> tcp3001).

However Im getting a 502 Bad Gateway from NGINX with the following error in the NGINX logs:

connect() failed (111: Connection refused) while connecting to upstream 

Containers

The following containers are running:

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                          NAMES 3b640f25af44        nginx               "nginx -g 'daemon ..."   3 seconds ago       Up 2 seconds        80/tcp, 0.0.0.0:443->443/tcp   hopeful_swartz f7b13bf2bdcd        ghost               "docker-entrypoint..."   21 hours ago        Up 21 hours         127.0.0.1:3001->2368/tcp       zen_carson 

Port 3001 Test

I can reach the backend server (container) on port 3001.

root@linode-server:~# curl -IL http://127.0.0.1:3001 HTTP/1.1 302 Found X-Powered-By: Express Location: /private/ Vary: Accept, Accept-Encoding Content-Type: text/plain; charset=utf-8 Content-Length: 31 Date: Sat, 07 Apr 2018 19:25:02 GMT Connection: keep-alive  HTTP/1.1 200 OK X-Powered-By: Express Cache-Control: no-cache, private, no-store, must-revalidate, max-stale=0, post-check=0, pre-check=0 Content-Type: text/html; charset=utf-8 Content-Length: 2655 ETag: W/"a5f-wAxdmCnbgI8/PCwspg8GKWyhtRw" Vary: Accept-Encoding Date: Sat, 07 Apr 2018 19:25:02 GMT Connection: keep-alive 

NGINX Config

worker_processes 5;  events { worker_connections 1024; }  http {   server {       listen              443 ssl;       ssl_certificate     /etc/nginx/packetflow.crt;       ssl_certificate_key /etc/nginx/packetflow.key;        location / {           proxy_pass http://127.0.0.1:3001;           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;           proxy_set_header X-Real-IP  $remote_addr;           proxy_set_header Host linode.packetflow.co.uk;       }   } } 

4 Answers

Answers 1

You have basic issue of reachability. When you have below in your Nginx Config

proxy_pass http://127.0.0.1:3001; 

Your are saying that within the same nginx container another service is running at port 3001. But the service is running in another container.

Next looking at your docker ps output

f7b13bf2bdcd        ghost               "docker-entrypoint..."   21 hours ago        Up 21 hours         127.0.0.1:3001->2368/tcp       zen_carson 

The port inside the container is 2368 and not 3001. Now comes the part of launching the container so you know its address

If you are you launching your docker container through command line then you will launch the container like below

docker run -d --name ghost ghost 

Then in your nginx config you will use

proxy_pass http://ghost:2368; 

The better way is to actually through docker-compose. So you will create docker-compose.yml file

version: 3 services:   ghost     build: ghost     image: ghost   web:     build: web     image: web     ports:       - 443:443 

You should look at below link

https://docs.docker.com/compose/overview/

Answers 2

If nginx configuration is in docker container , how you are able to proxy to other docker using 127.0.0.1:3001 (which is host machine port). Actually your bind to 127.0.0.1:3001 is to host machine.

Are you trying to curl from host machine and nginx config is in docker.

For a quick fix please use 172.17.0.1:3001 instead of 127.0.0.1:3001 . It will work if you didn't change anything in docker network adapter.

Answers 3

The simplest solution can be to move the nginx to the base machine (install it and run in a daemon). The loadbalancing between images happens on the host and only the application runs in docker.

Other solution can be creating hostname for the node image to reach it from the other images. This way you have to install docker-compose or create a script that runs the docker images with a specific name.

Answers 4

this line proxy_pass http://127.0.0.1:3001 in your NGINX Config is wrong, zen_carson doesn't works on your Nginx container localhost!they are works on different hosts!

if you want to access to zen_carson container from hopeful_swartz, first and simplest way is to use linking between containers, and use link alias instead of localhost IP!

Read More

URI::InvalidComponentError (bad component(expected scheme component): : https):

I'm working on a Ruby on Rails web app and I'm using Devise for user/password and OmniAuth for authentication using social media accounts. And I'm also using Nginx.

Authentication with username/password worked perfectly. But when I added ssl certificate to Nginx. I'm now able to login. But when I logout I get the error message in the title URI::InvalidComponentError (bad component(expected scheme component): : https): and telling that I have an exception generated from:

def check_scheme(v)   if v && parser.regexp[:SCHEME] !~ v     raise InvalidComponentError,       "bad component(expected scheme component): #{v}"   end 

How can I fix this problem in order for all types of authentications to work?

UPDATE

It throws the same exception after almost every redirect withing the web app. But redirects anyway, event user login. But it never logs out

*UPDATE 2 *

This question has more detailed description of my issue

This other question is another try to fix the issue

UPDATE 3

When I tried adding config.force_ssl = true to my environment .rb file, I get "page not found" and the following in ssl.error.log

2018/04/13 05:05:14 [error] 7317#0: *553 connect() failed (111: Connection refused) while connecting to upstream, client: <my laptop ip>, server: vps37181, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "<domain name>" 2018/04/13 05:05:14 [error] 7317#0: *553 open() "/var/www/<app-name>/public/50x.html" failed (2: No such file or directory), client: <my laptop ip>, server: vps37181, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "<domain name>" 

UPDATE 4

My current nginx configuration at /etc/nginx/conf.d/ssl.conf is:

server {     listen 443 ssl;     server_name <server-name>;      client_max_body_size 15M;     ssl                  on;     ssl_certificate      <.pem file path>;     ssl_certificate_key  <.key file path>;     ...     root <"public" directory inside my app directory>;     access_log <ssl.access.log path>;     error_log  <ssl.error.log path>;      include /etc/nginx/default.d/*.conf;      location / {         proxy_pass http://127.0.0.1:3000;          proxy_set_header X-Real-IP  $remote_addr;         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;         proxy_set_header Host $http_host;         proxy_set_header X-NginX-Proxy true;         proxy_set_header X-Forwarded-Proto: $scheme;  } 

UPDATE 5

After every request redirected from Nginx to the app, I get the following in the app console:

Cannot render console from <my laptop ip>! Allowed networks: 127.0.0.1, ::1, 127.0.0.0/127.255.255.255  URI::InvalidComponentError (bad component(expected scheme component): : https):  /home/tamer/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/uri/generic.rb:322:in `check_scheme' /home/tamer/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/uri/generic.rb:363:in `scheme=' actionpack (5.1.5) lib/action_dispatch/routing/redirection.rb:35:in `serve' actionpack (5.1.5) lib/action_dispatch/routing/redirection.rb:21:in `call' actionpack (5.1.5) lib/action_dispatch/routing/mapper.rb:17:in `block in <class:Constraints>' actionpack (5.1.5) lib/action_dispatch/routing/mapper.rb:46:in `serve' actionpack (5.1.5) lib/action_dispatch/journey/router.rb:50:in `block in serve' actionpack (5.1.5) lib/action_dispatch/journey/router.rb:33:in `each' actionpack (5.1.5) lib/action_dispatch/journey/router.rb:33:in `serve' actionpack (5.1.5) lib/action_dispatch/routing/route_set.rb:844:in `call' omniauth (1.8.1) lib/omniauth/strategy.rb:190:in `call!' omniauth (1.8.1) lib/omniauth/strategy.rb:168:in `call' omniauth (1.8.1) lib/omniauth/strategy.rb:190:in `call!' omniauth (1.8.1) lib/omniauth/strategy.rb:168:in `call' omniauth (1.8.1) lib/omniauth/strategy.rb:190:in `call!' omniauth (1.8.1) lib/omniauth/strategy.rb:168:in `call' omniauth (1.8.1) lib/omniauth/strategy.rb:190:in `call!' omniauth (1.8.1) lib/omniauth/strategy.rb:168:in `call' omniauth (1.8.1) lib/omniauth/strategy.rb:190:in `call!' omniauth (1.8.1) lib/omniauth/strategy.rb:168:in `call' omniauth (1.8.1) lib/omniauth/builder.rb:63:in `call' warden (1.2.7) lib/warden/manager.rb:36:in `block in call' warden (1.2.7) lib/warden/manager.rb:35:in `catch' warden (1.2.7) lib/warden/manager.rb:35:in `call' rack (2.0.4) lib/rack/etag.rb:25:in `call' rack (2.0.4) lib/rack/conditional_get.rb:25:in `call' rack (2.0.4) lib/rack/head.rb:12:in `call' rack (2.0.4) lib/rack/session/abstract/id.rb:232:in `context' rack (2.0.4) lib/rack/session/abstract/id.rb:226:in `call' actionpack (5.1.5) lib/action_dispatch/middleware/cookies.rb:613:in `call' activerecord (5.1.5) lib/active_record/migration.rb:556:in `call' actionpack (5.1.5) lib/action_dispatch/middleware/callbacks.rb:26:in `block in call' activesupport (5.1.5) lib/active_support/callbacks.rb:97:in `run_callbacks' actionpack (5.1.5) lib/action_dispatch/middleware/callbacks.rb:24:in `call' actionpack (5.1.5) lib/action_dispatch/middleware/executor.rb:12:in `call' actionpack (5.1.5) lib/action_dispatch/middleware/debug_exceptions.rb:59:in `call' web-console (3.5.1) lib/web_console/middleware.rb:135:in `call_app' web-console (3.5.1) lib/web_console/middleware.rb:20:in `block in call' web-console (3.5.1) lib/web_console/middleware.rb:18:in `catch' web-console (3.5.1) lib/web_console/middleware.rb:18:in `call' actionpack (5.1.5) lib/action_dispatch/middleware/show_exceptions.rb:31:in `call' railties (5.1.5) lib/rails/rack/logger.rb:36:in `call_app' railties (5.1.5) lib/rails/rack/logger.rb:24:in `block in call' activesupport (5.1.5) lib/active_support/tagged_logging.rb:69:in `block in tagged' activesupport (5.1.5) lib/active_support/tagged_logging.rb:26:in `tagged' activesupport (5.1.5) lib/active_support/tagged_logging.rb:69:in `tagged' railties (5.1.5) lib/rails/rack/logger.rb:24:in `call' sprockets-rails (3.2.1) lib/sprockets/rails/quiet_assets.rb:13:in `call' actionpack (5.1.5) lib/action_dispatch/middleware/remote_ip.rb:79:in `call' request_store (1.4.0) lib/request_store/middleware.rb:19:in `call' actionpack (5.1.5) lib/action_dispatch/middleware/request_id.rb:25:in `call' rack (2.0.4) lib/rack/method_override.rb:22:in `call' rack (2.0.4) lib/rack/runtime.rb:22:in `call' activesupport (5.1.5) lib/active_support/cache/strategy/local_cache_middleware.rb:27:in `call' actionpack (5.1.5) lib/action_dispatch/middleware/executor.rb:12:in `call' actionpack (5.1.5) lib/action_dispatch/middleware/static.rb:125:in `call' rack (2.0.4) lib/rack/sendfile.rb:111:in `call' railties (5.1.5) lib/rails/engine.rb:522:in `call' puma (3.11.2) lib/puma/configuration.rb:225:in `call' puma (3.11.2) lib/puma/server.rb:624:in `handle_request' puma (3.11.2) lib/puma/server.rb:438:in `process_client' puma (3.11.2) lib/puma/server.rb:302:in `block in run' puma (3.11.2) lib/puma/thread_pool.rb:120:in `block in spawn_thread' 

This message doesn't prevent the website from loading. But when user logs out (I'm using Devise for username/password user authentication). the website throws in the browser the error I mentioned earlier:

URI::InvalidComponentError bad component(expected scheme component): : https Extracted source (around line #322):  def check_scheme(v)   if v && parser.regexp[:SCHEME] !~ v     raise InvalidComponentError, # line 322       "bad component(expected scheme component): #{v}"   end 

and I get the console error message twice.

UPDATE 7

I'm running my server in development.

I dug deeper and I found that :

URI::InvalidComponentError (bad component(expected scheme component): : https) 

means that what was expected was : https while received https.

I tried modifying parser.regexp[:SCHEME] in check_shceme in /home/tamer/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/uri/generic.rb in which it approves : https also. But now, on logout, which supposed to send a request to

https://<my domain name>/users/log_out 

now redirectes to :

https://<my domain name>/users/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/:%20https://<my domain name>/ 

in which :%20 means whitespace.

0 Answers

Read More

How to get SSL working with Rails, AWS Elastic Beanstalk and Cloudflare

I have a site hosted on Elastic Beanstalk built with Ruby on Rails. I set up Cloudflare to configure the DNS and provide a CDN. Cloudflare also provides an SSL.

I can't get the SSL working with my app.

With Cloudflare's SSL set at "Flexible" I can load my main page but when I try to log in, I get these errors (edited for brevity):

INFO -- :  Started POST "/users/sign_in" for xxx.xxx.146.132 at 2018-03-19 16:45:24 +0000 INFO -- :  Processing by Users::SessionsController#create as HTML INFO -- :    Parameters: {"utf8"=>"✓", "authenticity_token"=>"f92CTIe5qlp7C624DZzZM2oWdFMcq6PhyfOJI16saV32yugMmJlenL/F3gTeBBsAjaAw92P1vncWBzI+JnK8wA==", "user"=>{"email"=>"test@test.com", "password"=>"[FILTERED]"}, "commit"=>"Log in"} WARN -- :  HTTP Origin header (https://[MY_URL].com) didn't match request.base_url (http://[MY_URL].com) INFO -- :  Completed 401 Unauthorized in 1ms (ActiveRecord: 0.0ms)   FATAL -- :  ActionController::InvalidAuthenticityToken (ActionController::InvalidAuthenticityToken): 

If I set Cloudflare's SSL to "Full" I get a 502 error with a Cloudflare-generated page (see image).

I came across this site (http://til.obiefernandez.com/posts/875a2a69af-cloudflare-flexible-ssl-mode-breaks-rails-5-csrf) which seems to have the exact same issue but setting to "full" didn't help me.

I've tried setting config.force_ssl = true in /config/environments/production.rb. That setting would not allow any access to the site. Just shows the same 502 error page from Cloudflare and nothing in my production or nginx logs.

I've tried messing around with custom nginx config's but haven't gotten anywhere. Here is my latest nginx confix attempt:

user nginx; worker_processes auto; error_log /var/log/nginx/error.log; pid /var/run/nginx.pid;  events {     worker_connections 1024; }  http {     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '                       '$status $body_bytes_sent "$http_referer" '                       '"$http_user_agent" "$http_x_forwarded_for"';      access_log  /var/log/nginx/access.log  main;      sendfile            on;     tcp_nopush          on;     tcp_nodelay         on;     keepalive_timeout   65;     types_hash_max_size 2048;      include             /etc/nginx/mime.types;     default_type        application/octet-stream;      include /etc/nginx/conf.d/*.conf;      index   index.html index.htm;      server {         listen       80 ;         listen       [::]:80 ;         server_name  localhost;         root         /usr/share/nginx/html;          # Load configuration files for the default server block.         include /etc/nginx/default.d/*.conf;           location / {           proxy_pass        http://localhost;           proxy_set_header  Host $host;           proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;           proxy_set_header  X-Forwarded-Proto $scheme;           proxy_set_header  X-Forwarded-Ssl on; # Optional           proxy_set_header  X-Forwarded-Port $server_port;           proxy_set_header  X-Forwarded-Host $host;         }         error_page 404 /404.html;             location = /40x.html {         }          # redirect server error pages to the static page /50x.html         #         error_page 500 502 503 504 /50x.html;             location = /50x.html {         }     } } 

Can anyone help? I'm sure I'm missing something obvious here.

1 Answers

Answers 1

502 Error: This error says that the Cloudfare server is unable to read the response your rails server sends, basically when you select the Full SSL the Cloudflare expects the response to be in SSL but here the rails app send a NON-SSL (HTTP) response which Cloudflare is unable to read.

I read the article you shared, looks like rails do not allow flexible SSL for security reasons.

Flexible SSL In flexible SSL you don't need to secure your rails app with SSL Certificate(HTTPS) but your visitors still see the site as being HTTPS enabled.

Flexible SSL: secure connection between your visitor and Cloudflare, but no secure connection between Cloudflare and your web server. You don't need to have an SSL certificate on your web server, but your visitors still see the site as being HTTPS enabled.

Full SSL
As mentioned in the article if you enable Full SSL, then you need to configure rails config.force_ssl = true to use a Self-signed SSL certificate or you can get a certificate from letsencrypt for free, once you did that you need change your Nginx to run on HTTPS port 443. This should fix the 502 error.

Here is a tutorial on Using HTTPs with Ruby on Rails

Full SSL: secure connection between your visitor and Cloudflare, and secure connection (but not authenticated) between Cloudflare and your web server. You will need to have your server configured to answer HTTPS connections, with a self-signed certificate at least.

Image Source

Read More

Error on Let's encrypt auto renewal (Nginx)

I am trying to set up greenlock-express to run behind nginx proxy.

Here is my nginx config

... # redirect server {     listen 80;     listen [::]:80;     server_name mydomain.com;      location / {         return 301 https://$server_name$request_uri;     } }  # serve server {     listen 443 ssl http2;     listen [::]:443 ssl http2;     server_name mydomain.com;      # SSL settings     ssl on;     ssl_certificate C:/path/to/mydomain.com/fullchain.pem;     ssl_certificate_key C:/path/to/mydomain.com/privkey.pem;      # enable session resumption to improve https performance     ssl_session_cache shared:SSL:50m;     ssl_session_timeout 1d;     ssl_session_tickets off;      # enables server-side protection from BEAST attacks     ssl_prefer_server_ciphers on;     # disable SSLv3(enabled by default since nginx 0.8.19) since it's less secure then TLS     ssl_protocols TLSv1 TLSv1.1 TLSv1.2;      # ciphers chosen for forward secrecy and compatibility     ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS';      # enable OCSP stapling (mechanism by which a site can convey certificate revocation information to visitors in a privacy-preserving, scalable manner)     resolver 8.8.8.8 8.8.4.4;     ssl_stapling on;     ssl_stapling_verify on;     ssl_trusted_certificate C:/path/to/mydomain.com/chain.pem;      # config to enable HSTS(HTTP Strict Transport Security) https://developer.mozilla.org/en-US/docs/Security/HTTP_Strict_Transport_Security     add_header Strict-Transport-Security "max-age=31536000; includeSubdomains; preload";      # added to make handshake take less resources     keepalive_timeout 70;      location / {         proxy_set_header X-Real-IP $remote_addr;         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;         proxy_set_header Host $http_host;         proxy_set_header X-NginX-Proxy true;         proxy_pass https://127.0.0.1:3001/;         proxy_redirect off;         proxy_set_header Upgrade $http_upgrade;         proxy_set_header Connection "upgrade";     } } ... 

I have node server running on port 3000 (http) and port 3001 (https). Everything else seems to be working, but certificates do not update and expire after 3 months.

If I closed nginx and ran node server on port 80 (http) and port 443 (https), then it updates certs.

I made sure that .well-known/acme-challenge is forwarded to node server, i.e when I go to url http(s)://mydomain.com/.well-known/acme-challenge/randomstr I get following response:

{    "error": {      "message": "Error: These aren't the tokens you're looking for. Move along."    }  } 

2 Answers

Answers 1

The easy way to separate the webroot for ACME authentication.

Create a webroot directory for ACME authentication.

C:\www\letsencrypt\.well-known 

In the nginx configuration, set the webroot for ACME authentication to the previously created directory.

http://example.com/.well-known/acme-challenge/token -> C:/www/letsencrypt/.well-known/acme-challenge/token

server {     listen 80;     listen [::]:80;     server_name mydomain.com;      location ^~ /.well-known/acme-challenge/ {         default_type "text/plain";         root C:/www/letsencrypt;     }      location / {         return 301 https://$server_name$request_uri;     } } 

Restart nginx.

You can change your webroot in certbot to get authentication again.

certbot certonly --webroot -w C:\www\letsencrypt\ -d exapmle.com --dry-run 

First, test it by adding the --dry-run option. Otherwise, you may experience issues limiting the number of authentication attempts.

Answers 2

The error you are seeing is that when a token is placed in your

webroot/.well-known/acme-challenge/token

Then Let’s Encrypt tries to verify that from the internet. going to http://yourdomain/.well-known/acme-challenge/token it gets a 404 error - page not found. Exactly why it get’s a 404 I can’t be certain. If you place a file there yourself, is it reachable from the internet ?.

If you are wondering there are a couple of automatic ways to renew your SSL's without restarting your nginx. The one most nginx users seem to prefer is the webroot plugin: first, obtain a new cert using something like:

certbot certonly --webroot -w /path/to/your/webroot -d example.com --post-hook="service nginx reload" 

Then set up a cron job to run certbot renew once or twice a day; it will only run the post-hook when it actually renews the certificate. You can also use --pre-hook flag if you prefer to stop nginx to run certbot in standalone mode.

There’s also a full nginx plugin, which you can activate with --nginx. It’s still being tested, so experiment at your own risk and report any bugs.

Note: post-hookFlag will take care of reloading nginx upload renewal of your certs

Read More

Nginx: readv() failed (104: Connection reset by peer) while reading upstream

I have read every other thread about this error with no luck.

I'm running Nginx 1.12.2 with PHP-FPM 5.6 on RHEL 6

I am getting these errors on any/all pages of my site. It is not related to slow requests. I have witnessed it personally and I immediately get a 500 error page without any waiting.

It affects both POST and GET requests.

I am not running any opcode caches (that I am aware of) I checked all config files, phpinfo() as well as php -v and did not see any mention of opcache or opcode.)

I have turned on catch_workers_output in PHP-FPM. I am not receiving any PHP errors

I have plenty of memory available and there are no errors in the system logs about memory, file descriptors or sockets.

Tried pm_max_requests with both high and low numbers

Not running xdebug plugin

Seems to happen at random times, even in the middle of the night when we aren't very busy at all.

I'm desperate for a solution to this issue which has gone on for months now.

Here's some of the information asked for in the comments below:

Nginx fastcgi.conf (differences from the stock file):

fastcgi_intercept_errors on; fastcgi_param  REDIRECT_STATUS    200; fastcgi_buffer_size   32k; fastcgi_buffers 8 16k; fastcgi_max_temp_file_size 0; fastcgi_keep_conn on; 

Nginx conf specifics:

sendfile        on; tcp_nopush      on; tcp_nodelay     on; server_tokens   off; gzip            off; gzip_static     on; gzip_comp_level 5; gzip_min_length 1024; types_hash_max_size 2048; client_body_timeout   900; client_header_timeout 900; keepalive_timeout     900; send_timeout          900; proxy_connect_timeout  900s; proxy_send_timeout  900s; proxy_read_timeout  900s; fastcgi_send_timeout 900s; fastcgi_read_timeout 900s; 

Nginx location specific config:

client_max_body_size 40M;  location ~ \.php$ {        fastcgi_pass   127.0.0.1:9000;        include        fastcgi.conf; } 

php-fpm config details:

listen = 127.0.0.1:9000 listen.allowed_clients = 127.0.0.1 pm = dynamic pm.max_children = 1000 pm.start_servers = 300 pm.min_spare_servers = 200 pm.max_spare_servers = 350 pm.max_requests = 1000 ; I've tried higher/lower with no difference slowlog = /var/log/php-fpm/www-slow.log catch_workers_output = yes php_admin_value[error_log] = /var/log/php_error_log php_admin_flag[log_errors] = on php_value[session.save_handler] = files php_value[session.save_path] = /var/lib/php/session 

Installed Versions:

php.x86_64                                                               5.6.33-1.el6.remi                                      @remi-php56                        php-bcmath.x86_64                                                        5.6.33-1.el6.remi                                      @remi-php56                        php-cli.x86_64                                                           5.6.33-1.el6.remi                                      @remi-php56                        php-common.x86_64                                                        5.6.33-1.el6.remi                                      @remi-php56                        php-devel.x86_64                                                         5.6.33-1.el6.remi                                      @remi-php56                        php-fedora-autoloader.noarch                                             1.0.0-1.el6.remi                                       @remi                              php-fpm.x86_64                                                           5.6.33-1.el6.remi                                      @remi-php56                        php-gd.x86_64                                                            5.6.33-1.el6.remi                                      @remi-php56                        php-gmp.x86_64                                                           5.6.33-1.el6.remi                                      @remi-php56                        php-imap.x86_64                                                          5.6.33-1.el6.remi                                      @remi-php56                        php-mbstring.x86_64                                                      5.6.33-1.el6.remi                                      @remi-php56                        php-mcrypt.x86_64                                                        5.6.33-1.el6.remi                                      @remi-php56                        php-mysqlnd.x86_64                                                       5.6.33-1.el6.remi                                      @remi-php56                        php-pdo.x86_64                                                           5.6.33-1.el6.remi                                      @remi-php56                        php-pear.noarch                                                          1:1.10.5-4.el6.remi                                    @remi                              php-pecl-igbinary.x86_64                                                 2.0.5-1.el6.remi.5.6                                   @remi-php56                        php-pecl-imagick.x86_64                                                  3.4.3-5.el6.remi.5.6                                   @remi-php56                        php-pecl-jsonc.x86_64                                                    1.3.10-2.el6.remi.5.6                                  @remi-php56                        php-pecl-jsonc-devel.x86_64                                              1.3.10-2.el6.remi.5.6                                  @remi-php56                        php-pecl-memcache.x86_64                                                 3.0.8-3.el6.remi.5.6                                   @remi-php56                        php-pecl-memcached.x86_64                                                2.2.0-3.el6.remi.5.6                                   @remi-php56                        php-pecl-msgpack.x86_64                                                  0.5.7-1.el6.remi.5.6                                   @remi-php56                        php-pecl-ssh2.x86_64                                                     0.13-3.el6.remi.5.6                                    @remi-php56                        php-pecl-zip.x86_64                                                      1.15.2-1.el6.remi.5.6                                  @remi-php56                        php-process.x86_64                                                       5.6.33-1.el6.remi                                      @remi-php56                        php-xml.x86_64                                                           5.6.33-1.el6.remi                                      @remi-php56    nginx version: nginx/1.12.2  Kernel: 2.6.32-696.20.1.el6 

0 Answers

Read More

HTTP status code names are missing when using Nginx

I'm using Nginx to

redirect all HTTP requests to HTTPS

in my spring boot application.This is the nginx configuration that i'm using,with that i was able to redirect all requests to Https but when i do it i get the status code returned correctly but it doesnt have the status code name anymore.if i remove nginx and run spring boot application alone i can get the http status with its code name and code.

server {    listen 80 default_server;   listen [::]:80 default_server;   server_name _ ;    access_log /var/log/nginx/access.log;   error_log /var/log/nginx/error.log;    proxy_set_header X-Forwarded-Proto $scheme;   proxy_set_header X-Real-IP $remote_addr;   proxy_set_header Host $host;   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;     if ( $http_x_forwarded_proto != 'https' ) {     return 307 https://$host$request_uri;   }    location / {     proxy_set_header X-Forwarded-Proto http;     proxy_pass http://localhost:7070;       expires -1;   }  } 

what am i doing wrong in here should i use proxy_redirect instead of proxy_pass, or am i missing anything in here.that'd be great if you can help.

1 Answers

Answers 1

In the same config file

listen on 80 to redirect req to https (443)

server {     listen 80;     listen [::]:80;     server_name your_url.com www.your_url.com;      return 301 https://your_url.com$request_uri; } 

listen on 433

server {     listen 443 ssl default_server;     listen [::]:443 ssl default_server;      location / {             # proxy pass to your app              proxy_pass http://localhost:7070;             proxy_http_version 1.1;             proxy_set_header Upgrade $http_upgrade;             proxy_set_header Connection 'upgrade';             proxy_set_header Host $host;             proxy_cache_bypass $http_upgrade;     } 

This is the way I do it, and works perfectly for me, cheers!

Read More

How to share a file (data) across multiple docker containers in azure

I want to run several docker containers in different regions (asia, eu, us) which host a nginx server.

However, they should all have the same configuration because I need to updated hostnames at runtime dynamically (one domain for every new tenant). So I guess it would be the easiest way to just share one config file among all containers and reload them...

So how can I share data/files among n containers on azure?

3 Answers

Answers 1

In general, unless you want to use proprietary solutions specific to the platform at hand, the best way to synchronise files between multiple systems is with the help of rsync.

For example, in DNS, there exists a specialised protocol for transferring domain zones directly within the DNS software, called AXFR; one of the authors of a newer DNS implementation suggests that this AXFR protocol is crap, and rsync over ssh works much better — http://cr.yp.to/djbdns/tcp.html — and the ssh part is a nice thing about rsync, in that it can work over plain old ssh protocol as far as interconnection between the hosts goes, not requiring any special firewall considerations.

Answers 2

Have you considered using the Azure file share.

Answers 3

Docker and Azure are out of scope here. You should consider they are all servers. You can run and scheduled bash script in each server (docker image) that retrieves the config file everyday using ftp, ssh, or even git (I would choose this options so you can keep older versions if something goes wrong).

nginx1 | nginx2 | <-----git clone yourRepository-----> |nginx.conf| nginx3 |

Of course you have to consider network/security restrictions.

Read More

“Heavy” simultaneous users Nginx - Laravel - Google compute engine

I'm running a server in nginx with Laravel (medium static web) and I'm doing for example 500 constant load simultaneous users during 1 minute (not distributed users during that minute).

And getting this error:

unix:/var/run/php/php7.1-fpm.sock failed - Resource temporarily unavailable

cginx.conf

worker_processes auto;  events {     use epoll;     worker_connections 1524; #in my case it should be 1024, but well..     multi_accept on; } http {     #with this I reduce disk usage a lot     client_body_buffer_size 10K;     client_header_buffer_size 1k;     large_client_header_buffers 2 1k;     reset_timedout_connection on;      sendfile on;     tcp_nopush on;     tcp_nodelay on; 

www.conf

pm.max_children = 500 pm.start_servers = 20 pm.min_spare_servers = 20 pm.max_spare_servers = 64 

Results with Google compute engine:

f1-micro (1 vCPU, 0,6 GB) - Is supporting 40 - 60 requests per second g1-small (1 vCPU, 1,7 GB) - Is maintaining 80 request per second n1-standard (1vCPU, 3,75 GB) - - Is maintaining 130 request per second n1-standard-2 (2vCPU, 7,5 GB) - Is maintaining 250 request per second . . n1-standard-16 (16 vCPU, 60 GB) - Is maintaining 840 request per second 

The last one is the first passing the test, the rest are dropping Bad Gateways errors from 200 users to 400

If I test for example not 2.000 users distributed in 30 secs with the micro instance then is fine, but not simultaneous sending requests.

Starting with 2 cores, CPUs level show perfectly fine, same as disk operations etc..

So after a loooot of tests I have some questions:

1) Is this normal? Not for me, is not normal to need 16 cores to run a simple web.. or the stress test is too heavy and it's normal?

2) Then, am I missing something? Is Google limiting request per second somehow?

3) What would be normal parameters for the given config files?

Any other help is more than welcome

2 Answers

Answers 1

TBH, it is not entirely clear what you're trying to achieve with this test, especially with bringing GCE into the equation.

If your "medium static web" site is doing a dozen SQL queries for each page, possibly with a few JOINs each, as well as various other resource intensive operations, then it is hardly a surprise that you're very far away from achieving C10K.

Your test results throughout various GCE instances look reasonably consistent, proving that it's your code that's to blame. If you want to rule out GCE as the cause of your performance issues, then it seems to be that the next logical step would be to test the performance outside of it.

It seems that you're most concerned with receiving the Bad Gateway errors on the cheaper instances, so, let's figure out why that happens.

• Your backend is only capable of processing a certain number of requests in a given amount of time, on the order of a few dozens per second on the cheapest plan.

• It is configured without a clear spec of what's supposed to happen once resources are exhausted. With the configuration at hand, you can only push 40 requests per second on the cheapest instance, yet, the configuration is set to have Laravel process 500 requests simultaneously, on 1 vCPU w/ 0.6GB of total RAM, leaving each request about 1MB of RAM, which is way on the lower scale for a "medium static web" powered by a dynamic framework, resulting in an impedance mismatch.

• It is then hardly a surprise that you're getting errors, which are clearly due to the impedance mismatch as the backpressure builds up, and the backend likely runs out of RAM trying to process the never-ending requests.

So, what is the solution?

The solution is to have a clear understanding of how many resources are required to generate each page on the backend, and subsequently limit the number of simultaneous connections to the backend from the reverse proxy to never exceed such a certain number of connections, with http://nginx.org/r/limit_req and/or http://nginx.org/r/limit_conn, as appropriate. This way, you could catch and monitor the overload conditions, and provide an appropriate error message to the user, and/or script automatic dynamic resizing of your infrastructure.

In addition to the above, another good idea is to cache results of your backend, provided that it's actually "static" content that's generated, without per-user customisation, which could then let you account for the realistic situation of when a link to your site is posted at Slashdot/Reddit/Twitter, causing a huge spike of traffic to a single "static" page, which can then be cached for the duration of the whole event. Else, if the content is not actually "static", then it's up to you to decide which way to go and which compromise to take — I'd suggest seeing if the per-request customisations are actually warranted, and whether an uncustomised version might be appropriate, especially for the Slashdot-like scenarios.

Answers 2

On a machine with 2vcpu and 7gb ram i can handle more 1000 request/second You didn't mentioned the ram per request do you need, also i suggest change php socket to tcp connection, it allow me to process 10x requests

Read More

Global auth_request for nginx

I have a server where I run multiple subdomains sites with nginx

each subdomain have a server { ... } block in configuration (in sites-enabled )

I want to add a global dynamic authentication backend (another http site) that will check requested subdomain and give access or ask user credentials

so the flow should be like

request stie1.example.com ---> [auth check in database]                                  /              \                               no               yes                               /                  \             show username/pass form        pass to site1.example.com 

I found auth_request module - but examples only about server context

and I want it on top level context (http context) - where of course location directive is not available

So basically what I want is sort of global nginx middleware where I can run some code (i.e. python) to check if user have permission to view one of the hosted domains

any suggestions ?

1 Answers

Answers 1

It would perhaps help if you provide actual context for what it is exactly that you want done.

As per http://nginx.org/r/auth_request, the auth_request directive is actually allowed "two" levels up of the location context — both in the server as well as http contexts, so, there is absolutely nothing that should prevent you from enabling it globally for every single server in your nginx instance.

Otherwise, if you want it enabled only within certain location directives and/or only certain server configurations, then you can always use the include directive in any context, as per http://nginx.org/r/include.

Read More