502 Bad Gateway; nginx/1.16.1
-
Seems that when I updated my NC last night, the nginx configuration was screwed. Now I'm left with trying to correct it. Simple enough I'm sure, except I've not done much configuration work with nginx. I installed NC a few months ago and accepted all the defaults.
Host: Vultr
OS Ver: Ubuntu 19.10I'm sure @scottalanmiller or @JaredBusch can point out the error in seconds....
-
Did you install Nextcloud with Nginx instead of Apche?
-
@JaredBusch said in 502 Bad Gateway; nginx/1.16.1:
Did you install Nextcloud with Nginx instead of Apche?
I do not recall - as that was months ago. I used the Vultr scripted base install. I've kept it update/upgraded but took the time to upgrade the distro from 18.04 to 19.10
-
root@TuDallasNC:~# netstat -lpn Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 759/nginx: master p tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 766/redis-server 12 tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 759/nginx: master p tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 681/systemd-resolve tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 760/sshd tcp 0 0 0.0.0.0:61209 0.0.0.0:* LISTEN 733/python3 tcp6 0 0 :::443 :::* LISTEN 759/nginx: master p tcp6 0 0 ::1:6379 :::* LISTEN 766/redis-server 12 tcp6 0 0 :::80 :::* LISTEN 759/nginx: master p tcp6 0 0 :::22 :::* LISTEN 760/sshd udp 0 0 127.0.0.53:53 0.0.0.0:* 681/systemd-resolve udp 0 0 ServerIP:68 0.0.0.0:* 656/systemd-network raw6 0 0 :::58 :::* 7 656/systemd-network Active UNIX domain sockets (only servers) Proto RefCnt Flags Type State I-Node PID/Program name Path unix 2 [ ACC ] SEQPACKET LISTENING 15426 1/init /run/udev/control unix 2 [ ACC ] STREAM LISTENING 23829 918/systemd /run/user/0/systemd/private unix 2 [ ACC ] STREAM LISTENING 15150 1/init /run/systemd/private unix 2 [ ACC ] STREAM LISTENING 23835 918/systemd /run/user/0/gnupg/S.gpg-agent.ssh unix 2 [ ACC ] STREAM LISTENING 23836 918/systemd /run/user/0/snapd-session-agent.socket unix 2 [ ACC ] STREAM LISTENING 23837 918/systemd /run/user/0/gnupg/S.gpg-agent unix 2 [ ACC ] STREAM LISTENING 15156 1/init /run/systemd/journal/stdout unix 2 [ ACC ] STREAM LISTENING 23838 918/systemd /run/user/0/pk-debconf-socket unix 2 [ ACC ] STREAM LISTENING 23839 918/systemd /run/user/0/gnupg/S.gpg-agent.browser unix 2 [ ACC ] STREAM LISTENING 23840 918/systemd /run/user/0/gnupg/S.dirmngr unix 2 [ ACC ] STREAM LISTENING 23841 918/systemd /run/user/0/gnupg/S.gpg-agent.extra unix 2 [ ACC ] STREAM LISTENING 15161 1/init @/org/kernel/linux/storage/multipathd unix 2 [ ACC ] STREAM LISTENING 15246 1/init /run/systemd/fsck.progress unix 2 [ ACC ] STREAM LISTENING 15420 1/init /run/lvm/lvmpolld.socket unix 2 [ ACC ] STREAM LISTENING 18729 1/init @ISCSIADM_ABSTRACT_NAMESPACE unix 2 [ ACC ] STREAM LISTENING 18724 1/init /var/snap/lxd/common/lxd/unix.socket unix 2 [ ACC ] STREAM LISTENING 18708 1/init /run/uuidd/request unix 2 [ ACC ] STREAM LISTENING 18710 1/init /run/acpid.socket unix 2 [ ACC ] STREAM LISTENING 18712 1/init /run/snapd.socket unix 2 [ ACC ] STREAM LISTENING 18714 1/init /run/snapd-snap.socket unix 2 [ ACC ] STREAM LISTENING 18720 1/init /run/dbus/system_bus_socketEdit: been so long since i've posted I had forgotten how to code blocking.
-
Nothing in the /var/log/nginx/error.log so I checked /var/log/nginx/nextcloud_https_error.log
cat nextcloud_https_error.log 2019/12/31 00:09:16 [error] 760#760: *281 connect() failed (111: Connection refused) while connecting to upstream, client: MY-IP, server: _, request: "GET /csrftoken HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "ServerIP" -
Testing the nginx config:
root@TuDallasNC:/var/log/nginx# sudo nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful -
Get us a...
netstat -tulpn
There are a lot of unknowns here. My guess is that whatever is running NextCloud, isn't.
-
@scottalanmiller said in 502 Bad Gateway; nginx/1.16.1:
netstat -tulpn
root@TuDallasNC:/var/log/nginx# netstat -tulpn Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 759/nginx: master p tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 766/redis-server 12 tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 759/nginx: master p tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 681/systemd-resolve tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 760/sshd tcp 0 0 0.0.0.0:61209 0.0.0.0:* LISTEN 733/python3 tcp6 0 0 :::443 :::* LISTEN 759/nginx: master p tcp6 0 0 ::1:6379 :::* LISTEN 766/redis-server 12 tcp6 0 0 :::80 :::* LISTEN 759/nginx: master p tcp6 0 0 :::22 :::* LISTEN 760/sshd udp 0 0 127.0.0.53:53 0.0.0.0:* 681/systemd-resolve udp 0 0 ServerIP:68 0.0.0.0:* 656/systemd-network -
Checking to see if nginx is running:
root@TuDallasNC:/var/log/nginx# systemctl status nginx ● nginx.service - nginx - high performance web server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2019-12-31 01:58:03 UTC; 40min ago Docs: http://nginx.org/en/docs/ Process: 741 ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf (code=exited, status=0/SUCCESS) Main PID: 759 (nginx) Tasks: 2 (limit: 2287) Memory: 5.9M CGroup: /system.slice/nginx.service ├─759 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf └─761 nginx: worker process Dec 31 01:58:03 TuDallasNC systemd[1]: Starting nginx - high performance web server... Dec 31 01:58:03 TuDallasNC systemd[1]: nginx.service: Can't open PID file /run/nginx.pid (yet?) after start: No such file or directory Dec 31 01:58:03 TuDallasNC systemd[1]: Started nginx - high performance web server. -
@gjacobse said in 502 Bad Gateway; nginx/1.16.1:
Checking to see if nginx is running:
root@TuDallasNC:/var/log/nginx# systemctl status nginx ● nginx.service - nginx - high performance web server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2019-12-31 01:58:03 UTC; 40min ago Docs: http://nginx.org/en/docs/ Process: 741 ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf (code=exited, status=0/SUCCESS) Main PID: 759 (nginx) Tasks: 2 (limit: 2287) Memory: 5.9M CGroup: /system.slice/nginx.service ├─759 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf └─761 nginx: worker process Dec 31 01:58:03 TuDallasNC systemd[1]: Starting nginx - high performance web server... Dec 31 01:58:03 TuDallasNC systemd[1]: nginx.service: Can't open PID file /run/nginx.pid (yet?) after start: No such file or directory Dec 31 01:58:03 TuDallasNC systemd[1]: Started nginx - high performance web server.You can fix the
nginx.service: Can't open PID file /run/nginx.pid (yet?) after start: No such file or directoryby doing following:sudo mkdir /etc/systemd/system/nginx.service.d printf "[Service]\nExecStartPost=/bin/sleep 0.1\n" | \ sudo tee /etc/systemd/system/nginx.service.d/override.conf sudo systemctl daemon-reload sudo systemctl restart nginx sudo systemctl stop nginxhttps://bugs.launchpad.net/ubuntu/+source/nginx/+bug/1581864
-
I was reading that a few minutes ago, and search to see if the file was there,.. and it is.
However, Let me run through it.
-
Issue continues
-
Nextcloud runs on Apache by default, or at least used to. So I have no idea what script you followed to get Nginx.
-
@gjacobse said in 502 Bad Gateway; nginx/1.16.1:
@JaredBusch said in 502 Bad Gateway; nginx/1.16.1:
Did you install Nextcloud with Nginx instead of Apche?
I do not recall - as that was months ago. I used the Vultr scripted base install. I've kept it update/upgraded but took the time to upgrade the distro from 18.04 to 19.10
thats why i dont like the scripted installs, they are good and gets you going fast, but when times come to troubleshoot... cause you didnt do it, it will take you x100 time to solve it.
-
@JaredBusch said in 502 Bad Gateway; nginx/1.16.1:
Nextcloud runs on Apache by default, or at least used to. So I have no idea what script you followed to get Nginx.
Hmm... Is apache running perhaps?
I've had the reverse of that issue before where apache was started and brought a site down
-
It was the Vultr stock server build for Nextcloud.
In trying to fix things, I think I have made matters worse, so at this point, I've downloaded the few files that where there, and will re-install the server nuking the current data.
Of note; the data was found at:
/var/www/html/data/user17049/files/SHARENAME -
You have the answer in the logs you posted. Nginx can't connect to php handler daemon on port 9000, most likely php-fpm.
