System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7
-
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
-rw-r--r-- 1 root root 244,855,752 Dec 14 16:25 Session.db-wal-rw-r--r-- 1 root root 78,403,632 Dec 16 08:33 Session.db-walrw-r--r-- 1 root root 4,128,272 Dec 17 02:17 Session.db-walGoing back and doing some comparisons and noticed this -
(commas added for 'dramatic effect'
-rw-r--r-- 1 root root 25,082,592 Dec 18 10:39 Session.db-wal -
So it is climbing back up and fast. Show that to support.
-
Memory is up to 398M used now.
SAR is bouncing around as well -
08:00:01 AM CPU %user %nice %system %iowait %steal %idle 08:10:01 AM all 25.18 0.00 0.58 0.07 0.16 74.01 08:20:01 AM all 7.17 0.00 0.34 0.03 0.10 92.37 08:30:01 AM all 25.70 0.00 0.52 0.12 0.23 73.42 08:40:01 AM all 6.75 0.00 0.34 0.03 0.13 92.75 08:50:01 AM all 24.80 0.00 0.50 0.07 0.17 74.45 09:00:01 AM all 7.47 0.00 0.34 0.02 0.08 92.08 09:10:01 AM all 28.54 0.00 0.63 0.09 0.48 70.27 09:20:01 AM all 7.15 0.00 0.36 0.17 0.17 92.16 09:30:01 AM all 26.84 0.00 0.54 0.09 0.34 72.20 09:40:01 AM all 8.76 0.00 0.36 0.07 0.27 90.54 09:50:01 AM all 26.93 0.00 0.53 0.07 0.31 72.15 10:00:01 AM all 8.60 0.00 0.35 0.03 0.18 90.84 10:10:01 AM all 28.29 0.00 0.63 0.08 0.47 70.53 10:20:01 AM all 9.19 0.00 0.33 0.08 0.28 90.12 10:30:01 AM all 27.27 0.00 0.52 0.08 0.32 71.81 10:40:01 AM all 8.15 0.00 0.33 0.07 0.23 91.22 10:50:01 AM all 25.58 0.00 0.46 0.07 0.27 73.61 11:00:01 AM all 41.85 0.00 0.66 0.14 0.29 57.06 11:10:01 AM all 26.54 0.00 0.62 0.10 0.47 72.27 11:20:01 AM all 8.44 0.00 0.39 0.03 0.21 90.92 11:30:01 AM all 43.08 0.00 0.68 0.15 0.58 55.50 11:40:01 AM all 27.83 0.00 0.52 0.11 0.47 71.07 11:50:01 AM all 24.17 0.00 0.53 0.07 0.38 74.84 12:00:01 PM all 8.93 0.00 0.42 0.04 0.31 90.30 12:10:01 PM all 26.29 0.00 0.60 0.10 0.40 72.61 12:20:01 PM all 9.36 0.00 0.61 0.05 0.17 89.80 12:30:01 PM all 23.48 0.00 0.66 0.07 0.19 75.60 12:40:01 PM all 9.61 0.00 0.50 0.06 0.21 89.63 12:50:01 PM all 23.44 0.00 0.62 0.06 0.18 75.70 01:00:01 PM all 8.63 0.00 0.48 0.03 0.10 90.76 01:10:01 PM all 24.10 0.00 0.70 0.07 0.21 74.93 Average: all 8.90 0.00 0.41 0.07 0.15 90.46 -
Yup that CPU is WAY too high.
-
Curious - system did a restart this evening.
05:05:15 PM LINUX RESTART 05:10:01 PM CPU %user %nice %system %iowait %steal %idle 05:20:01 PM all 3.56 0.00 0.15 0.02 0.09 96.19 05:30:01 PM all 14.36 0.00 0.23 0.09 0.07 85.25 05:40:01 PM all 4.74 0.00 0.16 0.03 0.09 94.99 05:50:01 PM all 16.36 0.00 0.26 0.14 0.11 83.13 06:00:01 PM all 3.68 0.00 0.13 0.02 0.06 96.12 06:10:01 PM all 16.09 0.00 0.27 0.10 0.13 83.41 06:20:01 PM all 4.60 0.00 0.24 0.13 0.09 94.96 06:30:01 PM all 15.20 0.00 0.24 0.32 0.11 84.14 06:40:01 PM all 3.54 0.00 0.13 0.01 0.05 96.27 06:50:01 PM all 15.19 0.00 0.24 0.09 0.07 84.40 07:00:01 PM all 5.10 0.00 0.14 0.02 0.07 94.68 07:10:01 PM all 15.11 0.00 0.29 0.13 0.11 84.36 07:20:01 PM all 3.91 0.00 0.12 0.03 0.06 95.89 07:30:01 PM all 14.58 0.00 0.24 0.08 0.08 85.02 07:40:01 PM all 4.20 0.00 0.18 0.02 0.05 95.55 07:50:01 PM all 14.34 0.00 0.26 0.07 0.08 85.25 Average: all 9.65 0.00 0.20 0.08 0.08 89.98DB is still on the upside for 25MB
]$ ls -l total 49596 -rw-r--r-- 1 root root 454 Aug 25 19:15 ExtensionConfiguration.xml drwxr-xr-x 2 root root 4096 Nov 8 09:44 Helper -rw-r--r-- 1 root root 654 Aug 25 19:15 License.xml -rw-r--r-- 1 root root 23699 Dec 16 11:26 Role.xml -rw-r--r-- 1 root root 25583616 Dec 18 19:48 Session.db -rw-r--r-- 1 root root 32768 Dec 18 19:50 Session.db-shm -rw-r--r-- 1 root root 25082592 Dec 18 19:50 Session.db-wal -rw-r--r-- 1 root root 4030 Oct 25 08:55 SessionEventTrigger.xml -rw-r--r-- 1 root root 5178 Dec 2 08:13 SessionGroup.xml drwxr-xr-x 14 root root 4096 Dec 12 12:55 Toolbox -rw-r--r-- 1 root root 0 Dec 14 17:24 User.cd60043d-556e-4b23-9ef1-9959e6cea952.xml -rw-r--r-- 1 root root 21908 Dec 17 10:41 User.xml -
After speaking with support this morning. Plan is to stop the SC service, rename the DB and start the service to allow the system to rebuild it.
-
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
After speaking with support this morning. Plan is to stop the SC service, rename the DB and start the service to allow the system to rebuild it.
what data will you lose?
-
@Dashrender said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
After speaking with support this morning. Plan is to stop the SC service, rename the DB and start the service to allow the system to rebuild it.
what data will you lose?
We should not lose any data. Just some down time while the process is completeed.
-
Process done:
sudo service screenconnect stopmv /opt/screenconnect/Session.db /opt/screenconnect/Session.db.baksudo service screenconnect start -
]$ ls -l total 29724 -rw-r--r-- 1 root root 454 Aug 25 19:15 ExtensionConfiguration.xml drwxr-xr-x 2 root root 4096 Nov 8 09:44 Helper -rw-r--r-- 1 root root 654 Aug 25 19:15 License.xml -rw-r--r-- 1 root root 23699 Dec 16 11:26 Role.xml -rw-r--r-- 1 root root 1232896 Dec 19 20:14 Session.db -rw-r--r-- 1 root root 24956928 Dec 19 20:11 Session.db.bak -rw-r--r-- 1 root root 32768 Dec 19 20:18 Session.db-shm -rw-r--r-- 1 root root 4128272 Dec 19 20:18 Session.db-wal -rw-r--r-- 1 root root 4030 Oct 25 08:55 SessionEventTrigger.xml -rw-r--r-- 1 root root 5178 Dec 2 08:13 SessionGroup.xml drwxr-xr-x 14 root root 4096 Dec 12 12:55 Toolbox -rw-r--r-- 1 root root 0 Dec 14 17:24 User.cd60043d-556e-4b23-9ef1-9959e6cea952.xml -rw-r--r-- 1 root root 21908 Dec 19 20:13 User.xml -
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
-rw-r--r-- 1 root root 244,855,752 Dec 14 16:25 Session.db-wal-rw-r--r-- 1 root root 78,403,632 Dec 16 08:33 Session.db-walrw-r--r-- 1 root root 4,128,272 Dec 17 02:17 Session.db-walGoing back and doing some comparisons and noticed this -
(commas added for 'dramatic effect'
-rw-r--r-- 1 root root 25,082,592 Dec 18 10:39 Session.db-walSession.db-wal is not the right file to be looking at... Should be looking at session.db
-
@Dashrender said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
After speaking with support this morning. Plan is to stop the SC service, rename the DB and start the service to allow the system to rebuild it.
what data will you lose?
The only thing we seem to have lost is Support Sessions, as they are temporary anyway.
-
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@Dashrender said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
After speaking with support this morning. Plan is to stop the SC service, rename the DB and start the service to allow the system to rebuild it.
what data will you lose?
The only thing we seem to have lost is Support Sessions, as they are temporary anyway.
Taht would be expected.
-
So how are we looking after a few hours?
-
@scottalanmiller said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
So how are we looking after a few hours?
We are at 320 of the 540 devices previously logg d.
I expect this as some devices are off,...
-
Any CPU spikes?
-
DB modification was done at 8:00pm(ish) last night, SAR going back to Midnight
]$ sar Linux 3.10.0-327.10.1.el7.x86_64 (server) 12/20/2016 _x86_64_ (1 CPU) 12:00:01 AM CPU %user %nice %system %iowait %steal %idle 12:10:01 AM all 0.42 0.00 0.07 0.02 0.02 99.48 12:20:01 AM all 1.46 0.00 0.16 0.07 0.02 98.29 12:30:01 AM all 0.34 0.00 0.05 0.01 0.01 99.59 12:40:01 AM all 1.02 0.00 0.09 0.08 0.03 98.78 12:50:01 AM all 0.34 0.00 0.05 0.02 0.01 99.58 01:00:01 AM all 1.01 0.00 0.10 0.07 0.01 98.81 01:10:01 AM all 0.64 0.00 0.11 0.02 0.04 99.19 01:20:01 AM all 1.12 0.00 0.09 1.03 0.06 97.71 01:30:01 AM all 0.39 0.00 0.05 0.02 0.03 99.51 01:40:01 AM all 1.06 0.00 0.09 0.07 0.02 98.75 01:50:01 AM all 0.35 0.00 0.05 0.01 0.01 99.57 02:00:01 AM all 0.96 0.00 0.09 0.08 0.02 98.85 02:10:01 AM all 0.57 0.00 0.11 0.02 0.02 99.29 02:20:01 AM all 1.08 0.00 0.10 0.18 0.01 98.62 02:30:01 AM all 0.34 0.00 0.05 0.02 0.02 99.58 02:40:01 AM all 1.02 0.00 0.09 0.07 0.02 98.80 02:50:01 AM all 0.32 0.00 0.05 0.11 0.01 99.51 03:00:01 AM all 1.00 0.00 0.10 0.06 0.02 98.82 03:10:01 AM all 0.39 0.00 0.06 0.02 0.02 99.52 03:20:01 AM all 1.28 0.02 0.15 0.14 0.03 98.38 03:30:01 AM all 0.34 0.00 0.05 0.02 0.02 99.56 03:40:01 AM all 1.05 0.00 0.10 0.08 0.03 98.75 03:50:01 AM all 0.33 0.00 0.06 0.02 0.02 99.58 04:00:01 AM all 1.04 0.00 0.10 0.07 0.03 98.76 04:10:01 AM all 1.07 0.00 0.14 0.04 0.07 98.67 04:20:02 AM all 1.17 0.00 0.13 0.08 0.06 98.57 04:30:01 AM all 0.44 0.31 0.12 0.04 0.04 99.06 04:40:01 AM all 1.07 0.00 0.11 0.08 0.04 98.70 04:50:01 AM all 0.47 0.00 0.07 0.02 0.02 99.42 05:00:01 AM all 1.04 0.00 0.11 0.17 0.05 98.64 05:10:01 AM all 0.76 0.00 0.12 0.04 0.05 99.02 05:20:01 AM all 1.02 0.00 0.09 0.07 0.06 98.76 05:30:01 AM all 0.55 0.00 0.07 0.03 0.04 99.31 05:30:01 AM CPU %user %nice %system %iowait %steal %idle 05:40:01 AM all 1.10 0.00 0.12 0.07 0.03 98.68 05:50:01 AM all 0.55 0.00 0.07 0.03 0.04 99.31 06:00:01 AM all 1.02 0.00 0.09 0.07 0.04 98.78 06:10:01 AM all 1.16 0.00 0.18 0.05 0.07 98.55 06:20:01 AM all 1.43 0.00 0.16 0.09 0.11 98.22 06:30:01 AM all 0.48 0.00 0.06 0.02 0.05 99.39 06:40:01 AM all 1.06 0.00 0.09 0.28 0.07 98.51 06:50:01 AM all 1.13 0.00 0.09 0.03 0.08 98.67 07:00:01 AM all 8.95 0.00 0.17 0.34 0.17 90.37 07:10:01 AM all 3.81 0.00 0.15 0.04 0.18 95.82 07:20:01 AM all 9.25 0.00 0.15 0.17 0.22 90.21 07:30:01 AM all 3.97 0.00 0.10 0.04 0.16 95.72 07:40:01 AM all 10.44 0.00 0.17 0.10 0.29 89.00 07:50:01 AM all 4.62 0.00 0.10 0.05 0.18 95.05 08:00:01 AM all 10.71 0.00 0.15 0.09 0.14 88.91 08:10:01 AM all 5.20 0.00 0.12 0.07 0.22 94.39 Average: all 1.84 0.01 0.10 0.09 0.06 97.90]$ ls -l total 30856 -rw-r--r-- 1 root root 454 Aug 25 19:15 ExtensionConfiguration.xml drwxr-xr-x 2 root root 4096 Nov 8 09:44 Helper -rw-r--r-- 1 root root 654 Aug 25 19:15 License.xml -rw-r--r-- 1 root root 23699 Dec 16 11:26 Role.xml -rw-r--r-- 1 root root 2383872 Dec 20 08:09 Session.db -rw-r--r-- 1 root root 24956928 Dec 19 20:11 Session.db.bak -rw-r--r-- 1 root root 32768 Dec 20 08:16 Session.db-shm -rw-r--r-- 1 root root 4136512 Dec 20 08:16 Session.db-wal -rw-r--r-- 1 root root 4030 Oct 25 08:55 SessionEventTrigger.xml -rw-r--r-- 1 root root 5178 Dec 2 08:13 SessionGroup.xml drwxr-xr-x 14 root root 4096 Dec 12 12:55 Toolbox -rw-r--r-- 1 root root 0 Dec 14 17:24 User.cd60043d-556e-4b23-9ef1-9959e6cea952.xml -rw-r--r-- 1 root root 21908 Dec 19 23:07 User.xmlCurrently 373 devices of the orginial 540
-
I expect that there are so 'few' devices is that they get powered off at the end of the day, and have not been powered on as of yet to call home.
-
Now showing 405 devices connected.
-
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
Now showing 405 devices connected.
How about the CPU?
