System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7
-
@hobbit666 said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
This is all i'm getting from SAR
[root@screenconnect6 ~]# sar
Linux 3.10.0-327.28.3.el7.x86_64 (screenconnect6) 15/12/16 x86_64 (2 CPU)16:20:43 LINUX RESTART
Takes ten minutes to accumulate the first line of data.
-
@hobbit666 said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@scottalanmiller said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
Resized and the issue remains. Look at that CPU load for an idle box!!
How do you get that pretty picture??
Command is htop, it's in the EPEL. Then I just screenshot it. It's the default terminal on Ubuntu 16.10.
-
OK so i'll leave my VM running over night as i'll be heading home soon.
I've put a sticky note on my monitor to remind me to run the commands and post screenshots
-
Tagging @LG-at-ConnectWise
-
Current status, SC wants us to run a DB maintainance job. But due to some bug in SC, it takes two days to run. So it was scheduled to run early this morning but won't kick off until Saturday. So no way to get feedback on the system till Monday. Considering the need for the maintenance was done Wednesday, could not be kicked off because of their GUI dependency till Thursday, takes two days to run and two days to respond, it has turned into a five day turn around, at best, on what should have been a thirty minute issue.
-
popping out for lunch when I get back I'll post some screen shots of our system.
-
@scottalanmiller Hey! Thanks for tagging me. I'm glad to see that Support has been working in this. I am going to share this thread with them as well so that they can see the conversation surrounding the issue. Let me know if you have any successes!
-
SAR as of now
Linux 3.10.0-327.10.1.el7.x86_64 (dny-lnx-sc) 12/16/2016 _x86_64_ (2 CPU) 12:00:02 AM CPU %user %nice %system %iowait %steal %idle 12:10:01 AM all 1.36 0.00 0.24 0.07 0.55 97.77 12:20:02 AM all 0.51 0.00 0.14 0.02 0.41 98.92 12:30:01 AM all 1.14 0.00 0.17 0.08 0.66 97.95 12:40:01 AM all 0.54 0.00 0.13 0.02 0.32 98.99 12:50:01 AM all 1.13 0.00 0.17 0.05 0.43 98.21 01:00:01 AM all 0.51 0.00 0.13 0.02 0.31 99.03 01:10:01 AM all 1.59 0.00 0.30 0.07 0.72 97.32 01:20:01 AM all 0.52 0.00 0.14 0.02 0.39 98.93 01:30:01 AM all 1.14 0.00 0.19 0.10 0.53 98.05 01:40:01 AM all 0.51 0.00 0.15 0.07 0.60 98.68 01:50:02 AM all 1.10 0.00 0.16 0.06 0.52 98.16 02:00:01 AM all 0.56 0.00 0.15 0.02 0.42 98.85 02:10:01 AM all 1.15 0.00 0.20 0.12 0.66 97.88 02:20:01 AM all 0.99 0.00 0.27 0.03 0.74 97.97 02:30:02 AM all 1.29 0.00 0.21 0.06 0.63 97.81 02:40:01 AM all 0.56 0.00 0.15 0.02 0.43 98.84 02:50:01 AM all 1.19 0.00 0.18 0.06 0.63 97.95 03:00:01 AM all 0.54 0.00 0.14 0.03 0.42 98.86 03:10:01 AM all 1.69 0.00 0.32 0.07 0.81 97.12 03:20:01 AM all 0.91 0.02 0.21 0.06 0.62 98.18 03:30:01 AM all 1.31 0.00 0.21 0.06 0.71 97.70 03:40:01 AM all 0.64 0.00 0.17 0.15 0.51 98.53 03:50:01 AM all 1.54 0.00 0.26 0.07 0.82 97.30 04:00:01 AM all 0.90 0.00 0.22 0.03 0.68 98.18 04:10:01 AM all 2.02 0.00 0.35 0.07 0.97 96.59 04:20:01 AM all 0.81 0.00 0.22 0.02 0.63 98.32 04:30:01 AM all 1.48 0.00 0.26 0.07 0.84 97.34 04:40:01 AM all 0.86 0.00 0.23 0.03 0.71 98.18 04:50:01 AM all 1.31 0.00 0.21 0.09 0.51 97.88 05:00:01 AM all 0.46 0.00 0.12 0.08 0.35 98.99 05:10:01 AM all 1.35 0.00 0.23 0.06 0.58 97.78 05:20:01 AM all 0.44 0.00 0.10 0.02 0.33 99.11 05:30:01 AM all 1.26 0.00 0.21 0.07 0.66 97.80 05:40:01 AM all 0.81 0.00 0.22 0.02 0.48 98.46 05:50:01 AM all 1.52 0.00 0.26 0.07 0.71 97.45 06:00:01 AM all 0.64 0.00 0.14 0.02 0.43 98.76 06:10:01 AM all 1.57 0.00 0.29 0.06 0.62 97.47 06:20:01 AM all 0.85 0.00 0.21 0.03 0.56 98.35 06:30:01 AM all 1.34 0.00 0.22 0.07 0.83 97.53 06:40:01 AM all 0.79 0.23 0.30 0.10 0.61 97.97 06:50:01 AM all 1.15 0.00 0.19 0.06 0.62 97.98 07:00:01 AM all 0.67 0.00 0.15 0.03 0.51 98.65 07:10:02 AM all 1.17 0.00 0.19 0.07 0.62 97.95 07:20:02 AM all 0.71 0.00 0.16 0.03 0.36 98.74 07:30:01 AM all 1.36 0.00 0.22 0.07 0.55 97.81 07:40:01 AM all 5.82 0.00 0.26 0.03 0.63 93.25 07:50:01 AM all 15.07 0.00 0.35 0.07 1.14 83.36 08:00:01 AM all 8.80 0.00 0.27 0.04 0.79 90.11 08:10:01 AM all 16.20 0.00 0.36 0.07 1.13 82.24 08:20:01 AM all 10.55 0.00 0.24 0.04 1.02 88.15 08:30:01 AM all 16.24 0.00 0.32 0.10 1.20 82.15 08:40:01 AM all 10.00 0.00 0.30 0.03 0.96 88.72 08:50:02 AM all 16.61 0.00 0.30 0.08 1.19 81.82 Average: all 2.72 0.00 0.22 0.06 0.64 96.36 -
HTOP
One thing I notice different from yesterday afternoon - Using about 300MB more memory, and use of SWAP.
-
Current size of the DB
$ ls -l total 100856 -rw-r--r-- 1 root root 454 Aug 25 19:15 ExtensionConfiguration.xml drwxr-xr-x 2 root root 4096 Nov 8 09:44 Helper -rw-r--r-- 1 root root 654 Aug 25 19:15 License.xml -rw-r--r-- 1 root root 22986 Dec 15 08:42 Role.xml -rw-r--r-- 1 root root 24621056 Dec 16 08:29 Session.db -rw-r--r-- 1 root root 163840 Dec 16 08:33 Session.db-shm -rw-r--r-- 1 root root 78403632 Dec 16 08:33 Session.db-wal -rw-r--r-- 1 root root 4030 Oct 25 08:55 SessionEventTrigger.xml -rw-r--r-- 1 root root 5178 Dec 2 08:13 SessionGroup.xml drwxr-xr-x 14 root root 4096 Dec 12 12:55 Toolbox -rw-r--r-- 1 root root 0 Dec 14 17:24 User.cd60043d-556e-4b23-9ef1-9959e6cea952.xml -rw-r--r-- 1 root root 21908 Dec 15 19:41 User.xml -
@gjacobse Thanks for posting - I will pass along to Support
-
-rw-r--r-- 1 root root 240685056 Dec 14 16:25 Session.db -rw-r--r-- 1 root root 491520 Dec 14 16:25 Session.db-shm -rw-r--r-- 1 root root 244855752 Dec 14 16:25 Session.db-wal -rw-r--r-- 1 root root 4030 Oct 25 08:55 SessionEventTrigger.xml -rw-r--r-- 1 root root 5178 Dec 2 08:13 SessionGroup.xmlThis is from two days ago for comparison.
-
Updated - Midnight to 8:50am
-
06:00:01 AM all 0.64 0.00 0.14 0.02 0.43 98.76 06:10:01 AM all 1.57 0.00 0.29 0.06 0.62 97.47 06:20:01 AM all 0.85 0.00 0.21 0.03 0.56 98.35 06:30:01 AM all 1.34 0.00 0.22 0.07 0.83 97.53 06:40:01 AM all 0.79 0.23 0.30 0.10 0.61 97.97 06:50:01 AM all 1.15 0.00 0.19 0.06 0.62 97.98 07:00:01 AM all 0.67 0.00 0.15 0.03 0.51 98.65 07:10:02 AM all 1.17 0.00 0.19 0.07 0.62 97.95 07:20:02 AM all 0.71 0.00 0.16 0.03 0.36 98.74 07:30:01 AM all 1.36 0.00 0.22 0.07 0.55 97.81 07:40:01 AM all 5.82 0.00 0.26 0.03 0.63 93.25 07:50:01 AM all 15.07 0.00 0.35 0.07 1.14 83.36 08:00:01 AM all 8.80 0.00 0.27 0.04 0.79 90.11 08:10:01 AM all 16.20 0.00 0.36 0.07 1.13 82.24 08:20:01 AM all 10.55 0.00 0.24 0.04 1.02 88.15 08:30:01 AM all 16.24 0.00 0.32 0.10 1.20 82.15 08:40:01 AM all 10.00 0.00 0.30 0.03 0.96 88.72 08:50:02 AM all 16.61 0.00 0.30 0.08 1.19 81.82 09:00:01 AM all 11.60 0.00 0.27 0.16 1.06 86.91 09:10:01 AM all 17.84 0.00 0.30 0.09 1.24 80.54 09:10:01 AM CPU %user %nice %system %iowait %steal %idle 09:20:01 AM all 11.77 0.00 0.27 0.13 0.90 86.93 09:30:01 AM all 17.04 0.00 0.29 0.09 1.16 81.41 09:40:01 AM all 13.90 0.00 0.31 0.06 1.14 84.59 09:50:02 AM all 23.21 0.00 0.42 0.08 1.70 74.60 10:00:01 AM all 12.95 0.00 0.27 0.06 1.15 85.56 Average: all 4.19 0.00 0.23 0.06 0.70 94.81No one is currently using SC, so no connections
-
-
Where is the DB stored?
-
htop @ 16:10
sar @ 16:00
-
@hobbit666 said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
Where is the DB stored?
/opt/screenconnect/App_Data
-
@hobbit666 That's what our top output looked like before this issue hit.
-
@hobbit666
How many clients (devices) do you have? We have hit 540 as of this week.
