Pi as a UPS monitor

gjacobse

checking on my UPS this morning

Since reboot, the monitor has been running all the time. So it may have been simply that.

As for having multiple UPS monitoring, I was curious what you thought about having a central display which showed the status of each UPS monitored. That way you have one display to glance at for a health check.

Maybe something similar to the UBNT controller software - central configuration ability.....

travisdh1

@gjacobse said in Pi as a UPS monitor:

checking on my UPS this morning

 ~ $ uptime
 09:59:20 up 6 days, 18:12,  2 users,  load average: 0.19, 0.25, 0.25

 ~ $ sudo upsc pandora
Init SSL without certificate database
battery.charge: 100
battery.charge.low: 10
battery.charge.warning: 50
battery.date: not set
battery.mfr.date: 2014/06/22
battery.runtime: 3172
battery.runtime.low: 120
battery.type: PbAc
battery.voltage: 13.6
battery.voltage.nominal: 12.0
device.mfr: APC
device.model: Back-UPS ES 550G
device.serial: 4B1425P54201
device.type: ups
driver.name: usbhid-ups
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.version: 2.7.1
driver.version.data: APC HID 0.95
driver.version.internal: 0.38
input.sensitivity: medium
input.transfer.high: 139
input.transfer.low: 92
input.voltage: 126.0
input.voltage.nominal: 120
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.firmware: 904.W1 .D
ups.firmware.aux: W1
ups.load: 13
ups.mfr: APC
ups.mfr.date: 2014/06/22
ups.model: Back-UPS ES 550G
ups.productid: 0002
ups.serial: 4B1425P54201
ups.status: OL
ups.timer.reboot: 0
ups.timer.shutdown: -1
ups.vendorid: 051d

Since reboot, the monitor has been running all the time. So it may have been simply that.

As for having multiple UPS monitoring, I was curious what you thought about having a central display which showed the status of each UPS monitored. That way you have one display to glance at for a health check.

Take the output of upsc and throw it up on a web page. Unless you want a local display, in which case it's actually more difficult.

thwr

@gjacobse said in Pi as a UPS monitor:

checking on my UPS this morning

 ~ $ uptime
 09:59:20 up 6 days, 18:12,  2 users,  load average: 0.19, 0.25, 0.25

 ~ $ sudo upsc pandora
Init SSL without certificate database
battery.charge: 100
battery.charge.low: 10
battery.charge.warning: 50
battery.date: not set
battery.mfr.date: 2014/06/22
battery.runtime: 3172
battery.runtime.low: 120
battery.type: PbAc
battery.voltage: 13.6
battery.voltage.nominal: 12.0
device.mfr: APC
device.model: Back-UPS ES 550G
device.serial: 4B1425P54201
device.type: ups
driver.name: usbhid-ups
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.version: 2.7.1
driver.version.data: APC HID 0.95
driver.version.internal: 0.38
input.sensitivity: medium
input.transfer.high: 139
input.transfer.low: 92
input.voltage: 126.0
input.voltage.nominal: 120
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.firmware: 904.W1 .D
ups.firmware.aux: W1
ups.load: 13
ups.mfr: APC
ups.mfr.date: 2014/06/22
ups.model: Back-UPS ES 550G
ups.productid: 0002
ups.serial: 4B1425P54201
ups.status: OL
ups.timer.reboot: 0
ups.timer.shutdown: -1
ups.vendorid: 051d

Since reboot, the monitor has been running all the time. So it may have been simply that.

As for having multiple UPS monitoring, I was curious what you thought about having a central display which showed the status of each UPS monitored. That way you have one display to glance at for a health check.

AMQP / MQTT may be awesome for this at the transport layer. MQTT is an interesting protocol that let you define a "last will and testament" action in case something goes south.

Ansible/Puppet etc. can handle the central configuration.

JaredBusch

Hey 3 months later.....

JaredBusch

Alright, I plugged the thing back in and got to testing.

Yeah! It still communicates.

pi@raspberrypi:~ $ sudo upsc jaredoffice
Init SSL without certificate database
battery.charge: 84
battery.charge.low: 10
battery.charge.warning: 50
battery.date: 2001/09/25
battery.mfr.date: 2010/12/15
battery.runtime: 1236
battery.runtime.low: 120
battery.type: PbAc
battery.voltage: 27.0
battery.voltage.nominal: 24.0
device.mfr: American Power Conversion
device.model: Back-UPS BR1000G
device.serial: 3B1051X20329
device.type: ups
driver.name: usbhid-ups
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.version: 2.7.2
driver.version.data: APC HID 0.95
driver.version.internal: 0.38
input.sensitivity: medium
input.transfer.high: 147
input.transfer.low: 88
input.transfer.reason: input voltage out of range
input.voltage: 124.0
input.voltage.nominal: 120
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.firmware: 868.L1 .D
ups.firmware.aux: L1
ups.load: 20
ups.mfr: American Power Conversion
ups.mfr.date: 2010/12/15
ups.model: Back-UPS BR1000G
ups.productid: 0002
ups.realpower.nominal: 600
ups.serial: 3B1051X20329
ups.status: OL CHRG
ups.test.result: No test initiated
ups.timer.reboot: 0
ups.timer.shutdown: -1
ups.vendorid: 051d

JaredBusch

Pulled the power cord from the wall. UPS beeps, but nothing from nut.

Ok back to the manual and checking the conf files.

Well it would probably help if I told the thing what to monitor..
/etc/nut/upsmon.conf

MONITOR jaredoffice@localhost 1 upsmon demo master

Oh but those details have to match /etc/nut/ups.conf and /etc/nut/upsd.users. Well I already know that my unit is called jaredoffice@localhost from ups.conf. So just edit upsd.users.

pi@raspberrypi:~ $ sudo nano /etc/nut/upsd.users
# put at end of file
[upsmon]
password = demo
upsmon master

Now reload the monitor

pi@raspberrypi:~ $ sudo upsmon -c reload
Network UPS Tools upsmon 2.7.2
pi@raspberrypi:~ $

Then unplug the UPS. and get nothing.

look in the logs..
tail/var/log/syslog

Feb  1 17:05:38 raspberrypi upsmon[732]: UPS jaredoffice@localhost on battery
Feb  1 17:05:38 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Wed Feb  1 17:06:08 2017 [try http://www.rsyslog.com/e/2007 ]
Feb  1 17:05:38 raspberrypi upssched[964]: Can't open /etc/nut/upssched.conf: Permission denied
Feb  1 17:05:48 raspberrypi upsmon[732]: UPS jaredoffice@localhost on line power
Feb  1 17:05:48 raspberrypi upssched[967]: Can't open /etc/nut/upssched.conf: Permission denied

Say to yourself, WTF did I do 3 months ago....

Time to reload and start over.

gjacobse

@JaredBusch said in Pi as a UPS monitor:

Pulled the power cord from the wall. UPS beeps, but nothing from nut.

Ok back to the manual and checking the conf files.

Well it would probably help if I told the thing what to monitor..
/etc/nut/upsmon.conf
MONITOR jaredoffice@localhost 1 upsmon demo master
Oh but those details have to match /etc/nut/ups.conf and /etc/nut/upsd.users. Well I already know that my unit is called jaredoffice@localhost from ups.conf. So just edit upsd.users.
pi@raspberrypi:~ $ sudo nano /etc/nut/upsd.users
# put at end of file
[upsmon]
password = demo
upsmon master
Now reload the monitor
pi@raspberrypi:~ $ sudo upsmon -c reload
Network UPS Tools upsmon 2.7.2
pi@raspberrypi:~ $
Then unplug the UPS. and get nothing.

look in the logs..
tail/var/log/syslog
Feb  1 17:05:38 raspberrypi upsmon[732]: UPS jaredoffice@localhost on battery
Feb  1 17:05:38 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Wed Feb  1 17:06:08 2017 [try http://www.rsyslog.com/e/2007 ]
Feb  1 17:05:38 raspberrypi upssched[964]: Can't open /etc/nut/upssched.conf: Permission denied
Feb  1 17:05:48 raspberrypi upsmon[732]: UPS jaredoffice@localhost on line power
Feb  1 17:05:48 raspberrypi upssched[967]: Can't open /etc/nut/upssched.conf: Permission denied
Say to yourself, WTF did I do 3 months ago....

Time to reload and start over.

think i got the same thing, and decide the same course of action, to which I have not moved forward with. Albeit, it's not on a rPi but a desktop - premise is the same.

Think we may have to create a user for the NUT system.... but again,.. can't work with it right now.

JaredBusch

Quick check of upsmon.conf found this, uncommented it.

# RUN_AS_USER nut

Rebooted Pi, unplugged UPS.

Broadcast message from nut@raspberrypi (somewhere) (Wed Feb  1 17:19:04 2017):

UPS jaredoffice@localhost on battery


Broadcast message from nut@raspberrypi (somewhere) (Wed Feb  1 17:19:24 2017):

UPS jaredoffice@localhost on line power

JaredBusch

Syslog had this though.

Feb  1 17:19:04 raspberrypi upsmon[734]: UPS jaredoffice@localhost on battery
Feb  1 17:19:11 raspberrypi upssched[792]: Failed to connect to parent and failed to create parent: No such file or directory
Feb  1 17:19:24 raspberrypi upsmon[734]: UPS jaredoffice@localhost on line power

Back to the config files we go!
/etc/nut/upssched.conf

# PIPEFN <filename>
# This is commented out by default to make you visit this file and think
# about how your system works before potentially opening a hole.

FFS, but it gets better! Right below that, also commented out.

# LOCKFN <filename>
#
# REQUIRED.  This was added after version 1.2.1.
#
# upssched needs to be able to create this filename in order to avoid
# a race condition when two events are dispatched from upsmon at nearly
# the same time.  This file will only exist briefly.  It must not be
# created by any other process.
#
# You should put this in the same directory as PIPEFN.
#

So I update the config and make a directory forgetting to chown it from root /sigh...

PIPEFN /etc/nut/upssched/upssched.pipe
LOCKFN /etc/nut/upssched/upssched.lock

JaredBusch

Once it was all chowned...

Feb  1 17:33:08 raspberrypi upssched[793]: Timer daemon started
Feb  1 17:33:09 raspberrypi upssched[793]: New timer: onbattwarn (30 seconds)
Feb  1 17:33:39 raspberrypi upssched[793]: Event: onbattwarn
Feb  1 17:33:54 raspberrypi upssched[793]: Timer queue empty, exiting

And output to the screen:

Broadcast message from nut@raspberrypi (somewhere) (Wed Feb  1 17:33:39 2017):

The UPS is currently running on battery power!

JaredBusch

Now time to wipe, replicate, test and document.

Then the command can be changed to send email.

JaredBusch

Following my own notes from post 62
At step 11 (installing nut form repo) I see an error.
May or may not have gotten this the first time around..

pi@bna-pwr-pi-01:~ $ sudo apt-get install nut
Reading package lists... Done
<snip>
Unpacking nut (2.7.2-4) ...
Processing triggers for man-db (2.7.0.2-5) ...
Processing triggers for systemd (215-17+deb8u6) ...
Setting up libnspr4:armhf (2:4.12-1+debu8u1) ...
Setting up libnss3:armhf (2:3.26-1+debu8u1) ...
Setting up libupsclient4:armhf (2.7.2-4) ...
Setting up nut-client (2.7.2-4) ...
Job for nut-monitor.service failed. See 'systemctl status nut-monitor.service' and 'journalctl -xn' for details.
invoke-rc.d: initscript nut-client, action "start" failed.
Setting up nut-server (2.7.2-4) ...
A dependency job for nut-server.service failed. See 'journalctl -xn' for details.
invoke-rc.d: initscript nut-server, action "start" failed.
Job for nut-driver.service failed. See 'systemctl status nut-driver.service' and 'journalctl -xn' for details.
Setting up nut (2.7.2-4) ...
Processing triggers for libc-bin (2.19-18+deb8u7) ...
Processing triggers for systemd (215-17+deb8u6) ...
pi@bna-pwr-pi-01:~ $ systemctl status nut-monitor
● nut-monitor.service - Network UPS Tools - power device monitor and shutdown controller
   Loaded: loaded (/lib/systemd/system/nut-monitor.service; enabled)
   Active: failed (Result: resources) since Thu 2017-02-02 02:10:27 UTC; 49s ago
pi@bna-pwr-pi-01:~ $

JaredBusch

ah here we go, this is normal because not configured.

pi@bna-pwr-pi-01:~ $ sudo journalctl -xn
-- Logs begin at Fri 2016-11-25 18:24:08 UTC, end at Thu 2017-02-02 02:33:03 UTC. --
Feb 02 02:32:47 bna-pwr-pi-01 sudo[23103]: pam_unix(sudo:session): session opened for user root by pi(uid=0)
Feb 02 02:32:47 bna-pwr-pi-01 systemd[1]: Starting Network UPS Tools - power device monitor and shutdown controller...
-- Subject: Unit nut-monitor.service has begun with start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit nut-monitor.service has begun starting up.
Feb 02 02:32:47 bna-pwr-pi-01 upsmon[23113]: upsmon disabled, please adjust the configuration to your needs
Feb 02 02:32:47 bna-pwr-pi-01 upsmon[23113]: Then set MODE to a suitable value in /etc/nut/nut.conf to enable it
Feb 02 02:32:47 bna-pwr-pi-01 systemd[1]: PID file /var/run/nut/upsmon.pid not readable (yet?) after start.
Feb 02 02:32:47 bna-pwr-pi-01 systemd[1]: Failed to start Network UPS Tools - power device monitor and shutdown controller.
-- Subject: Unit nut-monitor.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit nut-monitor.service has failed.
-- 
-- The result is failed.
Feb 02 02:32:47 bna-pwr-pi-01 systemd[1]: Unit nut-monitor.service entered failed state.
Feb 02 02:32:48 bna-pwr-pi-01 sudo[23103]: pam_unix(sudo:session): session closed for user root
Feb 02 02:33:03 bna-pwr-pi-01 sudo[23121]: pi : TTY=pts/0 ; PWD=/home/pi ; USER=root ; COMMAND=/bin/journalctl -xn
Feb 02 02:33:03 bna-pwr-pi-01 sudo[23121]: pam_unix(sudo:session): session opened for user root by pi(uid=0)

JaredBusch

So on to configuration. Nut has only a few config files you need to deal with.

We will go right down the list.

pi@bna-pwr-pi-01:~ $ ls -l /etc/nut
total 44
-rw-r----- 1 root nut  1538 Mar 26  2015 nut.conf
-rw-r----- 1 root nut  4572 Mar 26  2015 ups.conf
-rw-r----- 1 root nut  4578 Mar 26  2015 upsd.conf
-rw-r----- 1 root nut  2131 Mar 26  2015 upsd.users
-rw-r----- 1 root nut 15170 Mar 26  2015 upsmon.conf
-rw-r----- 1 root nut  3887 Mar 26  2015 upssched.conf

JaredBusch

First up is nut.conf. There is only a single option to set in this file, but it is important.

Assuming you are going to have only one device, or if multiple devices, that they will report in on their own, the simplest configuration is to choose standalone

MODE=standalone

Here are the various meanings from the conf file.

# - none: NUT is not configured, or use the Integrated Power Management, or use
#   some external system to startup NUT components. So nothing is to be started.
# - standalone: This mode address a local only configuration, with 1 UPS
#   protecting the local system. This implies to start the 3 NUT layers (driver,
#   upsd and upsmon) and the matching configuration files. This mode can also
#   address UPS redundancy.
# - netserver: same as for the standalone configuration, but also need
#   some more network access controls (firewall, tcp-wrappers) and possibly a
#   specific LISTEN directive in upsd.conf.
#   Since this MODE is opened to the network, a special care should be applied
#   to security concerns.
# - netclient: this mode only requires upsmon.

JaredBusch

Next up is ups.conf, again read the config file for more information, it is fairly thorough.

In this case I am going to connect it to the APC unit pictured at the beginning of this thread.

[bnajaredrouter]
driver = usbhid-ups
port = auto
desc = "Jared Router UPS"

JaredBusch

The file upsd.conf does not need modified for a typical standalone setup.

JaredBusch

The file upsd.users needs modified to have the authentication that upsmon will use added to the end of the file.

[bnaupsmon]
password = AGoodPassword
upsmon master

JaredBusch

The file upsmon.conf starts to get into the nuts and bolts of setting things up to do things for you.

First up will be to uncomment the run as user setting.

RUN_AS_USER nut

Next is creating the monitor line to tell it what system to monitor. This uses the settings from ups.conf and upsd.users that you previously configured.

MONITOR bnajaredrouter@localhost 1 bnaupsmon AGoodPassword master

continuing down the file, the next thing to do is to set notifycmd to point to the upssched program

NOTIFYCMD /sbin/upssched

The final part of this file is to uncomment all of the notify flags and add the EXEC flag to the ones you want to fire the above upssched application. Leaving WALL in while testing is useful, but pretty pointless later for me since the goal is a remote alert.

NOTIFYFLAG ONLINE       SYSLOG+WALL+EXEC
NOTIFYFLAG ONBATT       SYSLOG+WALL+EXEC
NOTIFYFLAG LOWBATT      SYSLOG+WALL
NOTIFYFLAG FSD          SYSLOG+WALL
NOTIFYFLAG COMMOK       SYSLOG+WALL
NOTIFYFLAG COMMBAD      SYSLOG+WALL
NOTIFYFLAG SHUTDOWN     SYSLOG+WALL
NOTIFYFLAG REPLBATT     SYSLOG+WALL
NOTIFYFLAG NOCOMM       SYSLOG+WALL
NOTIFYFLAG NOPARENT     SYSLOG+WALL

JaredBusch

If you never set any of the flags to EXEC in the previous section, there is nothing to do here, you can stop. But what would be the point of a remote alerting device that does not alert.

So now we come to the last conf file upssched.conf. This one is a bit more annoying because the developers intentionally chose to ship this broken to 'force' you to set up a file yourself.
By default, PIPEFN and LOCKFN are commented out and point to the /var/run/nut/upssched directory that does not exist.

# PIPEFN /var/run/nut/upssched/upssched.pipe
# LOCKFN /var/run/nut/upssched/upssched.lock

So we make a directory and the pipe file. You are instructed to not make the lock file in the comments.

sudo mkdir /etc/nut/upssched
sudo chown nut:nut /etc/nut/upssched
sudo touch /etc/nut/upssched/upssched.pipe
sudo chown nut:nut /etc/nut/upssched/upssched.pipe

Editing this file, note the location the shell script that will be called. This is the default, and there is an example script already there, ready to be modified.

CMDSCRIPT /bin/upssched-cmd

Uncomment and update the PIPE and LOCK lines.

PIPEFN /etc/nut/upssched/upssched.pipe
LOCKFN /etc/nut/upssched/upssched.lock

Now comes the part that does work, the AT commads. You will want to read the comments and likely the documentation to make full use of this. But here are a couple examples of AT commands to get you going.
When the unit goes on battery, call the shell script to start a 30 second timer named 'onbattwarn'. When the power comes back online, cancel the timer.

AT ONBATT * START-TIMER onbattwarn 30
AT ONLINE * CANCEL-TIMER onbattwarn
AT ONLINE * EXECUTE ongrid