See what the server is really sending

Connect and show the certificates the server sends. Always pass -servername so SNI picks the right certificate on servers that host more than one name. Without it, a shared server may hand you its default certificate, and the check looks fine while the real site is broken.

openssl s_client -connect vpn.contoso.com:443 -servername vpn.contoso.com -showcerts </dev/null

The output lists each certificate the server sent, numbered from 0 (the leaf). Each one shows s: (subject) and i: (issuer). A healthy chain from a public CA usually has the leaf at 0 and one or more intermediates after it. If only certificate 0 appears and its issuer is an intermediate CA, the server isn't sending its chain.

Reading the expiry date

Pipe the leaf into openssl x509 to read its dates:

echo | openssl s_client -connect vpn.contoso.com:443 -servername vpn.contoso.com 2>/dev/null \
  | openssl x509 -noout -subject -issuer -enddate

subject=CN = vpn.contoso.com
issuer=C = US, O = Example CA, CN = Example Intermediate R1
notAfter=Mar 10 23:59:59 2027 GMT

In scripts, I lean on -checkend SECONDS. It exits non-zero if the certificate expires within that many seconds, so you don't have to parse dates or deal with the differences between GNU and BSD date.

Checking the chain

Look at the verification result at the end of the s_client output:

openssl s_client -connect vpn.contoso.com:443 -servername vpn.contoso.com </dev/null 2>/dev/null | grep 'Verify return code'

You want Verify return code: 0 (ok). The failures we see most:

• 21 (unable to verify the first certificate) or 20 (unable to get local issuer certificate) usually means the server is missing an intermediate.
• 10 (certificate has expired) means what it says.
• 18 or 19 means a self-signed certificate, or a private CA this machine doesn't trust. For internal hosts, pass -CAfile with your root.

So why don't browsers complain about a missing intermediate? Many browsers cache intermediates or fetch them using the Authority Information Access URL in the leaf. Command-line tools, VPN clients, Java and Python applications, mail servers and older mobile devices often don't, so they fail while the browser looks fine. That's why I never test with a browser alone. The fix is to install the full chain on the server, meaning the leaf plus intermediates but not the root.

The script we run every morning

Put your endpoints in a text file, one host port per line:

vpn.contoso.com 443
mail.contoso.com 443
portal.contoso.com 8443
fw01.contoso.com 443

Then run this script from cron daily:

#!/usr/bin/env bash
# cert-check.sh - warn on certificates expiring soon or with broken chains
# usage: cert-check.sh hosts.txt [warn_days]
LIST="${1:-hosts.txt}"
WARN_DAYS="${2:-30}"
SECS=$((WARN_DAYS * 86400))
status=0

while read -r host port; do
  [[ -z "$host" || "$host" == \#* ]] && continue
  port="${port:-443}"
  out=$(timeout 15 openssl s_client -connect "$host:$port" -servername "$host" </dev/null 2>/dev/null)
  cert=$(printf '%s\n' "$out" | openssl x509 2>/dev/null)
  if [[ -z "$cert" ]]; then
    echo "ERROR  $host:$port  no certificate retrieved"; status=2; continue
  fi
  end=$(printf '%s\n' "$cert" | openssl x509 -noout -enddate | cut -d= -f2)
  verify=$(printf '%s\n' "$out" | grep -m1 'Verify return code' | sed 's/^ *//')
  if ! printf '%s\n' "$cert" | openssl x509 -noout -checkend 0 >/dev/null; then
    echo "EXPIRED $host:$port  $end"; status=2
  elif ! printf '%s\n' "$cert" | openssl x509 -noout -checkend "$SECS" >/dev/null; then
    echo "WARN   $host:$port  expires $end (under $WARN_DAYS days)"; status=1
  else
    echo "OK     $host:$port  expires $end"
  fi
  if [[ "$verify" != *"0 (ok)"* ]]; then
    echo "CHAIN  $host:$port  $verify"; [[ $status -lt 1 ]] && status=1
  fi
done < "$LIST"
exit $status

Schedule it so it only emails when the exit code is non-zero. That way, a certificate email in your inbox always means something needs attention.

0 7 * * * /opt/scripts/cert-check.sh /opt/scripts/hosts.txt 30 > /tmp/cert-check.out || mail -s 'Certificate check' noc@contoso.com < /tmp/cert-check.out

Most monitoring systems can do this natively too, with HTTP checks on certificate age or a TLS sensor. If you already have one, use it. We still find the script handy for appliances on odd ports and as a second opinion.

For us the hard part wasn't the script, it was building the list. VPN gateways, firewall admin pages, UPS cards and printers are the ones that expire unnoticed, so build the list from DNS and your firewall NAT rules rather than from memory.

What you get out of it

• Log collection and analysis. Agents ship Windows event logs, syslog and application logs to the manager, which decodes them and matches them against thousands of rules.
• File integrity monitoring (FIM), which alerts when watched files, directories or registry keys change
• Vulnerability detection. Agents inventory installed packages and the manager compares them against CVE feeds.
• Security configuration assessment (SCA), which scores endpoints against CIS-style benchmark checks
• Active response, which is optional automated action such as blocking an IP after repeated failures. I'd be careful with this one.

Lock down enrollment first

Out of the box, anything that can reach port 1515 can enroll as an agent. Turn on password enrollment on the manager. In /var/ossec/etc/ossec.conf, inside the <auth> section, set <use_password>yes</use_password>, then write the password to the auth file:

echo 'REPLACE-WITH-LONG-RANDOM-STRING' > /var/ossec/etc/authd.pass
chmod 640 /var/ossec/etc/authd.pass
chown root:wazuh /var/ossec/etc/authd.pass
systemctl restart wazuh-manager

One group per client

Groups let you push different configuration to different agents, and they let whoever is looking at the dashboard filter down to one client. Create them on the manager before you deploy anything:

/var/ossec/bin/agent_groups -a -g client-contoso -q
/var/ossec/bin/agent_groups -a -g client-fabrikam -q
/var/ossec/bin/agent_groups -l

Each group gets a shared config file at /var/ossec/etc/shared/GROUP/agent.conf, and agents in that group pull it automatically. That's where client-specific FIM paths or extra log sources go.

Pushing agents from the RMM

The agent packages accept deployment variables, so one command installs the agent, enrolls it and assigns the group. Match the agent version to your manager version or older, because an agent newer than the manager won't connect. On Windows:

$msi = "$env:TEMP\wazuh-agent.msi"
Invoke-WebRequest -Uri 'https://packages.wazuh.com/4.x/windows/wazuh-agent-4.x.x-1.msi' -OutFile $msi
Start-Process msiexec.exe -Wait -ArgumentList "/i `"$msi`" /q WAZUH_MANAGER=`"wazuh.contoso.com`" WAZUH_REGISTRATION_PASSWORD=`"REPLACE-ME`" WAZUH_AGENT_GROUP=`"client-contoso`""
Start-Service -Name WazuhSvc

Replace 4.x.x with the exact version you run. On Debian or Ubuntu, with the Wazuh repository already added:

WAZUH_MANAGER='wazuh.contoso.com' WAZUH_REGISTRATION_PASSWORD='REPLACE-ME' \
  WAZUH_AGENT_GROUP='client-contoso' apt-get install -y wazuh-agent
systemctl daemon-reload
systemctl enable --now wazuh-agent

Keep the registration password in an RMM secret or script variable instead of hard-coding it in a script every technician can see.

The first week is loud

Expect a flood at first. Don't disable whole rule groups. Write targeted overrides in /var/ossec/etc/rules/local_rules.xml instead, where a child rule with level 0 suppresses a specific match. For example, this silences successful logons from a known service account:

<group name="local,windows,">
  <rule id="100100" level="0">
    <if_sid>60106</if_sid>
    <field name="win.eventdata.targetUserName">^svc-backup$</field>
    <description>Ignore routine logons by the backup service account</description>
  </rule>
</group>

Use custom rule IDs from 100000 upward. Test by pasting a sample event into /var/ossec/bin/wazuh-logtest, then restart the manager.

Treat the manager like a domain controller

Your SIEM holds logs from every client and can push configuration to every agent. If someone gets into it, they get into everyone.

• Don't expose the dashboard to the internet without SSO and MFA in front of it, or keep it reachable only over VPN.
• Change the default admin passwords for the dashboard, indexer and API right after install.
• If agents connect over the internet, expose only 1514 and 1515, and consider restricting them to client office IPs.
• Keep the API port (55000) internal.
• Back up /var/ossec/etc (keys, rules, groups), and plan indexer retention so the disk doesn't fill.

/var/ossec/bin/agent_control -l
/var/ossec/bin/agent_groups -l -g client-contoso

Each week I look at the top rules by count in the dashboard and tune whichever one is worst. It only takes a few minutes, and it's what keeps our techs reading the alerts instead of muting them.

SNMPv3 on the device

Use SNMPv3 with authentication and privacy (authPriv). SNMPv2c community strings travel in clear text. Here's a generic Cisco IOS example with a read-only view and an ACL limiting who can poll:

ip access-list standard SNMP-POLLERS
 permit 10.10.20.5
!
snmp-server view LIBRENMS-VIEW iso included
snmp-server group LIBRENMS-GRP v3 priv read LIBRENMS-VIEW access SNMP-POLLERS
snmp-server user librenms LIBRENMS-GRP v3 auth sha AUTH-PASSPHRASE priv aes 128 PRIV-PASSPHRASE
snmp-server location Main Office MDF
snmp-server contact noc@contoso.com

On IOS, SNMPv3 users don't show up in the running config, so put the credentials in your password manager right away. You can check the user with show snmp user. It's also worth setting a meaningful location string, because LibreNMS uses it for maps and grouping.

Prove SNMP works before LibreNMS gets involved

I always test with snmpwalk from the LibreNMS host before adding a device. It tells you whether you have an SNMP problem or a LibreNMS problem:

snmpwalk -v3 -l authPriv -u librenms -a SHA -A 'AUTH-PASSPHRASE' -x AES -X 'PRIV-PASSPHRASE' 10.10.20.1 sysDescr

If that returns the device description, add it from the web UI (Devices, Add Device) or the CLI:

cd /opt/librenms
sudo -u librenms ./lnms device:add --v3 -l authPriv -u librenms \
  -a SHA -A 'AUTH-PASSPHRASE' -x AES -X 'PRIV-PASSPHRASE' 10.10.20.1

Discovery runs, and the device fills in ports, sensors and hardware details over the next few polling cycles.

A short list of alert rules

LibreNMS ships a set of default rules you can import from Alerts, Alert Rules. Start small. An alert stream nobody reads is worse than none. These are the rules we always deploy:

• Device down, meaning device status is 0 with ICMP and SNMP both failing
• Port down on uplinks, scoped to ports with a description you tag rather than every access port
• Sensor over limit, for temperature, PSU and UPS battery state
• Device rebooted, meaning uptime below a few minutes, which catches power events

Use device groups, for example one per client or site, to scope rules and route them to the right transport. Set a sensible delay so a single missed poll doesn't page anyone.

Where the alerts go

Transports are what deliver alerts. Under Alerts, Alert Transports, add one per destination:

• For Discord, paste the channel webhook URL. It's good for team visibility.
• Most ticket systems can create tickets by email, or through an API transport that POSTs JSON to an endpoint.
• For PagerDuty, use the integration key from a PagerDuty service. We only page for real outages, so nobody gets woken up for a warning.

Map rules to transports directly on the rule or through transport groups. Press the Test button on each transport before you rely on it.

cd /opt/librenms
sudo -u librenms ./validate.php

After any change to transports, I press Test on every one of them. One broken transport can silence all the others, and you won't find out until a real outage goes unreported.

What it looks like in the logs

Once you've seen it, the pattern is easy to recognize:

• Thousands to tens of thousands of authentication sessions per day, far more than your user count could explain
• Lots of usernames, often ones that don't exist in your directory, like admin, test, scanner, or first.last guesses from a scraped staff list
• A handful of source /24 networks, often hosting providers, rotating through addresses within each block
• No successes at all. If any attempts succeed, stop reading and start incident response.

Counting failures by network

Every product logs differently, so you'll need to adapt the field extraction. Here's a generic log line:

2026-10-01T03:12:44Z portal auth=fail user=jsmith src=203.0.113.45 reason=bad_password

Count failed attempts per source /24:

grep 'auth=fail' portal.log \
  | grep -oE 'src=[0-9.]+' | cut -d= -f2 \
  | awk -F. '{print $1"."$2"."$3".0/24"}' \
  | sort | uniq -c | sort -rn | head -20

Then check how many distinct usernames each network tried. A real user behind a NAT fails with one or two names. A sprayer tries hundreds.

grep 'auth=fail' portal.log \
  | awk '{for(i=1;i<=NF;i++){if($i~/^user=/)u=substr($i,6); if($i~/^src=/)s=substr($i,5)}
         split(s,o,"."); print o[1]"."o[2]"."o[3]".0/24", u}' \
  | sort -u | awk '{print $1}' | uniq -c | sort -rn | head -20

And confirm there were no successes from those networks:

grep 'auth=success' portal.log | grep -E 'src=(203\.0\.113|198\.51\.100)\.' | wc -l

For this client, five networks accounted for essentially all of the failures, each had tried hundreds of usernames, and the success count was zero.

Block it before it reaches login and MFA

Where you block matters. If the portal forwards each attempt to your MFA provider or directory, the spray costs you more than log space. It loads your domain controllers, can lock out real staff whose usernames happen to match, and can spam people with push prompts if a password ever does match. Drop the traffic before it gets to the login logic.

Most platforms have a named list you can block against. It might be an address list or address group on a firewall, a data group referenced by a load balancer rule, or an external dynamic list. Keep the list in one place so adding a network is a one-line change. On a Linux edge, ipset does the same job:

ipset create spray-block hash:net
ipset add spray-block 203.0.113.0/24
ipset add spray-block 198.51.100.0/24
iptables -I INPUT -p tcp --dport 443 -m set --match-set spray-block src -j DROP

Make the rule persistent using your distribution's mechanism, and put a comment or ticket reference on each entry. Whoever looks at that list next will want to know why it's there.

Geo-blocking, if your users are all in one country

Blocking five networks handles today's sprayer, and the next one will come from somewhere else. If every legitimate user is in one country, allowing only that country to reach the portal removes most of the noise with one rule. Most commercial firewalls have built-in geo objects. Keep an exception process for staff who travel. Geo-IP data is approximate and VPN providers can appear to be anywhere, so I treat this as noise reduction, not a security control on its own.

MFA and lockout still do the real work

Edge blocking cuts load and noise, but it doesn't protect accounts. Attackers rotate sources, and spraying is slow on purpose. What actually protects your users:

• MFA on every remote access path, preferably phishing-resistant or number-matching, so a guessed password alone isn't enough
• A lockout or smart-lockout policy that slows guessing without making it trivial to lock out the whole staff
• Banned-password lists so seasonal passwords like Fall2026! get rejected
• Alerts on any successful login from an unusual country or network, and on MFA prompts users report they didn't trigger

We re-run the per-network count on that portal every so often. New sprayers keep turning up from new places, and adding another /24 to the list takes about a minute.