Parsing auth.log: counting failed SSH logins

You're writing the Tuesday-morning report: how many failed SSH logins hit the bastion last night, and from which IPs. The evidence is /var/log/auth.log — hundreds of megabytes of syslog — and the answer has to be right, because the SOC lead will spot-check one IP against the raw file. This is the most common shape of security tooling: boring input, easy to get wrong, and nobody verifies your math until it matters.

A Debian/Ubuntu auth.log line looks like this:

Oct  1 03:12:47 bastion sshd[8811]: Failed password for invalid user admin from 203.0.113.10 port 51422 ssh2
Oct  1 03:13:02 bastion sshd[8823]: Failed password for root from 198.51.100.7 port 40211 ssh2

The text after sshd[pid]: is free-form prose and changes by message type — Failed password for admin from versus Failed password for invalid user admin from. Splitting on whitespace breaks immediately. Use one regex with named groups, tested against both shapes:

import re
from collections import Counter

FAILED = re.compile(
    r"Failed password for (?:invalid user )?(?P<user>\S+) "
    r"from (?P<ip>\S+) port \d+"
)

def failed_logins(path):
    by_ip = Counter()
    with open(path, "r", encoding="utf-8", errors="replace") as f:
        for line in f:
            m = FAILED.search(line)
            if m:
                by_ip[m.group("ip")] += 1
    return by_ip

errors="replace" matters: real logs contain stray non-UTF-8 bytes — binary junk, attacker-controlled usernames in odd encodings — and the default strict error handling kills your run at 3 a.m. on line four million. re.search (not re.match) skips the syslog prefix so you only model the message.

Rotated logs are gzipped (auth.log.2.gz, auth.log.3.gz, ...). gzip.open(path, "rt", encoding="utf-8", errors="replace") presents the same line-by-line interface, so a helper that picks open vs gzip.open by suffix lets you walk a whole rotation history:

import gzip

def open_log(path):
    if str(path).endswith(".gz"):
        return gzip.open(path, "rt", encoding="utf-8", errors="replace")
    return open(path, "r", encoding="utf-8", errors="replace")

Print the top offenders and you have the report:

for ip, count in failed_logins("/var/log/auth.log").most_common(10):
    print(f"{count:6d}  {ip}")

Try this

Extend the script to also count Accepted password events per user, then flag any IP with more than 50 failures and at least one success — that's your shortlist for credential-stuffing wins. Run it against a week of rotations.

Field notes

• Syslog timestamps have no year. Oct 1 could be last week or last October. On systemd hosts, journalctl -u ssh -o json gives you real timestamps and pre-parsed fields — often the better source entirely.
• The bug that ships: matching Failed password but forgetting invalid user, so your counts silently drop the username-enumeration traffic — usually the majority of failures.
• On RHEL-family hosts the same data lives in /var/log/secure, and the sshd message text is identical; the regex carries over untouched.