systemd logs: journalctl in plain language

Almost everything that happens on the system ends up in the systemd journal. A service won't start, the machine rebooted on its own, the site returns a 502 — the answer is usually already recorded, you just need to know how to ask.

Five commands that are enough

journalctl -u nginx -n 50 --no-pager      # last 50 lines of one service
journalctl -u nginx -f                    # follow in real time
journalctl -p err -b                      # only errors since boot
journalctl --since "10 min ago"           # everything for the last ten minutes
journalctl -b -1 -e                       # previous boot log, end of file

The last one is especially valuable after a sudden reboot: -b -1 opens the previous session, and at the end you can see whether it was a kernel panic, the OOM killer at work, or a clean shutdown.

The levels in -p: emerg, alert, crit, err, warning, notice, info, debug. The specified level includes all more important ones.

Diagnosing a failed service

systemctl status myapp
journalctl -u myapp --since today -p warning

systemctl status gives the exit code and the last lines, journalctl gives the full picture. Look at:

• Main process exited, code=exited, status=1 — the application exited with an error on its own, the cause is on the line above;
• Failed to start ... Permission denied — permissions on a file, directory, or a port below 1024;
• Killed process ... (java) total-vm... from the kernel — the OOM killer kicked in, read Out of memory;
• Start request repeated too quickly — the service is crash-looping, systemd has stopped starting it; fix the cause and run systemctl reset-failed myapp.

A list of everything broken on the system is systemctl --failed.

Filters that save time

journalctl --since "2026-08-22 03:00" --until "2026-08-22 04:00"
journalctl -u nginx -u php-fpm            # several services at once
journalctl _PID=1432                      # by specific process
journalctl -k                             # only kernel messages
journalctl -u sshd | grep "Failed password" | tail -20

Combine with grep, but remember: journalctl without --no-pager opens a pager, exit with q.

Keeping the journal from eating up the disk

By default, the journal grows to 10% of the partition. On VDS-1 with a 40 GB disk, that's four gigabytes — noticeable.

journalctl --disk-usage
journalctl --vacuum-size=200M
journalctl --vacuum-time=14d

The permanent limit is in /etc/systemd/journald.conf:

[Journal]
SystemMaxUse=300M
MaxRetentionSec=1month

Then systemctl restart systemd-journald.

Common mistakes

• Reading the journal without -u. The general stream is clogged with service records, and the line you need gets lost. Always narrow it down to a service or level.
• Losing history after a reboot. If the /var/log/journal directory doesn't exist, the journal lives only in memory. Create it: mkdir -p /var/log/journal && systemd-tmpfiles --create --prefix /var/log/journal.
• Looking for application errors there. Nginx, PostgreSQL, and similar also write to their own files in /var/log. If the journal is empty, check there.

When the disk is full, journalctl may refuse to work until space is freed up. This situation is covered in the article Out of disk space.