How I tweak my bootc base image

In the last post I explained why I decided to switch to BootC for all my servers. This time I want to walk through what I actually put into my base image - the packages, the config files, and the custom services that make every host feel like home.

The full source is in my gorge.os repository.

The Containerfile structure

Before getting into what’s in the image, I want to explain how I put things into it. The Containerfile is split into three stages: package management, file placement, and service activation.

For the package lists I use two plain text files (packages-add and packages-remove) and just pipe them into dnf. Comments start with # and get stripped by grep before passing to dnf. Keeps the Containerfile clean and the package list readable.

grep -vE '^#' scripts/packages-remove | xargs dnf -y remove
grep -vE '^#' scripts/packages-add    | xargs dnf -y install --allowerasing

For config files I use a naming convention instead of a full directory tree: double underscores encode path separators. A file named etc__systemd__journald.conf ends up at /etc/systemd/journald.conf. The Containerfile translates this with a one-liner using sed and find. It sounds odd at first, but it means all files to deploy sit flat in one files/ directory and there is no nested folder structure to maintain in the repo.

Every build ends with:

RUN bootc container lint

This catches common mistakes before the image ever leaves the build machine.

Packages

The Fedora BootC base image already ships a reasonable set of packages, but a few of them are either redundant or outright incompatible with my workflow.

Removed:

Added:

For everyday server work: neovim, tmux, git, rsync, net-tools, borgbackup, borgmatic, rclone.

For monitoring and troubleshooting: lm_sensors, sysstat, python3-psutil, python3-dotenv, btop, lshw, ncdu, nmap, pciutils, tcpdump.

For networking and storage: wireguard-tools, nfs-utils, cifs-utils, zram-generator, zram-generator-defaults.

For tuning: tuned – a small daemon that applies performance profiles and makes a noticeable difference on mixed-workload hosts.

I also symlink nvim to vim so muscle memory works:

RUN ln /usr/bin/nvim /usr/bin/vim

System configuration

Journald limits

By default journald will happily fill up a disk. I cap it early:

# /etc/systemd/journald.conf
SystemMaxUse=500M       # keep at most 500 MB of journal data
SystemKeepFree=200M     # reserve 200 MB free on the disk
SystemMaxFileSize=20M   # max size of a single journal file
MaxRetentionSec=30day   # keep entries for 30 days

Quieter kernel console

Kernel messages on the console are noisy and rarely useful on a server. A single sysctl line dials them back without touching the journal:

echo -e "kernel.printk = 2\t4\t1\t7" >> /usr/lib/sysctl.d/99-user.conf

sshd and /var/home

On Fedora BootC, /var/home is a separate mount. If sshd starts before that mount is ready, logins fail silently. A small drop-in makes sure sshd waits:

# /usr/lib/systemd/system/sshd.service.d/override.conf
[Unit]
RequiresMountsFor=/var/home

Custom update workflow

The default bootc-fetch-apply-updates.timer fetches and applies updates automatically. I prefer to separate the two steps: fetch regularly, but only reboot on my schedule. So I mask the default timer and replace it with my own.

bootc-fetch.service – just runs bootc update, which pulls the latest image and stages it without rebooting:

[Service]
Type=oneshot
ExecStart=/usr/bin/bootc update

bootc-fetch.timer – runs hourly with a randomised 30-minute delay so not all hosts hit the registry at the same time:

[Timer]
OnCalendar=hourly
AccuracySec=5m
RandomizedDelaySec=30m
Persistent=true

The actual reboot, to switch to the staged image, is a separate step I control manually or via another timer per host, depending on the host’s role.

Boot automation

Every time a host boots, bootc-boot.service runs a small Python script (/usr/local/sbin/bootc-boot.py). Right now it does one thing: merge system users from /usr/etc/passwd (and the companion files for group, shadow, gshadow) into /etc/.

This lets me ship user definitions inside the image, in /usr/etc/, and have them applied on every boot without overwriting entries that the host added locally to /etc/. It’s how I manage service accounts across all hosts from a single place.

def merge_user():
    merge_files("/usr/etc/passwd", "/etc/passwd")
    merge_files("/usr/etc/group",  "/etc/group")
    # ... shadow, gshadow

The script uses MD5 checksums to track whether source files have changed, so it can skip work when nothing is new.

Watchdog reboot

Applying a new BootC image and rebooting can go wrong. Especially Cloud-Init has proven to block the boot-up sometimes. If the image fails to reach multi-user.target, or if /run/nologin is still present five minutes after boot, I want the host to reboot itself automatically.

multiuser-reboot.timer fires 5 minutes after boot and runs multiuser-reboot-check.sh:

if ! systemctl is-active --quiet multi-user.target; then
    logger -p warning "multi-user.target not active after 5min - rebooting now"
    systemctl reboot
fi

if [ -f /run/nologin ]; then
    logger -p warning "/run/nologin available after 5min - rebooting now"
    systemctl reboot
fi

A reboot after a failed boot will land on the previous (working) image thanks to BootC’s rollback mechanism. Combined with the staged-update approach above, this means a bad image can never permanently lock me out of a host.

Metrics

Every host ships influx_monitor.py, a small Python script that collects CPU, memory and disk usage and pushes the metrics to an InfluxDB 2.x instance via the HTTP write API. Configuration (URL, token, org, bucket) comes from an .env file on a network mount, so I don’t have to bake credentials into the image.

The service runs as UID 1000, restarts automatically on failure, and reads its config from /mnt/gv0/environments/influx_monitor.env, a path that sits on shared GlusterFS storage, available on all hosts.

Shell defaults

The .bashrc that lands in /etc/skel (and therefore in every new user’s home) sets a few things I want everywhere:

• vi mode (set -o vi) in bash
• A coloured prompt that turns red inside a toolbox container, so I always know where I am
• Aliases for the commands I type dozens of times a day (la, ll, h, hg, svi, …)
• extract() – a function that unpacks any archive format with one command
• find_extension() – lists all file extensions in a directory tree

That covers the base image. In the next post I will show how to extend this base for specific host roles (adding a database, a container runtime, or an application stack) without touching the base layer.