DNF Regression Preventing Terra from Loading (Round 2)

I was packaging the spleen-font package, it was going pretty well, and the build initially succeeded. Then Roachy told me there's a better way to format the spec file to match the Fedora guidelines. I was like "oh that's not a bad idea", I committed my changes, pushed, and the CI failed.

"Interesting… It built locally on my laptop though…"

I looked into the GitHub Actions logs, and I saw the terrifying error again:

Updating and loading repositories:
 packages for the GitHub CLI            100% |  81.1 KiB/s |   3.0 KiB |  00m00s
 Fedora - Rawhide - Developmental packa 100% | 212.0 KiB/s |  28.4 KiB |  00m00s
 Terra rawhide                          100% |   3.8 KiB/s |   7.9 KiB |  00m02s
>>> repomd.xml GPG signature verification error: Signing key not found          
 Terra rawhide (Extras)                 100% |   8.4 KiB/s |   6.8 KiB |  00m01s
>>> repomd.xml GPG signature verification error: Signing key not found          
Repositories loaded.
Failed to resolve the transaction:
No match for argument: bdf2sfd
Package "fonts-srpm-macros-1:2.0.5-22.fc43.noarch" is already installed.
You can try to add to command line:
  --skip-unavailable to skip unavailable packages

Workarounds

If you are having this exact issue: don't worry, just downgrade dnf5:

sudo dnf downgrade dnf5

You could also then set a version lock on the faulty package for now:

sudo dnf versionlock add dnf5

Just don't forget to remove it when dnf5 gets fixed.

History Repeats Itself

As mentioned in the issue tracker (#2323), this is not the first time this bug has appeared. I reported the exact same issue back in March (#2134). While I'm not (at all) familiar with dnf5's development, I think they may need to perform more tests on a piece of software the entire RPM ecosystem relies upon.

Thankfully, I'm somewhat prepared since I still remember how I triaged the issue last time, so I did the same and I found the commit that caused the issue relatively quickly.

In response to my issue report, the author of the commit promised that a new CI test will be added so that it wouldn't happen yet again.

The Effects

A slight issue is that Terra Rawhide would basically die for the next week until dnf5 gets bumped, otherwise since this was discovered pretty early, and it's the same as last time, everyone already has experience on handling it and can mitigate issues pretty quickly.

I've notified the maintainers at Universal Blue already, they should be fine as their users don't use dnf5 directly; rather dnf5 is used during the image builds.

I've also posted about the issue in the Discord server, on Fediverse, and on Reddit.

Getting Support

Join one of our chats, the subreddit, or open an Issue on GitHub. We'll get you going again in no time.

While the entire blog post might sound like me just constantly blasting bullets on dnf5 maintainers, that's not my intention at all—I highly appreciate their time and effort on dnf5. It has has been a breeze to work with compared with dnf4, which was significantly slower. As one of the first adopters of dnf5 (earlier than Fedora!), our community is pretty happy with dnf5 in general, and I've never regretted making dnf5 the default package manager back in Ultramarine Linux 40 and Terra 38.