For anyone who has been upgrading their Fedora installs from version to version there was a surprise event with the upgrade from 38 to 39. At that point the boot process was changed from using grub to systemd-boot. It's a testament to the smoothness with which Fedora maintainers execute even large changes that this had gone largely unnoticed.

This didn't last long though because part of the change is that kernels and their associated intird images now where no longer installed into /boot but into /boot/efi. The EFI partition is a vfat partition which generally is of a quite small size (256MB in my installation), and it frankly is way too small to hold modern kernels. Now kernel updates were installed but since their post-install scripts failed they were never added to the boot manager and the system continues to boot the last kernel from F38, which is 6.8.8, from its old /boot location. Nothing broke, systems continued to boot, but also were stuck in time on old kernels. Depending on your desktop environment you may also have seen short messages pop up saying /boot/efi was running out of space.

The probably best fix would be to increase /boot/efi to a reasonable size. In many instances it tends to be wedged in between /boot and the LVM volume that holds the rest of the partitions though, and resizing this means shrinking the LVM pv. Not fun.

Moving Back to Grub

The other option is to revert the system back to grub booting and thus freeing the EFI partition from all that data. Here are some notes on how to do that. Also consult the Fedora forum thread linked at the bottom.

Note: proceed at your own risk. Ensure you have a complete and current backup of everything on your system before attempting any of this!

First check that your system is in fact using systemd-boot and your grub does not show current kernels. Basically you are looking for layout=bls or the variable KERNEL_INSTALL_LAYOUT=bls¹:

kernel-install
grubby --info=ALL

Next, a prerequisite to force the kernel-core package to utilize grub is to remove the BLS artifacts. Go to /boot/efi and examine the existence of a subdirectory with a long, random-looking name such as 1fc4aee1c0a349e2a6ad4c525925cc4f/ containing subdirectories similar to this structure:

1fc4aee1c0a349e2a6ad4c525925cc4f/
├── 0-rescue
│   └── linux
├── 4.15.15-300.fc27.x86_64
├── 4.15.16-300.fc27.x86_64
├── 4.16.5-200.fc27.x86_64
├── 6.9.4-200.fc40.x86_64
│   └── initrd
└── 7.00

tar the entire directory to somewhere safe and remove it from /boot/efi. If this directory remains, kernel-core will fall back to the BLS layout no matter what other settings you may specify.

Thirdly, the post suggests to set "layout=other" in either /usr/lib/kernel/install.conf (which by default exists but is completely commented out) or /etc/kernel/install.conf (which by default doesn't exist and would override the other config file if it did). I found this to not be necessary, leaving everything as it is.

Then, switch back to using grub by executing bootctl remove².

Lastly, (re-)install the kernel update by running dnf reinstall kernel-core which will also trigger the scripts contained inside the package. If all went well another run of grubby --info=ALL will show the kernel installed successfully and was also added to the grub config (the entry for 6.9.4 in this case):

# grubby --info=ALL
index=0
kernel="/boot/vmlinuz-6.9.4-200.fc40.x86_64"
args="ro rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet"
root="/dev/mapper/fedora-root"
initrd="/boot/initramfs-6.9.4-200.fc40.x86_64.img"
title="Fedora Linux (6.9.4-200.fc40.x86_64) 40 (Workstation Edition)"
id="1fc4aee1c0a349e2a6ad4c525925cc4f-6.9.4-200.fc40.x86_64"
index=1
kernel="/boot/vmlinuz-6.8.8-100.fc38.x86_64"
args="ro rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet"
root="/dev/mapper/fedora-root"
initrd="/boot/initramfs-6.8.8-100.fc38.x86_64.img"
title="Fedora Linux (6.8.8-100.fc38.x86_64) 38 (Workstation Edition)"
id="1fc4aee1c0a349e2a6ad4c525925cc4f-6.8.8-100.fc38.x86_64"
[...]

Only if this is the case should you attempt a reboot. Otherwise you need to go back through the above steps and verify what you did.

What If It Doesn't Work?

If this fails to solve the issue and your system is left in an unbootable stage your best course of action is to boot from a live USB thumb drive. You can use any current Fedora installer to do that. Do not attempt an installation but boot into the live system, mount the disk(s) with your installation, execute a chroot, and then attempt to fix the issue again.

Here's a good article to follow: How I recovered my Linux system using a Live USB device.

Links

Fedora discussion board thread: https://discussion.fedoraproject.org/t/new-kernel-versions-get-stored-in-boot-efi-and-dont-show-in-grub-after-fedora-39-update/92847