Thanks for the pointers @CransNeighbour, I finally figured that there is something wrong with the checksums as you said, so I decided to restore the stock OS via a PC.
I was able to update the OS using a PC by reinstalling, without losing my user data, as described here: Unable to install August security update on FP3 - #25 by StevenHachel
After that, I rooted my phone again. This time (instead of TWRP) I used Magisk’s instructions for patching the boot.img and flashing, (and re-flashing the vbmeta image), will see whether that allows for proper OTA behaviour. Anyway, everything works properly now, and I have the latest security update as well. 