As long as TWRP works (and you have a backup of your data), you are in control.
You could use TWRP to wipe the phone completely and install the OS from scratch.
For Fairphone OS, you can slightly adapt the Fairphone Open OS procedure given here, you would just use the Fairphone OS files you downloaded instead of Fairphone Open OS …