Do you use that “one-off” mode of TWRP, the one using it just for one time instead of installing it fully? Ubports needs their own recovery for OTA updates to work.
Which partitions do you backup and restore?
Sounds like an interesting project but I think I need a bigger SD card for that (the only spare one I had is a 2 GB card from my old dumb phone). 