I have the same issue with Wireguard. On my home wifi, my phone has an IPv4 address and a link-local (useless) IPv6 address (fe80:) and on 4G, my phone has an IPv6 address and no IPv4 address.
If Wireguard connect to an IPv4 address, it works only on wifi. If I use an IPv6 address, Wireguard only works on 4G.
I worked nicely before moving to Android 10, but it’s certainly also linked to the IP versions available from my mobile provider.