My phone (Fairphone 4) connected to Fairphone True Wireless Stereo (TWS) Earbuds keeps asking to turn on Google Assistant

Don’t want to turn on Google Assistant, but the message requesting to do so when I use the earbuds keeps flashing up. Even when in the case, but when in ear it interferes with whatever I am playing from the phone at that moment. You can imagine: highly annoying! Tried with unpairing and pairing the earbuds, without result. Anyone knows how to get this resolved? Thanks!