I’m pretty sure in the case of the FP4 with it’s stereo output it would be fine for earbuds, not that this is meant to be an argument against removing the 3.5 jack.
To those that are wondering about sustainability rather than the technical issues:
Fairphone, from what I can tell, made the phone more sustainable i.e. better water and dust resistant by removing the jack; that’s their business.