I sounds a bit like an issue from the early FP3 days. Until it was fixed, it could be that the phone didn’t properly react to the 3.5mm headphones being removed. The symptom was the other way round: the phone loudspeakers did not work anymore as the phone “was convinced” the headphones were still connected.
Back then, the workaround was to reboot the phone if that problem occurred after unplugging one’s headphone.
So maybe a reboot helps here, too, until the problem is resolved properly?