I had the same issue on CalyxOS and finally managed to get it working a few days ago.
Duolingo uses Google’s TTS service for speech recognition, haven’t been able to get it to detect anything else.
You need to reboot the phone after installing it and manually give the Google TTS app microphone permissions because it doesn’t ask for them (I was missing that part).
Of course running Google speech recognition isn’t ideal so I have all that stuff relegated to the work profile and just turn that off after use, I can live with that.