I am having the same problem. As far as I know it worked perfectly fine until I updated to 1.7.1 (I was at something way earlier, 1.3 or something similar). If it was hard to understand me before, people just didn't tell me. But after the update people didn't understand me at all anymore when I talked to them, unless of course I put the phone on speaker mode.
Reading through this thread I tried some of the things suggested. I poked a little in the mic on top and tried to clean it. I did test calls with the phone and I was able to hear myself again after that, but there was a lot of crackling and background noise, especially when I properly put the phone next to my ear and my hair were in the way as well.
After that I took out the top module first, to maybe be able to clean it more, since it had worked at first. In the core module I poked through the little hole above the microphone. On the inside it is covered by a small black tissue, but mine was loose on one side (maybe because I had poked at it before) and seemed to be misplaced a little. I am not sure it covered the entire hole.
The actual micro looked clean and fine.
Then I also took out the bottom module, but I couldn't find much that I could have done other than blow in the hole again, so I put it back in.
Restarting the phone and calling myself again there were noticeable differences, however it got worse instead of better: on normal phone mode I couldn't hear myself at all again, and on speaker I could hear myself only faintly with a lot of crackling. On the fairphone side however, there was a loud background noise now. It wasn't crackling, but more a constant humming of sorts.
My flatmate has a fairphone as well. I want to try to export her version of the dialer app and use it on my phone, because I am still a little hung up on the fact that I only noticed a difference after I updated. She still has an earlier version, 1.4 I think, so I want to try that.