I noticed similar symptoms last week on my Fairphone 5 (with /e/OS), while trying to use a brand new Yubikey. While it worked perfectly with NFC, or on a USB-C port of my computer, it never worked properly on my Fairphone’s USB port. Either the LED flash one time for less that a second, or the LED blinks for 4 series of 4 flashes, which looks a lot like an error code to me.
While trying to understand what was wrong, I ended up concluding that the Yubikey is fine, but the Fairphone is not. I noticed that USB tethering is not working anymore (despite it working a few months ago). And it seem to be working weird when connected to computers : sometime it works fine, sometime it goes like charging-unplugged-charging-unplugged-charging-unplugged in a very fast succession for like 3 seconds, then stops detecting the computer and does not charge.
The USB port itself seemed OK to me, but I performed a thorough cleanup, which did not solve anything. Reading some topics here, I purchased a new USB port and swapped the old one with a new one. Nothing changed.
At this point, I don’t have any idea on how to fix the issue. I’m reluctant to call for Fairphone’s support because I have had a bad experience with it, but if this a hardware problem, I might not have a choice. Any idea ?