In other words: Android can run without meeting all requirements, but if you want to ship a phone with Google services, you’ll need to fulfill the requirements. As has been discussed here often, shipping a phone without Google services is a no-go for the vast majority of Fairphone’s customers.
Splitting the OS into two branches - one on Android 6 and one on Android 7/8/9/…, is something that would require too much time from the devs, so won’t happen. But even if the devs had infinite time to spend, there are more issues surrounding the proprietary parts of the OS. Here’s my take on what’s going on (which is likely to not be 100% accurate)
These are the bits of code that allow the operating system to talk to the hardware in the phone, and are supplied by the platform vendor in machine-readable form (i.e. not as modifiable source code). Which brings us to:
It isn’t clear what access Fairphone themselves have to the underlying code, but in most cases phone builders will only get the binaries. It took years negotiate access to the code underlying the FP1 binaries (only when the chipset was way out of date did FP get access in way that they cannot release the code itself), and even then updating the code to a newer version wasn’t successful. Updating to a newer version may require changes in the binaries that will need to be implemented by the chipset vendor (or access to the source code). If the chipset is missing features that Google deems necessary for newer version of Android, it becomes less likely that the chipset vendor will put in the effort to provide the updates. Same goes for if you have a smaller number of devices.
The binary licensing issue is one that ended being discussed on the forum in terms of what it meant for custom mods (see here). In short, Fairphone does redistribute them, but only for non-commercial use on a Fairphone 2.
Why does that matter? Extracting and using binaries a bit of a grey area, and legal won’t like it - let alone that you risk the wrath of the platform vendor that may stop official support and/or no longer sell you chips. And then we haven’t even mentioned distributing anything you don’t have express permission for. If you don’t buy chips and don’t sell your software or devices that run your software (like LineageOS) this is less of an issue. So some lineage ports are adapted around the binaries originally shipped with the phone, some will use binaries from other devices with a comparable chipset. In some cases this means that you can get features to work for which licensing wasn’t in place in the first place (example), which lineageOS themselves don’t directly do (activated only for devices for which support was present in stock OS).
All in all it becomes too risky/too much effort in a legal, development, and support sense (there’s no chipset vendor support for testing, so more chances you break some rare use case, and getting stuff fixed - think also about stuff like meltdown patches) for a company to take it on. A community-based effort is a different thing - the risks and responsibilities stay with the users.