I’ve made an attempt at explaining my arguments here. But a TL;DR: I think focussing on upstream development would force SoC vendors to think about code quality and quality assurance, because ultimately their drivers will be reviewed by several outsiders on their way to the mainline kernel. That’s not going to catch every bug, but there’s been quite a few monstrous holes introduced by certain vendors that would definitely have not made it to mainline.
Furthermore I believe that the current large fragmentation in kernels (as every SoC and every consumer device has its own “fork” effectively) makes for a lot of duplicated effort when trying to fix bugs and plug security leaks. Now it’s inevitable that every consumer device will end up with its own kernel binary, as you don’t want to end up shipping all device drivers to all kernels like a desktop OS does (that’s hundreds of megabytes of wasted space!), but the smaller the list of vendor-specific changes that go on top of the fork, the easier a rebase or a merge will be. If vendors of SoCs can get their systems working by taking an upstream or LTS kernel and creating a device tree + kernel configuration (to select which drivers should and should not be compiled along), without touching any of the kernel or driver code, they can grab whichever kernel they want and make it work without breaking a sweat. The initial investment could be a little bigger because now SoC manufacturers are forced to think about the design of their drivers, but on the middle-to-long run I can only see benefits in this for vendors and end-users.