I don’t think you can judge the Android kernel by its version:
1 Like
Not by its version alone 
But the older the base version is, the less likely it is to see back-ports of important recent upstream work.
Simply put, every new version of the upstream kernel changes a large amount of files for reasons of various significance. Back-porting tends to only happen of those changes that are greatly significant to make sure effort isn’t wasted. But those significant changes do build on prior work. Statistically, the bigger the gap between upstream and downstream kernels, the more of those insignificant changes must be considered for a significant back-port. This increases the required effort for the back-port as a whole, skewing the investment/returns balance towards not back-porting.
Now from experience the Android devs always prioritise security over effort, which is extremely praise-worthy. But for improvements of recent kernels in, say, battery life (like this one for instance - although effects on ARM platforms have not been measured), it is unlikely to see a back-port to the 3.4 kernel because the scheduler has seen too many other changes between 3.4 and 4.17. The work involved would be great, and with limited dev time/budget and a focus on newer SoCs for economical reasons this kind of work easily drops to the bottom of the list.
1 Like
I don’t see any reason they should invest time in updating (=major version) the kernel, as 3.4 works fine.
I am not really an expert in Android blobs either but as the current blobs work fine on Nougat without shims, it shouldn’t make a difference.
1 Like
I don’t really think so much exciting features are released for a kernel to be “must haves” for Android.
BTW, development of the AOSP kernel doesn’t work like that. They do not backport newer kernel changes to the old AOSP Linux codebase, but the other way around. When they start a new version of AOSP, they (if they consider it) fork the latest longterm release (i.e. not 4.17) of the Linux kernel and apply needed patches from AOSP on top. And after that, they only patch the kernel for security updates (remember it’s a longterm release). So no exciting features to be costly backported, in fact.
But there sure are a lot of “would-like-to-haves” that we miss out on thanks to the current development model. The Linaro folks tweak the heck out of the upstream kernel. Some of that might be “forward-ports” (for lack of a better term) from AOSP (“downstream”) kernels, but in other cases AOSP simply misses out on improvements until it’s based on the next LTS release. There’s been a lot of scheduler work upstream recently, which does not come from the AOSP world as it’s code shared across all architectures, and which has real impact on performance and battery life of multi-core processors.
Because it’s too costly, for reasons that I described…
There’s nothing “the other way around” about what you describe, it’s the same thing but from the perspective of the SoC vendor.
As I said, but with another layer of indirection. These security updates are generally back-ports from the current upstream (a.k.a. “mainline”) kernel to the LTS release, whoever performed them. To incorporate these changes in the downstream (AOSP) kernel, will then have to rebase on top of this LTS and pray their AOSP-patches still apply - fairly likely for LTS releases - or manually cherry-pick (back-port) from the LTS branch into theirs. This process gets too tedious when trying to keep up with upstream.
Realising who I’m replying to here 
I’ll go one step further and demonstrate why this is an undesirable model. Take for example a look at CVE-2015-9016, a kernel bug for which a fix was published by Canonical for upstream kernel 4.3 in August 2015. This patch wasn’t back-ported to 3.18 kernels until November 2017 and didn’t appear in the Android security bulletin until Feb 2018. Surely if all the drivers and device trees in that AOSP release didn’t live downstream but upstream, it would have been dirt cheap to rebase to a 4.3 release and have this security issue mitigated two years before people even realised it was a security issue. Who knows what other important but seemingly irrelevant bug fixes live upstream but not in the Qualcomm 3.4 fork… That is why you care about the base version of that AOSP kernel.
1 Like
Are you a developer? Do you know the Git version control system? (I bet you do, based on your words) What you described is a merge. What I described is a rebase of some sort. It’s effectively the other way around, re-applying the changes of your fork on top of the new codebase. It’s less costly. But of course it’s not free.
They cherry-pick, not rebase. But their patches will apply because it’s a longterm kernel, i.e. it doesn’t have brutal changes precisely because it’s made for this kind of support.
Android bussiness model is based on being forked and independently developed by OEMs which will develop their proprietary HAL drivers that depend on those changes made to the kernel by Android. (Note: of course they’ll earn money by including the proprietary GMS, but it’s not the topic). Can you see living in the edge is impractical?
I don’t like this proprietary-dependant model —I like a model based on free software, and that’s why I purchased a Librem 5—, but when your code is crucial for an insanely number of parties… better be stable. It’s the vendor responsibility to update to newer Android versions/apply Android security bulletin, so make their lives easy or they won’t do it.
By the way, Android’s Linux kernel could not be completely upstreamed (and that’s good and one of the bases of the open source development model). Some Android features are not meant for general use, just for Android or mobile devices and couldn’t live upstream, like wakelocks (IIRC, Linus Torvalds soid some naughty things about that feature). You’re right in that they could work closely with upstream developers to improve the code of some features and upstreaming it (like Linaro does), but it’s Google and a fast-paced industry, 
2 Likes
A kernel dev in fact! If you forgive me, I will skip over the explanations of merge and rebase. I feel we both have an understanding of the mechanics, but where my knowledge largely comes from the upstream world (I’ve worked for ARM for a small while, met some of their upstream kernel devs. Nice bunch! :-)) you seem to be more immersed in the Android world. Hopefully we can complement our knowledge over a beer some time @ FOSDEM 
This is certainly the status quo, but I think most SoC vendors hide behind this argument mainly because it’s the quick(-and-dirty) way. Because really, what is there to hide in a driver for your pin muxes, power management controller or SD card reader. It’s all mindless bit-banging. They just prefer copy-pasta and hard-coding parameters over engineering versioning schemes and topology descriptions in device trees. Even for the complex devices like GPUs, AMD and Intel show there’s no compelling business reason why a driver needs to be closed source (although firmwares may still be). The obvious downside is lack of quality assurance, as code doesn’t pass by a few layers of kernel maintainers.
Interestingly, collaboration actually led to inclusion of wakelock functionality in Linux kernel 3.5! Not quite the same as the Android kernel had them before that time, but better integrated in existing Linux infrastructure.
3 Likes
I’m just a merely observer, though. It’ll be great to share a talk and knowledge with some beberage in between, in fact, Although I’m yet to go to FOSDEM for the first time, 
(I’m the multidisciplinar kind of person, not just a developer).
Wow, such a curriculum, that of yours! I can now understand why you’d love to have some features you’re absolutely familiar with because of your closenest to kernel codebase.
I completely agree will all that paragraph word by word. Although there are (few) exceptions, right? Sony has (had?) its AOSP mainlining project. Do you think updating the kernel more aggresively with each Android version could improve overall quality?
Woah! That was an unexpected turn of events! Hahaha. Of course, the improvements were expected, because it’s normal that most downstream features are not well-integrated and need re-work.
2 Likes
I’ve made an attempt at explaining my arguments here. But a TL;DR: I think focussing on upstream development would force SoC vendors to think about code quality and quality assurance, because ultimately their drivers will be reviewed by several outsiders on their way to the mainline kernel. That’s not going to catch every bug, but there’s been quite a few monstrous holes introduced by certain vendors that would definitely have not made it to mainline.
Furthermore I believe that the current large fragmentation in kernels (as every SoC and every consumer device has its own “fork” effectively) makes for a lot of duplicated effort when trying to fix bugs and plug security leaks. Now it’s inevitable that every consumer device will end up with its own kernel binary, as you don’t want to end up shipping all device drivers to all kernels like a desktop OS does (that’s hundreds of megabytes of wasted space!), but the smaller the list of vendor-specific changes that go on top of the fork, the easier a rebase or a merge will be. If vendors of SoCs can get their systems working by taking an upstream or LTS kernel and creating a device tree + kernel configuration (to select which drivers should and should not be compiled along), without touching any of the kernel or driver code, they can grab whichever kernel they want and make it work without breaking a sweat. The initial investment could be a little bigger because now SoC manufacturers are forced to think about the design of their drivers, but on the middle-to-long run I can only see benefits in this for vendors and end-users.
3 Likes
Aaaah, so you are the same guy that made that thread time ago! I’m absolutely terrible at remembering names and nicknames (please-put-on-an-avatar).
I agree with all of that. And reading the linked post, you brought up a really good idea: Fairphone being a vendor offering a almost-mainlined FP Open and trying to push forward the industry for security and durability. That’d be awesome, in fact, although I don’t know if they have the engineering power now. I remember @keesj (former engineer at Fairphone) at least considering the option in the past to use mainline drivers for the screen. He may be interested in giving a helping hand to the work @z3ntu already did upstream, 
.
2 Likes
Unfortunately, I suspect that given the allocated engineering time for the Android 7 upgrade already came at the cost of regular monthly updates, I don’t think Fairphone is currently equipped for that job. Just to clarify: that’s an observation, not an accusation. They’re doing so many awesome things that I can’t blame them for not chasing my pet peeve 
. Equally I currently don’t have the time to chase this up myself as it’s hard enough to balance a PhD with a healthy and social lifestyle. So all I can do is encourage OSS enthousiasts those who do have the time and help them find their way through the kernel and the upstream community (Qualcomm has a team of upstream kernel developers, Red Hat has the mad hatter who reverse engineered the Adreno graphics and implemented a driver… the situation isn’t all bad!).
2 Likes
Hello !
That is a think that I fear as well, that due to the old kernel FP2 will miss some opportunities for new functionalities…
One example is Anbox, an Android emulator for Linux-based operating systems: UBPorts decided to include it on their system, to be able to run proprietary apps and get more users to Ubuntu Touch. For the moment it runs only on non-Qualcomm devices, due to the need of kernel patch…
Same thing for Anbox on SailfishOS, according to @mal:
My question is: what is the status of the mainline kernel fork 4.12-rc5 by @z3ntu? Is it working enough to be used for an alternative OS like UbuntuTouch or SailfishOS, or does it requires so much work that it’ll be easier to backport patches to the 3.4 kernel?
That fork (and the portion that can be found in the upstream 4.17 kernel) is far away from usability at this point. There’s many drivers missing or unfinished, including display, GPU, DSP and 4G modem. It’s impressive work, but from a users POV only the first few baby-steps towards a functional upstream kernel.
That being said, anyone comfortable enough to play with device trees and fairly simple drivers can probably get more stuff up-and-running in a matter of days. There is a functional driver for the GPU, and a WIP for the display driver already. If it gets to a point where the only missing features are the modem and DSP (probably requiring proprietary drivers for proper functionality as especially the modem is a patent minefield), rebasing FP(O)OS to it becomes a lot easier.
3 Likes
Display could work better with the “simplefb” driver other msm8974 devices are using. About GPU & modem: I should be able to add support for that after efct18 as it works already on other msm8974 devices.
6 Likes
That’s… more impressive than I thought! 
Do you know how well the DVFS functionality compares to the Android kernels? I’m sure all clocks (CPU, L2, GPU etc…) are supported, but how do the policies compare in terms of battery life/power consumption?
1 Like
With help of another developer I made some progress with Anbox on FP2 Sailfish OS using only one additional patch instead of the previously estimated 30-40 patches. I managed to get the Anbox Android running in the container but couldn’t get graphics working yet. I’ll have to look into that again when I have time.
8 Likes
That’s good news, @mal 
!
Do you think your work can beneficiate the UBPorts team? Do you have a public repository for your patched kernel?
Wow @mal that is really great news! So there is hope that in a not too far away future Anbox will be usable on SFOS? That would make the FP2 my ideal device!
P.S.: really a pity that we can’t honour your efforts besides saying thank you in the forum
1 Like
The patches I used are on my github project but I think I should make sure those are latest versions and improve the commit messages (5 latest commits in https://github.com/mlehtima/android_kernel_fairphone_fp2/commits/anbox-new).
2 Likes
This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.