Project Treble (Android O)

Stefan · May 31, 2017, 9:29pm

Browsing Ars Technica I found this article about Project Treble:

Apparently Android O includes the outcome of Project Treble, which, in essence, makes Android upgrades independent from the chipset manufacturer. If we assume that Fairphone releases Android upgrades as long as they can, this means that a Fairphone with Android O can get all the other flavours that follow. (This is a difference to other OEMs, which modify Stock Android.)

In my opinion this is a great development and it will save future Fairphones from the problems FP1 is facing and FP2 might face if Qualcomm refuses to update the Snapdragon 801.

erotavlas · June 11, 2017, 8:05am

Android O or 8 will bring the huge change since the beginning of Android in term of lifespan and upgradability of the OS. The project is called treble:

Project Treble aims to do what CTS did for apps, for the Android OS framework.
The core concept is to separate the vendor implementation — the
device-specific, lower-level software written in large part by the silicon
manufacturers — from the Android OS Framework

So for every device, it will be simpler to receive an update since it will not necessary new drivers or kernel support.

RSpliet · June 11, 2017, 9:09pm

Bear in mind though that 90% of all security fixes for Android as published in the security bulletin are in this “device-specific lower-level software” (aka the kernel + drivers). As soon as the SoC vendor decides to stop updating the kernel, your Android device is unsafe.

erotavlas · June 12, 2017, 7:36am

You are right (even if the ratio is half of 90%), but separating the low level software (drivers) from the OS framework is great deal since after this every device could receive update of the OS.
Moreover, as describe here, android OS is made of several components (android framework, android runtime, native libraries, HAL and linux kernel). With project treble, it will be possible to update all components except the kernel by updating the OS framework.

Finally, updating the linux kernel is not a job of the android OS itself since the kernel is a single separated component and the majority of driver are closed source. Hence the responsibility is only in the hand of the SoC vendor. What we need is an open source drivers.

paulakreuzer · June 12, 2017, 7:46am

I merged above posts into a new topic to keep all Project Treble discussion in one place.

I haven’t made up my mind about whether this is a good thing yet (that’s why I hope for an interesting discussion), but generally when a headline reads “Google’s Project XY solves…” I rather believe it removes a barrier for Google to get an even tighter grasp on their users then solving an actual issue for the users. Please convince me otherwise!

RSpliet · June 12, 2017, 10:08am

This is a misconception: most drivers tend to be open-source. Take a look in the Fairphone Open OS kernel repository and look at the Qualcomm components for yourself. NVIDIA Tegra has their work here, including the kernel side of the GPU driver, an old Allwinner kernel with many drivers etc. etc. The trouble exists mostly with GPU and baseband drivers, where the kernel driver is developed in lock-step with firmware and/or user-space components. Vendors don’t keep the kernel<->user-space interface (API/ABI) stable between driver releases, which causes several problems

The upstream kernel folks are unhappy to accept drivers for which the API hasn’t been thought out, because they don’t want a kernel update to break a system. SoC vendors tend to not care, because android lets them control the user-space/kernel combination.
For new SoC support, vendors tend to copy-paste the old driver, hack it up a bit and call it a day. There’s no thought being poured into driver architecture to support multiple revisions of hardware, incremental feature addition and so forth. This means that when you try to rebase to a new kernel, they need to update a kernel tree for every SoC rather than keep their one well-designed driver in check.
The code they ship as drivers doesn’t go through the layers of independent code quality control put in place by upstream kernel maintainers. This is a source of trivial bugs and problems, especially for the low-budget SoC vendors.

Oh yes, and because the kernel internals do change, you often can’t just copy-paste these open-source drivers into a newer kernel tree and get them running, even if it remains compatible with user-space. In contrast, drivers that live upstream get updated by the person proposing the internal-kernel change, which would give SoC vendors the luxury of compatibility at a low cost.

Qualcomm is doing quite a good job at developing clean upstream open-source drivers. For most devices in the Fairphone’s SoC there already is an upstream open-source driver. This work hasn’t poured back into the Android stratosphere though. I’d like to see that happen

aljoscha · June 12, 2017, 2:02pm

So, Qualcomm is providing drivers for most of the hardware of our phone, but they are integrated in the ‘normal’ Linux kernel and not the Android fork, is that correct? Would that include the gpu driver and wouldn’t it be possible to backport it for the Fairphone team or something like this?

Questions from a layman…

RSpliet · June 12, 2017, 2:52pm

Correct! … although I presume they are keen to start using many of these drivers when Android versions on top of new kernels start seeing the light.

Well, a GPU driver. Dubbed freedreno. Quite a cool one, supports OpenGL better than the closed-source one, and performance is mostly on-par with the closed-source driver too if I recall correctly. No OpenCL or Vulkan for now though… If you happen to be a talented developer, contributions welcome!

[quote]and wouldn’t it be possible to backport it for the Fairphone team or something like this?

Questions from a layman…
[/quote]
You mean backporting to an Android kernel? Technically there’s little to prevent this from happening, and the main developer of this driver works with 3.10/4.4 kernels used by most Android Marshmellow distributions (Fairphone seems to be stuck with 3.4 for now unfortunately) as well as with the upstream kernel. However, freedreno was written for the drm framework (don’t worry, it’s nothing to do with digital rights management), while Android 6 expects a different framework behind their Hardware Abstraction Layer. Because of this, I’m not sure whether Android 6 would play nice with Freedreno. Collabora has worked on trying to make this work. I’d recommend you ask the experts on Freenode IRC in channel ##linux-msm for the current status and roadmap.

erotavlas · June 12, 2017, 9:10pm

This is a misconception: most drivers tend to be open-source. Take a look in the Fairphone Open OS kernel repository and look at the Qualcomm components for yourself. NVIDIA Tegra has their work here, including the kernel side of the GPU driver, an old Allwinner kernel with many drivers etc. etc.
The trouble exists mostly with GPU and baseband drivers, where the kernel driver is developed in lock-step with firmware and/or user-space components. Vendors don’t keep the kernel<->user-space interface (API/ABI) stable between driver releases, which causes several problems

You are right, I was not precise. There are a lot of open source drivers: unfortunately most of them are integrated in the mainline of the linux kernel, but not in the android linux kernel. So in practice, it is like if they would not exist.

Qualcomm is doing quite a good job at developing clean upstream open-source drivers. For most devices in the Fairphone’s SoC there already is an upstream open-source driver.
This work hasn’t poured back into the Android stratosphere though. I’d like to see that happen

My question is why android linux kernel and mainline linux kernel are not the same thing?
Latest android linux kernel is v4.9

Edit: I found the answer directly from an android linux kernel developer:

The interesting thing about Android’s design is how little we modified the kernel. Most embedded systems on which I have worked have made drastic changes to the kernel, only to leave user-space alone—for example, a heavily-modified “realtime” kernel but X11 for a GUI.

Android is the opposite: only minimal changes to the kernel, but a user-space wholly unlike that of any other Unix system. In fact, Android’s user-space is so different from stock Linux, you can easily say that Android is not in any way a Linux system, except for the kernel.

RSpliet · June 12, 2017, 9:09pm

This is all a lot of “finger in the air” talk, but I reckon for GPUs there’s a historic reason. The Direct Rendering Management (DRM) framework, the current framework for open-source GPU drivers, was conceived as a replacement for user-space desktop drivers. Thus its design was geared towards use by X.org and devices that do 2D/3D rendering and display in the same hardware. In mobile SoCs, these tasks tend to be split between various components. Not to mention X.org being way too clunky for mobile use. I bet that during the start of Android development, trying to fix these problems for SoCs in existing software was too big a task. iPhone was already on the market, so it had to be done under a lot of time pressure. Instead, they built something simpler. Obviously, because it was trying to perform the same task as DRM, upstream would never accept this code. Hence Google maintained and developed it themselves.

ben · August 21, 2017, 7:17pm

Edit: Quite short (and boring) stream. But we could take the opportunity to discuss Tremble.

I look forward to more information on Treble, which should improve the update situation on future devices.

Douwe · August 21, 2017, 6:54pm

So Oreo it is!

Still some issues over at Google: https://www.android.com/oreo

ben · August 21, 2017, 6:58pm

This works:

My Highscore was 202.

Stefan · August 21, 2017, 9:01pm

I have to admit, the solar eclipse reference is not the worst idea they’ve had…

Edit:

Where did you find a game??? I’m clicking on the version number, but nothing happens…

My personal highlights from the “changelog”:

Install unknown apps: Hostile downloader apps can’t operate without permission; users now permit the installation of APKs per-source.

Will it finally be possible to whitelist only F-Droid for “unknown sources”?

Notification snoozing: Lets users hide notifications for a period of time, similar to Inbox snoozing.

Sounds like Peace of Mind…

Wi-Fi Assistant: Auto-connects you to high quality open WiFi and secures your connection with a VPN back to Google.

Yay. Read: …and routes all your traffic via Google servers so that we know all the websites that you visit, even when you aren’t using Chrome.

ben · August 22, 2017, 10:52am

Wifi Assistant is optional, as far as I know.

Roboe · August 22, 2017, 11:39am

Not only websites, but servers, all internet communications. An VPN routes all internet traffic, not only web traffic (the Web being a set of protocol(s) [HTTP/S] over Internet protocols and architecture [TCP/IP, SSL, DNS]).

Stefan · August 22, 2017, 9:14pm

I bet it is opt-out though…!

oriold · August 23, 2017, 12:34am

And step by step the pace of new versions in Android is faster. It will be harder very soon to keep an Android phone, at least patched and supported. It is really upsetting.

ben · August 23, 2017, 5:03am

Hmm, where do you get that conclusion from? Looking at the table in your link, before Jelly Bean, it’s often been only about 10 months between the releases, then 15month between Jelly Bean and KitKat. Since then, it’s been about a year between each release, with the release shifted from October to August with Nougat.

I’m sure Google settles with the industry standard of predictable release dates (actually set by Ubuntu, then quickly followed by Apple) of one mayor release per year and I bet it will around August next year again.

This gives developers (of apps) and manufacturers (of devices) as well as chipset manufacturers predictability and allows planning. This could help improving the update situation.

oriold · August 23, 2017, 3:55pm

Have you read why Fairphone 2 won’t likely have Nougat? The reality is that Qualcomm doesn’t want to bother with the updates. You are supposed to change mobile phone every year. So while for the vast majority of the world is acceptable, us fairphone users is a matter of concern.

So right now KitKat is somewhat supported, but no manufacturer pushing updates, so basically is dead. In my opinion Android is on the road to be the next IE6. Massive adoption, full of bugs. Google as long as it gets our data they don’t seem to look forward long term usability of the devices. I think it will backfire eventually.

By accelerating the pace of releases, the usability of your device decreases, as manufacturers don’t like to bother with the updates, just sell you the stuff. Also, Google Apps are entangling more and more within the system. For us with FOSS Android as OS, it will become harder and harder to use Android without Google. Check safetynet, they are already blocking some apps, that won’t be available on custom roms, rooted or de-googled androids.