✏ Porting TWRP recovery

This is a wiki post. Everyone can improve it, by clicking the green pencil above.

This topic was originally about porting TWRP to the Fairphone 2. Reference information on how to use TWRP on a FP2 has been moved into this topic: Using TWRP on the Fairphone 2.

The post below does not contain up-to-date information


Today I started looking into TWRP recovery and made good progress. I want to share my experience, patches etc. here with you. Maybe there are some interested people (I know there are :wink:) who wants to collaborate, test, give hints etc.


:information_source: If you just want TWRP on your phone and you are ok with not having Google Services preinstalled, simply download and install FP Open OS. This version includes an older TWRP version (2.8.7.0) without support for an encrypted data partition.

:information_source: If you are looking for the easiest way to flash a more recent compiled TWRP recovery.img to your rooted FP2, go to Installation guide for the compiled recovery.img

:information_source: If you are looking for the procedure to install an update of FP-OS after once having a more recent TWRP installed (+root access), scroll down under “Update Procedure”,


Links

So lets start with the links to get good information from:

Adding repositories

Okay, so here we go. When going to the Howto on XDA-Developer, we need to replace bootable/recovery. So I added a .repo/local_manifests/local_manifest.xml file:

<?xml version="1.0" encoding="UTF-8"?>
    <manifest>
        <remote name="github" fetch="https://github.com"  />

        <remove-project name="fp2-dev/platform/bootable/recovery" />
        <project path="bootable/recovery" name="omnirom/android_bootable_recovery" clone-depth="1" remote="github" revision="android-5.1" />
        <project path="external/busybox" name="omnirom/android_external_busybox" clone-depth="1" remote="github" revision="android-5.1" />
</manifest>

As you can see here. Busybox will also be required. You may than remove the old bootable/recovery first to get it synced. Repo will show you the message.

Device configuration

Next I edited device/fairphone_devices/FP2/BoardConfig.mk and added these options at the end:

# TWRP Recovery
TW_THEME := portrait_hdpi
RECOVERY_GRAPHICS_USE_LINELENGTH := true
TW_NO_USB_STORAGE := true
TW_TARGET_USES_QCOM_BSP := true

I just copied the settings from the Sony device and went through the descriptions in the guide. These were the only ones that made sense to me for a first try :smile:
TW_TARGET_USES_QCOM_BSP is required to get the blue colors (instead of orange).

Next thing is the fstab. I added device/fairphone_devices/FP2/twrp.fstab with this:

# mount point   fstype  device                  [device2]   fstype2

/boot           emmc    /dev/block/platform/msm_sdcc.1/by-name/boot
/system         ext4    /dev/block/platform/msm_sdcc.1/by-name/system
/cache          ext4    /dev/block/platform/msm_sdcc.1/by-name/cache
/data           ext4    /dev/block/platform/msm_sdcc.1/by-name/userdata  length=-16384
/external_sd    vfat    /dev/block/mmcblk1p1    /dev/block/mmcblk1  flags=display="Micro SDcard";storage;wipeingui;removable
/recovery       emmc    /dev/block/platform/msm_sdcc.1/by-name/recovery
/misc           emmc    /dev/block/platform/msm_sdcc.1/by-name/misc
/bootselect     emmc    /dev/block/platform/msm_sdcc.1/by-name/bootselect

Modifications

Makefiles

These are the required changes in the build/core/Makefile:

project build/
diff --git a/core/Makefile b/core/Makefile
index fceb6db..ffbc229 100644
--- a/core/Makefile
+++ b/core/Makefile
@@ -854,7 +854,7 @@ $(INSTALLED_RECOVERYIMAGE_TARGET): $(MKBOOTFS) $(MKBOOTIMG) $(MINIGZIP) $(RECOVE
                $(recovery_fstab) \
                $(RECOVERY_INSTALL_OTA_KEYS)
        @echo ----- Making recovery image ------
-       $(hide) rm -rf $(TARGET_RECOVERY_OUT)
+#      $(hide) rm -rf $(TARGET_RECOVERY_OUT)
        $(hide) mkdir -p $(TARGET_RECOVERY_OUT)
        $(hide) mkdir -p $(TARGET_RECOVERY_ROOT_OUT)/etc $(TARGET_RECOVERY_ROOT_OUT)/tmp
        @echo Copying baseline ramdisk...
@@ -874,6 +874,7 @@ $(INSTALLED_RECOVERYIMAGE_TARGET): $(MKBOOTFS) $(MKBOOTIMG) $(MINIGZIP) $(RECOVE
          cp -rf $(item) $(TARGET_RECOVERY_ROOT_OUT)/)
        $(hide) $(foreach item,$(recovery_fstab), \
          cp -f $(item) $(TARGET_RECOVERY_ROOT_OUT)/etc/recovery.fstab)
+       cp -f device/fairphone_devices/FP2/twrp.fstab $(TARGET_RECOVERY_ROOT_OUT)/etc/
        $(hide) cp $(RECOVERY_INSTALL_OTA_KEYS) $(TARGET_RECOVERY_ROOT_OUT)/res/keys
        $(hide) cat $(INSTALLED_DEFAULT_PROP_TARGET) $(recovery_build_prop) \
                > $(TARGET_RECOVERY_ROOT_OUT)/default.prop

If recovery output is removed before creating it, busybox, recovery etc is removed again. So I removed this line. This isn’t also available in omnirom repository.
The seconds change is a dirty hack. I don’t know how to copy the twrp.fstab in the best way. I’ve tried with PRODUCT_COPY_FILES in FP2.mk, but didn’t work. Maybe someone can find that out.

SELinux

Of course there are also some SELinux issues (would be boring otherwise :smiling_imp:). I could fix one with this modification:

project bootable/recovery/
diff --git a/etc/init.rc b/etc/init.rc
index 4e7b845..cfb3c39 100644
--- a/etc/init.rc
+++ b/etc/init.rc
@@ -88,6 +88,7 @@ service healthd /sbin/healthd -r
     seclabel u:r:healthd:s0
 
 service recovery /sbin/recovery
+    seclabel u:r:recovery:s0
 
 service adbd /sbin/adbd --root_seclabel=u:r:su:s0 --device_banner=recovery
     disabled

Policies

Omnirom doesn’t have this, but I needed it to get a first working version.
I also made these changes, but I’m not sure if they’re really required:

project external/sepolicy/
diff --git a/recovery.te b/recovery.te
index 821da01..89d7d98 100644
--- a/recovery.te
+++ b/recovery.te
@@ -7,7 +7,10 @@ type recovery, domain;
 # But the allow rules are only included in the recovery policy.
 # Otherwise recovery is only allowed the domain rules.
 recovery_only(`
-  allow recovery self:capability { chown dac_override fowner fsetid setfcap setuid setgid sys_admin sys_tty_config };
+  # Allow recovery to set permissive mode
+  permissive recovery;
+
+  allow recovery self:capability { chown dac_override fowner fsetid setfcap setuid setgid sys_admin sys_tty_config sys_time };
 
   # Set security contexts on files that are not known to the loaded policy.
   allow recovery self:capability2 mac_admin;
@@ -22,6 +25,10 @@ recovery_only(`
   allow recovery unlabeled:filesystem ~relabelto;
   allow recovery contextmount_type:filesystem relabelto;
 
+  allow recovery selinuxfs:file rw_file_perms;
+  allow recovery rootfs:dir { add_name write remove_name };
+  allow recovery rootfs:{file lnk_file} { create_file_perms relabelfrom relabelto };
+
   # Create and relabel files and directories under /system.
   allow recovery exec_type:{ file lnk_file } { create_file_perms relabelfrom relabelto };
   allow recovery system_file:{ file lnk_file } { create_file_perms relabelfrom relabelto };

diff --git a/service_contexts b/service_contexts
index 8fe6972..05f5460 100644
--- a/service_contexts
+++ b/service_contexts
@@ -88,6 +88,7 @@ procstats                                 u:object_r:system_server_service:s0
 radio.phonesubinfo                        u:object_r:radio_service:s0
 radio.phone                               u:object_r:radio_service:s0
 radio.sms                                 u:object_r:radio_service:s0
+recovery                                  u:object_r:system_server_service:s0
 restrictions                              u:object_r:system_server_service:s0
 rttmanager                                u:object_r:system_server_service:s0
 samplingprofiler                          u:object_r:system_server_service:s0

Adding sys_time to capability made the clock working in emulator. Unfortunately not on device.

Kernel config

I added

CONFIG_KERNEL_LZMA=y

to the kernel config in kernel/arch/arm/configs/fairphone_defconfig to reduce size of the image. So we have a little space left, just in case.
I made the change with

make kernelconfig

what reorders config file a little bit.

Building

You can then run

make -j8 recoveryimage

Clean before and don’t apply the binary blobs. It should build without it. Furthermore with binary blobs my recovery.img was too big for flashing. And also we don’t want this here if possible. So maybe it can be distributed in future.

Testing

For testing I added a virtual device based on Nexus 5 in the emulator. Then I started it with the ramdisk-recovery.img:

emulator -avd AVD_for_Fairphone -ramdisk ramdisk-recovery.img -show-kernel

This gives the kernel output, e.g. to see the policy warnings/errors. There are still some remaining:

type=1400 audit(1453078388.150:4): avc:  denied  { read write } for  pid=35 comm="adbd" name="qemu_pipe" dev="tmpfs" ino=1919 scontext=u:r:adbd:s0 tcontext=u:object_r:device:s0 tclass=chr_file permissive=0
type=1400 audit(1453078388.170:5): avc:  denied  { write } for  pid=33 comm="toolbox" name="enforce" dev="selinuxfs" ino=4 scontext=u:r:recovery:s0 tcontext=u:object_r:selinuxfs:s0 tclass=file permissive=1
...

I’m not sure yet if they need to be solved. Maybe someone can look at them.

When it started in emulator, I flashed it to my phone. It booted up and looked good so far. I also was able to flash a zip file.

Beside from further testing, there are still some things which need to be resolved. E.g. clock shows wrong time (due to wrong timezone setting). Would be great if someone could help me to solve this :grinning:

Things I’ve successfully tested so far:

Known issues:

* Encrypted userdata (/data) can’t be accessed. This is most likely because of some vendor/hardware specific crypto features. Need to try this.

Versions

:exclamation: This is an experimental version. It runs on my phone and I did first tests, but nothing more. There are things which are not working yet and I can’t promise that it will not break anything. :exclamation:

Here is a first version for testing. And Here is the newest version (please update if newer version available). It can be flashed as usual with fastboot (see below for the installation guide):

Checksums of recovery.img (first version):
MD5: e6f229d4324fcdaaa79c2d325adbf59a
SHA256: 3032ea733319997a72ab521a11225783ffca5fd1d137e49d297a8b183a89544f

Checksum of recovery.img (newer version from @OuinOuin’s post):
MD5: 8a634c3be94136d6c4abb859057599c7

Latest Versions compiled by @z3ntu
Download links: https://twrp.me/fairphone/fairphone2.html
https://forum.fairphone.com/t/pencil2-porting-twrp-recovery/12558/175?u=sky
https://forum.fairphone.com/t/pencil2-porting-twrp-recovery/12558/170?u=sky

As I didn’t apply the binary blobs, there shouldn’t be any legal issues with that. If Fairphone has another opinion about that, I will remove the file immediately.

Installation guide for the compiled recovery.img

[:link: direct link](https://forum.fairphone.com/t/pencil2-porting-twrp-recovery/12558#installation) https://forum.fairphone.com/t/pencil2-porting-twrp-recovery/12558/176?u=sky

We are following similar procedure as for rooting, for which you have to install all software and drivers stated there under step 3a-c and continue as follows:

1. First of all to make things easy, download the pre-compiled TWRP recovery.img (see above under Versions and copy it directly to your
C:\ drive (C:\recovery.img)

2. Start up your minimal ADB and Fastboot by double clicking the
shortcut on your desktop/program folder. A command promt will appear. Your FP2 should be switched on and connected via USB.

3. First we will start the FP2 into Fastboot mode by entering:

adb reboot bootloader

Your phone should reboot but instead of starting normally it will just display the black Fairphone logo.
If you have errors: see here or here

4. Make sure, you still have connection to the phone by entering:

fastboot devices

You should get back I prompt without errors saying “”.
If you have errors: see here or here
If your phone displays a prompt to authorize your PC to connect via ADB, allow it.

5. Next we will flash the image to the FP2 by typing:

fastboot flash recovery c:\recovery.img

The flashing only takes some seconds and should not produce any errors.

6. At last we will restart our FP2 by typing:

fastboot reboot

Your phone should start normally and flashing is complete.

6.b Verify that TWRP was permanently installed:
Sometimes, when booting into the Android system normally right after flashing the TWRP recovery image, the system restores the previous default recovery image. Verify which recovery is active:

  • Switch off your FP2
  • Boot into recovery mode (by pressing VOLUME UP and ON in parallel until you see the recovery screen, this takes about ~10 seconds)

If you see the TWRP logo followed by a “graphical” user interface, everything worked fine.
If, on the other hand, you see a miniature text-based interface, saying something like “Android system recovery…” in the first line, you need to flash TWRP again and boot into recovery mode as the very first step right after flashing the image (instead of booting into Android system normally):

  • Flash the TWPR image again as described above
  • type fastboot reboot → but do not press ENTER yet!
  • press the VOLUME UP button on your FP2 (and keep it pressed)
  • hit ENTER key to execute reboot and take you directly into the TWRP recovery

After this, TWRP should be flashed permamantly.

7. If you want to continue to install XPosed Framework + GravityBox:

  • install XPosed Framework:

  • Switch off your FP2

  • boot into TWRP recovery (by pressing VOLUME UP and ON in parallel until you see the TWRP screen, this takes about ~10 seconds)

  • install the downloaded XPosed ZIP (the current is xposed-v86-sdk22-arm.zip)

  • To manage the modules you still need to install the “XPosed Installer” from the XDA-site (current version is XposedInstaller_3.0_alpha4.apk) (*)

  • You have to enable the XPosed Framework and restart your phone

  • After one of the reboots your Android maybe will show “Optimizing Apps # from #”.

  • If you want to install GravityBox, download the module within the XPosedInstaller and be happy to tweak your FP2

(*) In case this does not work for you out of the box, you may need to rename /system/vendor/lib/libart.so (and perhaps /system/vendor/lib/libart-compiler.so as well), e.g. by appending .bak to the file(s) (/system/vendor/lib/libart.so.bak). At least, this did the trick for me on a rooted FP Open OS.
Afterwards, I tried to install the XposedInstaller, Amaze told me it did not work, but on rebooting I saw the optimising apps message. I then "re"installed the XposedInstaller after the reboot and this worked.
But as @fp1_wo_sw_updates rightfully put it:

It looks like proprietary libs from qualcomm are in the way. On rooted phones running the google-free FP Open OS, people seem to have success by renaming /system/vendor/lib/libart.so and /system/vendor/lib/libart-compiler.so so the system uses the libs installed by the xposed installer. (This is just a guess, please [read the corresponding posts]! Don’t brick your phone! Also this will remove the optimized QC libs from the running system).

Update Procedure when FPOS OTA-Update is available

If the FP updater informs you via WLAN about an FPOS update, you have to do it MANUALLY with TWRP, if you flashed TWRP before!

Here are the procedure (including re-establish root acces, link for root infos):

  1. Download the FP2 ZIP as the “update” version (latest version can be found in Fairphone’s support article Manually install Fairphone OS for the Fairphone 2), save it on SD card or internal storage
  2. Start phone in TWRP (Power+VolUp > 10s) and install the image via ZIP
  3. If desired, without reboot also (re-)flash XPosed ZIP with TWRP (details)

I you want to (re-)gain superuser/root access flash newest version of boot.img, ([details]
(https://forum.fairphone.com/t/pencil2-howto-root-with-superuser/12375))

→ 4. Then reboot into fastboot mode (by choosing “Restart” and “bootloader” in TWRP
→ 5. Reboot (fastboot reboot) and let your phone “optimize apps” (since this takes quite a while)

Procedure tested for:

  • FP2 v1.? (original) → 1.2.8 (April 2016) having TWRP and root access via superuser before

History

  • 2016-01-20:
  • Fixed color of gui (orange → blue).
  • Removed note regarding Xposed (here for this topic).
  • Added screenshot.
  • Added list of tested things.
  • 2016-01-22:
  • Updated list of tested things
  • 2016-01-25:
  • Added known issue.
  • 2016-02-13
  • Updated the installation guide for the compiled recovery.img (@therob)
  • 2016-03-03
  • Updated the installation guide for the compiled recovery.img by adding a side note on the installation of the XposedInstaller on rooted FP Open OS. (@merci)
  • 2016-04-10
  • Added procedure to install FPOS updates, when TWRP is installed (@therob)
  • 2016-12-xx
  • Added description on (sometimes) necessary steps to have TWRP be installed permanently
  • 2017-03-04
  • Manual system updates: add link to Fairphone’s official support article for latest FPOS images
  • 2017-03-13
  • Versions: Add link to images of @z3ntu
  • Installation guide: Short hint for flashing images direct from TWRP
  • Known Issues: Delete data encryption issue
32 Likes

I don’t think this should be a wiki at this point. We should first move this to the dev category (as soon as it is possible for us), and then I’d agree to wiki this post. :smile:

Oh, yes. That should be in in the dev category. Can you move it? I’ve only access to my fp1 at the moment.

Hey there,
thank you very much for your effort! I think having a fully working TWRP recovery would be great!

Unfortunately I can not help you, since I am not a software developer :frowning:

Somehow related (I was searching for something “grub” like for Android) I saw this Multi ROM BootManager some time ago, and I like the idea. Maybe there are other projects like it?

“MultiROM is one-of-a-kind multi-boot mod for Nexus 7. It can boot any Android ROM as well as other systems like Ubuntu Touch, Plasma Active, Bohdi Linux or WebOS port. (…) The main part of MultiROM is a boot manager, which appears every time your device starts and lets you choose ROM to boot. (…) ROMs are installed and managed via modified TWRP recovery”.

Not sure if something similar it will work well with Android’s security system and all that, but I like the idea of being able to use multiple ROMs on a single phone. I somehow dislike the idea that a device has to be “owned” by one ROM to be secure. But a phone that is able to boot different ROMs would fix a lot of problems at once (if possible and somehow stable).

Update: Here are some more details. Maybe people that know more about Andorid can have a look at it and can tell us how ugly this hack is and if it has a useful future for Multibooting OSs on FP with the standard FP OS as secure “Default”.

1 Like

@Max_S a working TWRP would be fantastic! I am currently tying to flash a working xposed framework without a custom recovery but it seems to be a bit tricky. So I can only appreciate your efforts here a lot! :slight_smile:

I can’t. The Developers category is read only. Maybe you can give me access to it. But we can also leave it here.

I have it on my tablet and it works like a charm (I can even fire up Ubuntu (not Ubuntu Touch, the real Ubuntu !)). However the devs responsible for the post say it’s a giant hack, as it requires a patched kernel, and a custom TWRP recovery (“stock” twrp is not enough).

1 Like

[quote=“Max_S, post:3, topic:12558”]
Can you move it? I’ve only access to my fp1 at the moment.[/quote]

I could but, as @Moritz said, nobody would be able to write besides @admins and @moderators. So you need to be a bit more patient until we find an easy way to grant you write access for #software:dev

Ah, yes, that makes sense :smile: then leave it here please.

Thank you very much for this tutorial and bringing TWRP to the FP2.
Rooted my FP2 according to root with superuser and then just flashed the image you provided, and everything works fine so far (only tried a normal Backup to SD-card).
The clock works fine after I’ve set the correct timezone.

However, when I reboot from TWRP it wants to install SuperSU as I thinks my phone is not rooted. I have ignored this so far, as I DO have root access on my phone.

Thanks again for providing the image as it makes the whole process very fast and easy!
I will try to install Xposed and let you know, how it works for me.

1 Like

Very nice, installed the image and no problems so far.
Made a Backup of the whole system. Now trying to flash Chainfires SuperSU to get FolderMount to work.

Edit: Until now, SuperSU from Chainfire doesnt work.

I suggest you read @Max_S’ Post about flashing xposed in this threat, if you haven’t already:
https://forum.fairphone.com/t/flashing-xposed-framework-to-the-fp2-without-custom-recovery/12642/6
I’m a bit stranded at this for now, but quite curious on the results of your experiments :slight_smile:

Flashing XPosed worked without any problems.
I used the image xposed-v79-sdk22-arm.zip from here and installed the XPosedInstaller from the same page. So far I did not notice any problems with the libart.so etc, I did not have to rename/delete anything… Everything went smooth (took only about 10 min after the first reboot) and I installed GravityBox. I will let you know of any problems I encounter.

But so far, awesome job @Max_S!

2 Likes

Are you on stock firmware or on a self-built Android? Can you check if you have such files in /system/vendor/lib/?

Edit:
I checked the system.img from stock firmware and there aren’t libart*.so files in system/vendor/lib. They are in /system/lib as expected and also have a different size. I wonder how they built it.

Interesting. I’ll need to give it another try :wink:
Edit: Did you build on your own?

No, I used the image you provided in your link.

Yes, I still use the stock firmware.

GravityBox manages to change the appearance of my clock in the statusbar, however it does not remove the ‘missing SIM’ symbol after restart. Maybe it needs SuperSU for that? I only installed Superuser…

There are some more phones which a quite alike the Fairphone. What I would recommend is to have a look at the HTC One M8.

https://twrp.me/devices/htconem8gsm.html

Here you can see that there is only a recovery.fstab included, not any twrp.fstab. Maybe that makes it a bit easier to understand:

To get more similar phones, you can go to

Then click “PDAmaster”. There select:

Processor: Qualcomm Snapdragon 801 MSM8974AB v3
Operating System Family: Android
Min Resolution: 1080 x 1920
Max Resolution: 1080 x 1920

and then: START

Many of them do have a TWRP already.

I wisehd I could contribute testing, but until mine, as I just read, there are still 800 phones to go…
Edit: Actually, it should be on it’s way!

@Max_S you did a perfect job there. Did you compile it on Gentoo? :wink:
I built my own recovery image according to your instructions (you forgot the “sh build/environment.sh” and “choosecombo …” stuff but I could cope with that) :smiley:
It booted right away, I was able to write a backup to my 128GB, FAT32-formatted microSD and after setting the timezone, the time was displayed correctly.
Thank you very much!

I wikified the topic, so everyone can improve it now. :slight_smile:

2 Likes