TWRP for Fairphone 4

lucaweiss · January 18, 2022, 11:29am

super is not encrypted so sure. Not 100% sure how well the backup from TWRP itself is but you can definitely just use dd from the command line to just make a 1:1 image of the partition that you should be able to flash back via fastboot.
The dd image will also be the full size of the partition, and not just the system data it contains, maybe it’s possible to use the img2simg tool to shrink it down and convert it into a sparse image.

Discostu36 · January 18, 2022, 11:44am

So do I understand it correctly that the guide I linked to doesn’t work for FP4? If that is the case, it should be mentioned in the guide.

Do I also understand correctly that a full backup and restore is not possible at all?

hirnsushi · January 18, 2022, 11:44am

Backing up super in TWRP fails, I just tried.
I do have backups of the super partition and all the others, but from a running system. I don’t know if I can flash those back to my device safely, since we have had problems with AVB in other topics before. At around 100 partitions (with duplicates, full storage device backups…) the chance for failure seems pretty high to me. Are you sure this works?

Well maybe I’ll just have to flash /e/ and try to go back to vanilla

@Discostu36 You can’t backup /data and super from TWRP at this point.
I wouldn’t suggest flashing /e/ if you are not sure you want to stay there, if that’s what you’re asking

Discostu36 · January 18, 2022, 11:51am

My use case is that I can do a factory reset / lose my phone / wreck my mainboard / whatever and then have a backup that I can restore and have everything in place like nothing ever happened.

hirnsushi · January 18, 2022, 11:53am

That won’t be possible, no.
Your best bet is setting up something like Syncthing for your important data or trust Google with your stuff and activate their cloud backup (that will backup app settings for compatible apps as well)

oli.sax · January 26, 2022, 10:26am

I followed build instructions and I was able to build the recovery image

[100% 8355/8355] build out/target/product/FP4/recovery.img
#### build completed successfully (43:03 (mm:ss)) ####

Image is booting correctly in TWRP 3.6.0_11-0
however the file differs from yours:

SHA256 hash di recovery.img:
7f9bc1836f83de679eba2992176ce8d25b1190e0a666d8a3f19e1c689efaf6c9

SHA256 hash di .\twrp-3.6.0_FP4-UNOFFICIAL-20211216.img:
9358dc6e623d213992e1c1c14b340f07524641491453f3c7065b723e4024df46

Is accessing encrypted data a general-TWRP function that is still a work-in-progress, or is this somehow already available and needs to be ported specifically to FP4?

Having this function will help a lot making backups and resolving some stupid mistakes

hirnsushi · January 26, 2022, 10:30am

Accessing encrypted /data through TWRP is possible on other devices, this has to be ported specifically for the FP4.

lucaweiss · January 26, 2022, 10:45am

Great!

I don’t think TWRP images can be built reproducibly (so the file is really identical) and even if, your sources are very likely newer than the ones I built it with.

Should work in theory, it’s even relatively well documented at GitHub - TeamWin/android_device_qcom_twrp-common: TWRP standard device files for Qualcomm SoCs but last time I tried I couldn’t get it to work.

oli.sax · January 27, 2022, 10:09am

Is FP4 a FBE (File-based encryption) or a FDE (full-disk encryption) device?

On the “Troubleshooting/Debugging” section, they advice to add a line or two (setprop prepdecrypt.loglevel 2 and setprop prepdecrypt.setpatch true) to have debug logs, did you try that already?

eudaldgr · January 30, 2022, 11:52am

It seems to be FBE:

$ adb shell getprop ro.crypto.type
file

https://android.stackexchange.com/a/195754

oli.sax · February 3, 2022, 4:17pm

Ok. I’m not familiar with code compilation, but I’m eager to learn .
Even if this Github page is “relatively well documented”, I had to read it several times to (kind of) understand what are the steps.

Here I made a summary

Prerequisites

Vendor service binaries and dependencies for FBE : qseecomd, keymaster, gatekeeper

the binaries should be placed in system/bin
check the dependencies with the “ldcheck” script.

1b)

init.recovery.$(ro.hardware).rc file in device tree with symlink for bootdevice included:

symlink /dev/block/platform/soc/${ro.boot.bootdevice} /dev/block/bootdevice

BoardConfig flags

BOARD_USES_QCOM_FBE_DECRYPTION := true

2b)

The packages will need to be added to the device tree device.mk as indicated below:

PRODUCT_PACKAGES += \
    qcom_decrypt \
    qcom_decrypt_fbe

2c)

To import the decryption rc files into your device tree, add this line to your init.recovery.$(ro.hardware).rc file:

import /init.recovery.qcom_decrypt.rc

Troubleshooting/Debugging

adding the following code to the on init section of your init.recovery.$(ro.hardware).rc file:

setprop prepdecrypt.loglevel 2 
setprop prepdecrypt.setpatch true

So to my understanding, the files to be modified we are speaking about are:

1b, 2c, 3 ->[device/fairphone/FP4]/recovery/root/init.recovery.qcom.rc
2-> [device/fairphone/FP4]/BoardConfig.mk

Those two files are present in the z3ntu/android_device_fairphone_FP4 repo

My doubts remaining:

1 - where to extract those 3 binaries from?? From a repo somewhere? From the vendor partition of FP4?
2b - where is the device.mk file I have to modify? Looking in my local directory there are plenty of them!
Is there something to pull/sync from the github /TeamWin/android_device_qcom_twrp-common or not?

eudaldgr · February 5, 2022, 7:11pm

I made a fork and I’m playing with TWRP (my first time ), here the link.
I extracted the binary blobs from my own FP4, booting from a patched image with Magisk.
I have tried several things and managed to compile it, but i’m shooting blind.

I’m going to review the steps I did with your explanation, it will be very helpful.

oli.sax · February 7, 2022, 2:08pm

Hello!
Nice to see others are playing on TWRP too

Did you try booting it already? What is the result?
Maybe if it doesn’t decrypt correctly you can add the two debugging lines I listed as “3”?

eudaldgr · February 8, 2022, 7:54am

Don’t boot, it just stays in the ‘powered by android’ image.
These debug flags only work, if the recovery is installed and not if you boot from fastboot boot ... as specified in the README.
You can try to compile it yourself and test it.

oli.sax · February 26, 2022, 11:50am

Compile is done, I’ll try it soon

Terminal output

PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=127
TARGET_PRODUCT=twrp_FP4
TARGET_BUILD_VARIANT=eng
TARGET_BUILD_TYPE=release
TARGET_ARCH=arm64
TARGET_ARCH_VARIANT=armv8-a
TARGET_CPU_VARIANT=kryo
TARGET_2ND_ARCH=arm
TARGET_2ND_ARCH_VARIANT=armv8-a
TARGET_2ND_CPU_VARIANT=cortex-a73
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-5.4.0-100-generic-x86_64-Ubuntu-20.04.4-LTS
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
[ 27% 4874/17836] //bionic/libc:libc.llndk versioner preprocess include
warning: attempted to generate guard with empty availability: obsoleted = 21
warning: attempted to generate guard with empty availability: obsoleted = 23
[ 27% 4886/17836] //bionic/libc:libc.llndk versioner preprocess include [arm]
warning: attempted to generate guard with empty availability: obsoleted = 21
warning: attempted to generate guard with empty availability: obsoleted = 23
[ 39% 6999/17836] Prebuilt:  (out/target/product/FP4/kernel)
calling depmod on prebuilt modules
depmod: WARNING: could not open modules.order at /home/oli/fp4-twrp2/out/target/product/FP4/recovery/root/vendor/lib/modules/1.1: No such file or directory
depmod: WARNING: could not open modules.builtin at /home/oli/fp4-twrp2/out/target/product/FP4/recovery/root/vendor/lib/modules/1.1: No such file or directory
[ 99% 17828/17836] Target vendor buildinfo: out/target/product/FP4/vendor/build.prop
Target vendor properties from: device/fairphone/FP4/vendor.prop
[ 99% 17834/17836] Install: out/target/product/FP4/root/system/bin/qcom_decrypt_fbe-timestamp
 
Running Service_Cleanup script for Qcom decryption...

OUT Folder set to: out/target/product/FP4

FBE Status: true

Blobs parsed:
android.hardware.keymaster@4.1-service-qti
android.hardware.gatekeeper@1.0-service-qti
 
Services in rc file:
android.hardware.keymaster@3.0-service
android.hardware.keymaster@3.0-service-qti
android.hardware.keymaster@4.0-service
android.hardware.keymaster@4.0-service-qti
android.hardware.keymaster@4.1-service
android.hardware.keymaster@4.1-service-qti
android.hardware.keymaster@4.1-service.citadel
android.hardware.gatekeeper@1.0-service
android.hardware.gatekeeper@1.0-service-qti
 
Services not included:
android.hardware.gatekeeper@1.0-service
android.hardware.keymaster@3.0-service
android.hardware.keymaster@3.0-service-qti
android.hardware.keymaster@4.0-service
android.hardware.keymaster@4.0-service-qti
android.hardware.keymaster@4.1-service
android.hardware.keymaster@4.1-service.citadel
 
Removing unneeded service: gatekeeper-1-0
Removing unneeded service: keymaster-3-0
Removing unneeded service: keymaster-3-0-qti
Removing unneeded service: keymaster-4-0
Removing unneeded service: keymaster-4-0-qti
Removing unneeded service: keymaster-4-1
Removing unneeded service: keymaster-4-1-citadel
 
Service_Cleanup script complete.

[100% 17836/17836] build out/target/product/FP4/recovery.img

#### build completed successfully (01:27:39 (hh:mm:ss)) ####

EDIT: ldcheck tells me that the dependency libandroidicu.so is missing for qseecomd!

ldcheck for qseecomd

/home/oli/fp4-twrp2/ldcheck/ldcheck -p system/lib64:vendor/lib64 -d system/bin/qseecomd
readelf: ERREUR: « libandroidicu.so »: pas de tel fichier
libs: ['system/bin/qseecomd', 'system/lib64/libcutils.so', 'system/lib64/libutils.so', 'system/lib64/liblog.so', 'vendor/lib64/libQSEEComAPI.so', 'vendor/lib64/libdrmfs.so', 'system/lib64/libc++.so', 'system/lib64/libc.so', 'system/lib64/libm.so', 'system/lib64/libdl.so', 'system/lib64/libbase.so', 'system/lib64/libprocessgroup.so', 'system/lib64/libvndksupport.so', 'system/lib64/libion.so', 'vendor/lib64/libdiag.so', 'system/lib64/libxml2.so', 'system/lib64/ld-android.so', 'system/lib64/libcgrouprc.so', 'system/lib64/libdl_android.so', 'libandroidicu.so']
nm: « libandroidicu.so »: pas de tel fichier
system/lib64/libxml2.so   UNRESOLVED #####          U ucnv_close_android
system/lib64/libxml2.so   UNRESOLVED #####          U ucnv_convertEx_android
system/lib64/libxml2.so   UNRESOLVED #####          U UCNV_FROM_U_CALLBACK_STOP_android
system/lib64/libxml2.so   UNRESOLVED #####          U ucnv_open_android
system/lib64/libxml2.so   UNRESOLVED #####          U ucnv_setFromUCallBack_android
system/lib64/libxml2.so   UNRESOLVED #####          U ucnv_setToUCallBack_android
system/lib64/libxml2.so   UNRESOLVED #####          U UCNV_TO_U_CALLBACK_STOP_android
nm: « libandroidicu.so »: pas de tel fichier
unused: {'system/lib64/libutils.so', 'system/lib64/libcgrouprc.so', 'system/lib64/libdl_android.so', 'system/lib64/libxml2.so', 'system/lib64/libvndksupport.so', 'libandroidicu.so', 'system/lib64/libprocessgroup.so', 'vendor/lib64/libdrmfs.so', 'vendor/lib64/libQSEEComAPI.so', 'system/lib64/libion.so', 'vendor/lib64/libdiag.so', 'system/lib64/libm.so'}

EDIT 2: gatekeeper and keymaster seems to have all dependencies satisfied

ldcheck for gatekeeper and keymaster

oli@CRMG5261:~/fp4-twrp2/out/target/product/FP4/recovery/root$ /home/oli/fp4-twrp2/ldcheck/ldcheck -p system/lib64:vendor/lib64 -d system/bin/android.hardware.gatekeeper@1.0-service-qti
libs: ['system/bin/android.hardware.gatekeeper@1.0-service-qti', 'system/lib64/android.hardware.gatekeeper@1.0.so', 'system/lib64/libhardware.so', 'system/lib64/libhidlbase.so', 'system/lib64/libutils.so', 'system/lib64/liblog.so', 'system/lib64/libc++.so', 'system/lib64/libc.so', 'system/lib64/libm.so', 'system/lib64/libdl.so', 'system/lib64/libcutils.so', 'system/lib64/libvndksupport.so', 'system/lib64/libbase.so', 'system/lib64/libprocessgroup.so', 'system/lib64/ld-android.so', 'system/lib64/libdl_android.so', 'system/lib64/libcgrouprc.so']
unused: {'system/lib64/libhardware.so'}
oli@CRMG5261:~/fp4-twrp2/out/target/product/FP4/recovery/root$ /home/oli/fp4-twrp2/ldcheck/ldcheck -p system/lib64:vendor/lib64 -d system/bin/android.hardware.keymaster@4.1-service-qti
libs: ['system/bin/android.hardware.keymaster@4.1-service-qti', 'system/lib64/liblog.so', 'system/lib64/libcutils.so', 'system/lib64/libbase.so', 'system/lib64/libutils.so', 'system/lib64/libhardware.so', 'system/lib64/libhidlbase.so', 'system/lib64/android.hardware.keymaster@4.0.so', 'system/lib64/android.hardware.keymaster@4.1.so', 'vendor/lib64/libqtikeymaster4.so', 'system/lib64/libcrypto.so', 'system/lib64/libc++.so', 'system/lib64/libc.so', 'system/lib64/libm.so', 'system/lib64/libdl.so', 'system/lib64/libprocessgroup.so', 'system/lib64/libvndksupport.so', 'system/lib64/android.hardware.keymaster@3.0.so', 'vendor/lib64/libkeymasterdeviceutils.so', 'vendor/lib64/libkeymasterutils.so', 'vendor/lib64/libqcbor.so', 'system/lib64/ld-android.so', 'system/lib64/libcgrouprc.so', 'system/lib64/libdl_android.so', 'system/lib64/libion.so', 'vendor/lib64/libQSEEComAPI.so']
unused: {'system/lib64/libcgrouprc.so', 'system/lib64/libm.so', 'system/lib64/libprocessgroup.so', 'system/lib64/libhardware.so', 'system/lib64/android.hardware.keymaster@3.0.so'}

eudaldgr · February 28, 2022, 11:42am

libandroidicu.so are compiled from source: link

oli.sax · February 28, 2022, 12:08pm

Ok good

And a (probably stupid) question: why, in this BoardConfig file, in the section:

TARGET_RECOVERY_DEVICE_MODULES there are 3 elements (libandroidicu libion libxml2)
RECOVERY_LIBRARY_SOURCE_FILES there are only the 2 last ones…

Looking randomly for those keywords on google I found a device.mk for the xiaomi vayu, and they wrote the path of libandroidicu.so in the section PRODUCT_COPY_FILES, so I assume they took it from the device instead of compiling it, even if it’s also in TARGET_RECOVERY_DEVICE_MODULES …

oli.sax · March 5, 2022, 5:27pm

Just to try: I searched libandroidicu.so on my FP4 device (boot from a rooted boot.img) with lsof |grep icu.so and found it there:

/apex/com.android.art/lib64/libandroidicu.so

I pulled it with adb and put it in the device/fairphone/FP4/recovery/root/vendor/lib64/ directory (not sure if it’s the good path, as I read elsewere “/system/lib64”)
Then running again ldcheck it complained about two other missing libraries , libicuuc.so and libicui18n.so :

/apex/com.android.art/lib64/libicuuc.so
/apex/com.android.art/lib64/libicui18n.so

Pulled them to the same vendor/lib64 directory, and build it again, now ldcheck seems happy:

ldcheck output

~/fp4-twrp2/out/target/product/FP4/recovery/root$ /home/oli/fp4-twrp2/ldcheck/ldcheck -p system/lib64:vendor/lib64 -d system/bin/qseecomd 
libs: ['system/bin/qseecomd', 'system/lib64/libcutils.so', 'system/lib64/libutils.so', 'system/lib64/liblog.so', 'vendor/lib64/libQSEEComAPI.so', 'vendor/lib64/libdrmfs.so', 'system/lib64/libc++.so', 'system/lib64/libc.so', 'system/lib64/libm.so', 'system/lib64/libdl.so', 'system/lib64/libbase.so', 'system/lib64/libprocessgroup.so', 'system/lib64/libvndksupport.so', 'system/lib64/libion.so', 'vendor/lib64/libdiag.so', 'system/lib64/libxml2.so', 'system/lib64/ld-android.so', 'system/lib64/libcgrouprc.so', 'system/lib64/libdl_android.so', 'vendor/lib64/libandroidicu.so', 'vendor/lib64/libicuuc.so', 'vendor/lib64/libicui18n.so']
unused: {'vendor/lib64/libQSEEComAPI.so', 'vendor/lib64/libicuuc.so', 'vendor/lib64/libicui18n.so', 'vendor/lib64/libdrmfs.so', 'system/lib64/libprocessgroup.so', 'system/lib64/libdl_android.so', 'system/lib64/libutils.so', 'vendor/lib64/libandroidicu.so', 'system/lib64/libcgrouprc.so', 'system/lib64/libm.so', 'system/lib64/libion.so', 'system/lib64/libxml2.so', 'vendor/lib64/libdiag.so', 'system/lib64/libvndksupport.so'}

However when I fastboot boot the new recovery.img, I get stuck at a blinking TWRP welcome screen:

Picture

Pulling /tmp/recovery.log doesn’t gives many hints on what is happening…

recovery.log

Starting TWRP 3.6.0_11-0-2c53845c on Sat Mar 5 17:05:31 2022
(pid 1298)
I:Lun file ‘/sys/class/android_usb/android0/f_mass_storage/lun0/file’ does not exist, USB storage mode disabled
PRODUCT_USE_DYNAMIC_PARTITIONS := true
TW_INCLUDE_CRYPTO := true
I:TW_BRIGHTNESS_PATH := /sys/class/backlight/panel0-backlight/brightness
I:Found brightness file at ‘/sys/class/backlight/panel0-backlight/brightness’
I:TWFunc::Set_Brightness: Setting brightness control to 127
I:TW_EXCLUDE_ENCRYPTED_BACKUPS := true
I:LANG: en
I:AB_OTA_UPDATER := true
Starting the UI…
setting DRM_FORMAT_BGRA8888 and GGL_PIXEL_FORMAT_BGRA_8888
setting DRM_FORMAT_ARGB8888 and GGL_PIXEL_FORMAT_RGBA_8888
setting DRM_FORMAT_ARGB8888 and GGL_PIXEL_FORMAT_RGBA_8888

It’s actually exactly the same first lines as the working TWRP from @z3ntu, then it stops before the line Using drm graphics.

eudaldgr · March 6, 2022, 10:29am

Maybe they can help you here. I opened a topic for Fairphone 4 inside suport-device stream, time ago.
I can’t dedicate hours to this right now.

lucaweiss · June 2, 2022, 1:28pm

I’ve just released twrp-3.6.2_FP4-UNOFFICIAL-20220602 which fixes MTP and is rebuilt from latest TWRP sources.

The initial post is updated with the new download link.