English

[HOWTO] ✏ Compiling Fairphone Open OS / Rooting

Tags: #<Tag:0x00007f05d783efa8> #<Tag:0x00007f05d783ee18>

:warning: Phones can be life savers. Don’t rely on this ROM as your sole communication means. :warning:

This is mostly for users that want to play with their phones :smile: and want to have a bit more control over their devices. Let’s update this together (code, spelling, …).

ROOTING: If you’re just here for rooting the phone, please take a look at the section called “Ways for adding setuid/setgid’ed su/adbd and other binaries to the image (Rooting)” below first. Maybe you don’t need to build from source to do that, the information on this subject is changing quickly!

Compile your own FP2 ROM

  • Have a SSD (not needed, but faster …) with lots of space ready. The checkout and full build as described here needs around 46G (around 67GB for others). A docker image needs around 52G(?). The full build and all repositories checked out as described in the official guide takes around 75G.

  • Use gcc-4.8 (or 4.6). Newer gcc’s will give you trouble with webkit and gcc is not loved anymore by the Android-SoC people anyway. I think there will be/is a switch to LLVM underway. Read about it. It comes with interesting implications licenses-wise. If you have issues on a Debian based system, try using update-alternatives to change the gcc-version. (Add more info on how to do this later)

Prepare your system (check the official build instructions and install all the software needed that is described there)


###Software install on different OSs/distributions

Still in drafting. Follow the official guide first!. Most people install on already “used” machines so packages and experience vary. The Dockerfile can be a good start for peeking as well.

Ubuntu 14.04 LTS

Installation of zlib1g-dev:i386 will result in issues if repo has not been installed before. Install repo (inluded in phablet-tools) first:

$ sudo apt-get install phablet-tools

Follow up with the suggested build environment:

$ sudo apt-get install openjdk-7-jdk bison g++-multilib \
gcc-multilib git gperf  libxml2-utils make \
python-networkx zlib1g-dev:i386 zip

Ubuntu 15.10

(add more info here if it is not working …gcc version must be changed as well, I assume.)

$ sudo apt-get install openjdk-7-jdk bison g++-multilib git gperf \
libxml2-utils make unzip python-networkx zlib1g-dev:i386 zip

Gentoo

OpenJDK can be installed with package dev-java/icedtea or dev-java/icedtea-bin (binary).
Use

eselect java-vm set user icedtea-bin-7

to select the right version at least for the user you’re compiling with…
Python must be switched to version 2.7:

eselect python set python2.7

It may be required to call “make update-api” before “make -j8”.
A build error will state this. Compilation should work afterwards.

(add other systems here)


Using Docker (Windows, other Unix-systems or OS X)

Docker is different. You can get a pre-built image with all software installed so you don’t have to worry about software dependencies. Such a docker image runs in a virtual machine (OS X/Windows) or in a separate container (Linux). You can try the docker image, provided by @jftr.

Keep in mind that installing generated images on your phone will include more steps than currently listed here. The docker image already includes most of the steps that are described below, so you don’t have to worry too much about software dependencies. In the end you will get the *.img files, so make sure you have a working setup for your OS that will allow you to transfer them to your phone (adb/fastboot).


Pulling the Source (Updated for FP2-open-16.05.0)

:exclamation: Clean up ~70 GB of space first! :exclamation:

Go where you have lots of free space, make a directory there and use repo to checkout the code base (the “small” checkout, thanks to @jftr! - only checking out the latest version instead of cloning all repositories entirely)

$ mkdir fairphone_os && cd fairphone_os

$ repo init --depth=1 \
          -u http://code.fairphone.com/gerrit/fp2-dev/manifest \
          -b fp2-sibon
$ repo sync -c

#####Timeconst

diff --git a/kernel/timeconst.pl b/kernel/timeconst.pl
index eb51d76..063fcc0 100644
--- a/kernel/timeconst.pl
+++ b/kernel/timeconst.pl
@@ -370,7 +370,7 @@ if ($hz eq '--can') {
        }
 
        @val = @{$canned_values{$hz}};
-       if (!defined(@val)) {
+       if (!(@val)) {
                @val = compute_values($hz);
        }
        output($hz, @val);

Download the bin blobs and run the ‘unpacking’ script on your computer in the same directory. I think a plain tar ball would be much better. Are we still in the age of shrink-wrap licenses?

$ wget http://code.fairphone.com/downloads/FP2/blobs/fp2-sibon-16.06.0-blobs.tgz
$ tar zxvf fp2-sibon-16.06.0-blobs.tgz
$ sh fp2-sibon-16.06.0-blobs.tgz

Load the helper functions:

$ . build/envsetup.sh

If your build works later, you can add more patches/code steps here (like the ones for Unified NLP, see below). But first ensure that your first build works before adding patches to the existing default code.

Building

Start the default build

$ choosecombo 1 FP2 2
$ make -j8

If all works well, you should finally see something like this after an hour or more depending on your system:

(...) 
#### make completed successfully (09:24 (mm:ss)) ####

The images will be found here: out/target/product/FP2/*.img

Flashing the images

The phone has to be in bootloader mode, either switching it on with keeping volume down button pressed or booting it into bootloader mode with adb reboot bootloader. The phone is in fastboot mode then, and can only be controlled by fastboot on the PC (or switched off by long pressing the power button >10s). It will only show the Fairphone logo in this state. Of course, you have to connect your phone to the PC via USB.

You can flash all at once with fastboot flashall, which flashes boot.img, recovery.img and system.img and automatically reboots the phone (this only works, if you are stil in the same terminal session, see below). Or you flash the respective image and reboot with

  • $ fastboot flash recovery recovery.img
  • $ fastboot flash system system.img (in case of problems use -w [= wipe])
  • $ fastboot reboot

If you’ve opened a new terminal session (e.g. you’ve rebooted your computer in the meantime), the fastboot flashall command will fail. To be able to use this command, you have to set up the build environment again before flashing:

$ . build/envsetup.sh
$ choosecombo 1 FP2 2
$ fastboot flashall

Encrypting the phone (working around a bug)

Due to some weird wiping bug, the built-in disk encryption of the userdata partition (accessible via Settings --> Security --> Encrypt phone) doesn’t work on a self compiled FPOSOS. The workaround is to overwrite the userdata partition by one of the following (very similar) methods. But (for the last two methods) remember to carefully backup your apps and data before and to make a plan for restoring it!

  • If you’re flashing the images one by one, you can insert
    fastboot flash userdata userdata.img
    in between flashing the system image and rebooting.
  • If you’ve used the flashall command or already flashed a while ago, you can
  • do a factory reset via
    Settings --> Backup & reset --> Factory data reset
  • or switch to fastboot mode and solely flash the userdata image:
    fastboot flash userdata userdata.img fastboot reboot

Afterwards, the encryption in the Settings should work.

If you want to set different passwords for encryption and the lock screen, have a look at this topic.

###Ways for adding setuid/setgid’ed su/adbd and other binaries to the image (Rooting)

  • The newest approach is described by @Max_S in his howto. It looks like like a fully scripted way for changing the boot.img of an exiting OTA image. It will save users that just want root from downloading a lot of code! If you want root fast, have a look there.

    Read Max_S’s post (see link above) and check the script in scripts/su/changes.sh after the checkout for better understanding. The shell scripting is a bit odd, but it seems to work. And yes, you will still need unix/a virtual machine to do this. The very short version:

      git clone https://github.com/mstaz/super-bootimg.git1
      cd super-bootimg
      bash download.sh known-imgs/fairphone/fp2
      bash build.sh fairphone/fp2
    
  • Modifying the official image by installing a modified recovery and side-loading the packages you want (Thanks, @lklaus/@vivia!)

  • Follow the instructions on the top of this thread, but stop right before the make -j8 command.

  • Download Busybox from here: https://drive.google.com/file/d/0B-9cNi0ys0CMeGdhTXFlMW9qaUk/view?usp=docslist_api (One-shot compilation by @lklaus )

  • Put rcbins.tar inside your fairphone_os directory and extract it.

  • Download SuperSU from here: https://download.chainfire.eu/696/supersu/ (Do we know how safe this source is?)

  • Put SuperSU inside your fairphone_os directory, but don’t extract it.

  • Apply this patch: https://gist.github.com/vivia/410a8647e0273405fd77
    You can either open it and apply it manually (just a few lines to be added) or look at this: https://groups.google.com/forum/#!topic/repo-discuss/43juvD1qGIQ

  • Now start compiling
    $ make -j8

  • If the compilation is successful:

  • Reboot into the phone’s bootloader
    $ adb reboot bootloader (or press down arrow + power)

  • Connect your phone to the computer if you haven’t done so

  • Wait for the device to appear
    $ fastboot devices

  • Flash the recovery image
    $ fastboot flash recovery out/target/product/FP2/recovery.img
    If this is successful:

  • Reboot into recovery mode again (up arrow + power)

  • From the menu on the phone, select “Apply update from ADB”
    $ adb sideload UPDATE-SuperSU-v2.46.zip

  • Reboot and you’re ready!

  • Compiling your own image and adding the packages you need (will even be signed?) (Add more info here, all this is untested on a real device – @fp1_wo_sw_update)

###Issues
If you have problems compiling, check your gcc version, try make clean, remove the out dir and start over again. Don’t forget to run . build/envsetup.sh and to unpack the bin blobs. Try to post a meaningful part of the error message you get! Often, you have to scroll up a bit to find useful information.

For a full report use something like make -j1 2>&1 | tee build.log and post the more meaningful parts just before the end here.

###GAPPS
If you want to run GAPPS on your phone make sure you read about the apk signing issue here.

###Notes
For comparison and analysis (see @jochensp 's post) have a look at the official image. To find it, just have a look inside the updater.xml file from here. Current is: FP2-gms36-1.1.7-ota.zip. You can also sideload this file to your phone from the recovery.

###Helper scripts

Useful for applying repo diffs, that function is not there right now (by Kapil?). Ugly rm in there, could need a fix with mktemp, anyone?

#!/bin/bash
## Script to patch up diff created by `repo diff`  
 
    if [ -z "$1" ] || [ ! -e "$1" ]; then
        echo "Usages: $0 <repo_diff_file>";
        exit 0;
    fi

    rm -fri _tmp_splits*
    cat $1 | csplit -qf '' -b "_tmp_splits.%d.diff" - '/^project.*\/$/' '{*}' 

    working_dir=`pwd`

    for proj_diff in `ls _tmp_splits.*.diff`
    do 
        chg_dir=`cat $proj_diff | grep '^project.*\/$' | cut -d " " -f 2`
        echo "FILE: $proj_diff $chg_dir"
        if [ -e $chg_dir ]; then
            ( cd $chg_dir; \
                cat $working_dir/$proj_diff | grep -v '^project.*\/$' | patch -Np1;);
        else
            echo "$0: Project directory $chg_dir don't exists.";
        fi
    done
23 Likes

that’s a simple 404 error Where exactly did it turn up?

EDIT: By checking the install instruction paths on the webserver, I think it’s just not online yet…

1 Like

A few seconds after starting repo sync. And it’s repeatable, so I guess there are some missing / obsolete / outdated entries in the manifests.

Nevertheless, I wonder what the “fp2-sibon” is about. Probably, sibon is the code name of the first release version (like it was on FP1), but why not just use tags for that and use branches for refering to actual Android versions like most other projects do?

I downloaded everything according to the explanation and compiled it. Did work so far for me. Trying to get it starting in the android emulator… Up to now I don’t see anything … Emulator process runs and runs…

1 Like

Did you compile it for the emulator target? If so - what did you provide for the the choosecombo command?

ACtually, no… I attacked it totally naively… Just thought it’s an emulator… :wink: Your question sent me to a google search, and it looks I have to provide a kernel also… Will see…

@lklaus Actually, if you just run
lunch
instead ofhis choosecombo command you are presented with a selection of quite a few devices. I have chosen the following one:
mini_emulator_x86_64-userdebug
Update: Didn’t work. Now trying with:
aosp_arm-eng

1 Like

Just to give a short update: the emulator build failed, but the normal build went fine. But I can’t run the emulator, it gives me the following error:
emulator: WARNING: system partition size adjusted to match image file (346 MB > 200 MB) ko:Invalid cache partition image type: yaffs2 (expected ext4)

That looks like the default disk size is used, which has 200MB. I used that also, because I thought it will complain if too small. For the image, you have to use simg2img for conversion first. The emulator can’t handle sparse image. This tool is also built in host commands

Hmmm, rebuilt the FP2 ROM, and started the emulator, according to some documentation this should work. And indeed, after converting system.IMG and cache.img the emulator starts. But I don’t get any output except for startup messages… I’m running on a probably not so powerful virtual machine, but still…

Klaus

Kuleszdl: if you start the emulator with “-debug all”, you cab can see error messages about not being able to connect to a local port. according to a Google search this looks like a problem with the android SDK. Looks like it isn’t fixed up to now, the entry was from June, iirc. Tried also with a downloaded Google android SDK, but to no avail, but I had not too much time for it

I just tried to get my hands on the Fairphone Open Source OS, trying to follow the build instructions. While setting up the build environment the SSH access failed while HTTP worked. See cmd output below. This might be a problem on my side. Or does anybody of you have the same problems? I then would let the support know. :smile:

`simon@laptop:~/src/fairphone_os$ repo init -u ssh://code.fairphone.com:29418/fp2-dev/manifest -b fp2-sibon
gpg: Schlüsselbund /home/simon/.repoconfig/gnupg/secring.gpg erstellt
gpg: Schlüsselbund /home/simon/.repoconfig/gnupg/pubring.gpg erstellt
gpg: /home/simon/.repoconfig/gnupg/trustdb.gpg: trust-db erzeugt
gpg: Schlüssel 920F5C65: Öffentlicher Schlüssel “Repo Maintainer repo@android.kernel.org” importiert
gpg: Schlüssel 692B382C: Öffentlicher Schlüssel “Conley Owens cco3@android.com” importiert
gpg: Anzahl insgesamt bearbeiteter Schlüssel: 2
gpg: importiert: 2 (RSA: 1)
Get https://gerrit.googlesource.com/git-repo/clone.bundle
Get https://gerrit.googlesource.com/git-repo
Get ssh://code.fairphone.com:29418/fp2-dev/manifest
The authenticity of host ‘[code.fairphone.com]:29418 ([37.187.31.110]:29418)’ can’t be established.
RSA key fingerprint is SHA256:0Zj8UF2NXJvVss+YwwMGyAseoACjhVADFiiQPl44nlg.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘[code.fairphone.com]:29418 ([37.187.31.110]:29418)’ can’t be established.
RSA key fingerprint is SHA256:0Zj8UF2NXJvVss+YwwMGyAseoACjhVADFiiQPl44nlg.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘[code.fairphone.com]:29418,[37.187.31.110]:29418’ (RSA) to the list of known hosts.
Permission denied (publickey).
^Caborted by user
simon@laptop:~/src/fairphone_os$ repo init -u ssh://code.fairphone.com:29418/fp2-dev/manifest -b fp2-sibon
Permission denied (publickey).
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: cannot obtain manifest ssh://code.fairphone.com:29418/fp2-dev/manifest
simon@laptop:~/src/fairphone_os$ repo init -u http://code.fairphone.com/gerrit/fp2-dev/manifest -b fp2-sibon
remote: Counting objects: 12, done
remote: Finding sources: 100% (12/12)
remote: Total 12 (delta 3), reused 12 (delta 3)
Von http://code.fairphone.com/gerrit/fp2-dev/manifest

  • [neuer Branch] fp2-sibon -> origin/fp2-sibon
    Your identity is: xxxx xxxxx@xxxxxx
    If you want to change this, please re-run ‘repo init’ with --config-name
    Testing colorized output (for ‘repo diff’, ‘repo status’):
    black red green yellow blue magenta cyan white
    bold dim ul reverse
    Enable color display in this user account (y/N)? y
    repo has been initialized in /home/simon/src/fairphone_os`

At code.fairphone.com there is written that you have to register an account and then you have to upload your public SSH key into your account. Before that, generate a SSH keypair if not already done.

I’ve done that. :slight_smile: Maybe I did something wrong…?
I created an account/logged in via launchpad and uploaded the contents of ~/.ssh/id_rsa.pub. Looking at the output of the checkout via HTTP you can see that it recognised me (I’ve x-ed out the private information).

Additionally: currently I’m running repo sync and curl gives me quite some 404 errors. Not sure if that is how it is supposed to be…

@sjjh: I mentioned the 404 errors above - they seems to be “normal” in runs of repo sync on several Android codebases. Also, you can check out the code without registering using https instead of ssh- worked fine for me.

I tried to compile but experienced following error. :frowning: Did anybody of you succeed in building? Has anybody an idea what I did wrong?
external/chromium_org/third_party/WebKit/Source/core/rendering/RenderBox.h:498:40: internal compiler error: Segmentation fault bool isUnsplittableForPagination() const; ^ target thumb C++: third_party_WebKit_Source_core_webcore_html_gyp <= external/chromium_org/third_party/WebKit/Source/core/html/shadow/SliderThumbElement.cpp Please submit a full bug report, with preprocessed source if appropriate. See <http://source.android.com/source/report-bugs.html> for instructions. build/core/binary.mk:619: recipe for target 'out/target/product/FP2/obj/STATIC_LIBRARIES/third_party_WebKit_Source_core_webcore_html_gyp_intermediates/third_party/WebKit/Source/core/html/shadow/MeterShadowElement.o' failed make: *** [out/target/product/FP2/obj/STATIC_LIBRARIES/third_party_WebKit_Source_core_webcore_html_gyp_intermediates/third_party/WebKit/Source/core/html/shadow/MeterShadowElement.o] Error 1 make: *** Auf noch nicht beendete Prozesse wird gewartet … target thumb C++: third_party_WebKit_Source_core_webcore_html_gyp <= external/chromium_org/third_party/WebKit/Source/core/html/shadow/SpinButtonElement.cpp #### make failed to build some targets (02:32:18 (hh:mm:ss)) ####

edit: the “^” is supposed to be under the “const”

Which OS and compiler are you using?

Ubuntu 15.10 and following the build instructions from code.fairphone.com, so gcc:

simon@laptop:~$ g++ --version g++ (Ubuntu 5.2.1-22ubuntu2) 5.2.1 20151010

It’s been working for me. Only problem is, you need lots of mem, and in case, swap. Can’t compile successfully unless I give my Dev VM 4G mem and 4G swap. Check with dmesg whether you hit oom

@lklaus, I have 8GB of mem (although 4GB are reserved for /tmp) and 8.5GB swap. I don’t think that this is a bottleneck (and the error message didn’t read like it, although I have to admit that I never compiled android before).
I acctually twice hit hdd limits, didn’t expect to need 52GB…