[HOWTO] ✏ Compiling Fairphone Open OS / Rooting

@lklaus, I have 8GB of mem (although 4GB are reserved for /tmp) and 8.5GB swap. I don’t think that this is a bottleneck (and the error message didn’t read like it, although I have to admit that I never compiled android before).
I acctually twice hit hdd limits, didn’t expect to need 52GB…

No, then I would expect an oom situation… gcc can give all kind of errors, when a subprocess gets killed, though. But then, the more misterious. I’d definitely check dmesg. Worst thing might be it could get too hot for reliable operation… Did you use a -j N flag for make? Maybe that’s been to high? I did a successful compile with a 2CPU 4G VM on my 8G real system, took quite some time but did finish successful. So, this is supposed to work :smile:

EDIT: as this lead to confusion, in the first sentence a “not” is missing. Of course that should be plenty RAM and not lead to OOM… Except there are some ray tracing or big data apps running in parallel :wink:

sorry, I believe I don’t get you. What did you (not) expect?

As the failure is some time ago (and the compilation runs 2:30h), do you have something to grep for in dmesg? Not sure what I’m looking after…
I indeed have some temperature warnings in dmesg, but the CPU throttled itself down in that cases – so should be ok, shouldn’t it? I did run make -j8 as described in the build instructions. Having an i5 with 4 cores, should I reduce it to -j3 or so?

I would first try with -j1, that’s the safest way. Also, your compiler might be “too new” as the original build instructions are meant for Ubuntu 14.04 which worked well for me.

So I did give it a try with -j1 and it failed pretty soon. With another error message:

Install: out/host/linux-x86/bin/apicheck
Checking API: checkpublicapi-last
Checking API: checkpublicapi-current
out/target/common/obj/PACKAGING/public_api.txt:20: error 5: Added public field android.Manifest.permission.BACKUP
out/target/common/obj/PACKAGING/public_api.txt:82: error 5: Added public field android.Manifest.permission.INVOKE_CARRIER_SETUP
out/target/common/obj/PACKAGING/public_api.txt:106: error 5: Added public field android.Manifest.permission.READ_PRIVILEGED_PHONE_STATE
out/target/common/obj/PACKAGING/public_api.txt:116: error 5: Added public field android.Manifest.permission.RECEIVE_EMERGENCY_BROADCAST

******************************
You have tried to change the API from what has been previously approved.

To make these errors go away, you have two choices:
   1) You can add "@hide" javadoc comments to the methods, etc. listed in the
      errors above.

   2) You can update current.txt by executing the following command:
      make update-api

  To submit the revised current.txt to the main Android repository,
  you will need approval.
******************************

build/core/tasks/apicheck.mk:62: recipe for target 'out/target/common/obj/PACKAGING/checkpublicapi-current-timestamp' failed
make: *** [out/target/common/obj/PACKAGING/checkpublicapi-current-timestamp] Error 38

#### make failed to build some targets (06:01 (mm:ss)) ####

I have no idea what’s going on here. I did following before I tried to recompile, has that been enough?

simon@laptop:~$ rm -r src/fairphone_os/out/*
simon@laptop:~$ cd src/fairphone_os/
simon@laptop:~/src/fairphone_os$ . build/envsetup.sh
including device/asus/flo/vendorsetup.sh
including device/asus/grouper/vendorsetup.sh
including device/asus/deb/vendorsetup.sh
including device/asus/fugu/vendorsetup.sh
including device/asus/tilapia/vendorsetup.sh
including device/generic/mini-emulator-x86_64/vendorsetup.sh
including device/generic/mini-emulator-x86/vendorsetup.sh
including device/generic/mini-emulator-armv7-a-neon/vendorsetup.sh
including device/generic/mini-emulator-arm64/vendorsetup.sh
including device/generic/mini-emulator-mips/vendorsetup.sh
including device/htc/flounder/vendorsetup.sh
including device/qcom/common/vendorsetup.sh
including device/samsung/manta/vendorsetup.sh
including device/lge/mako/vendorsetup.sh
including vendor/qcom/proprietary/vendorsetup.sh
Build with proprietary libs
including sdk/bash_completion/adb.bash
simon@laptop:~/src/fairphone_os$ choosecombo 1 FP2 2
Build type choices are:
     1. release
     2. debug

Which would you like? [1] 1


Product choices are:
     1. FP2
     2. msm8974
Which product would you like? [full] FP2


Variant choices are:
     1. user
     2. userdebug
     3. eng
Which would you like? [eng] 2

============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=5.1
TARGET_PRODUCT=FP2
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a-neon
TARGET_CPU_VARIANT=krait
TARGET_2ND_ARCH=
TARGET_2ND_ARCH_VARIANT=
TARGET_2ND_CPU_VARIANT=
HOST_ARCH=x86_64
HOST_OS=linux
HOST_OS_EXTRA=Linux-4.2.0-22-generic-x86_64-with-Ubuntu-15.10-wily
HOST_BUILD_TYPE=release
BUILD_ID=FP2
OUT_DIR=out
============================================

simon@laptop:~/src/fairphone_os$ make -j1

Hi A few comments here:

-You need at least 8Gig of memory to compile webkit
-First test that you uploaded the correct ssh key by doing an ssh to the server
e.g. ssh -p 29418 code.fairphone.com
-If you run into problems try building a pure AOSP (5.1) first (targeting the nexus for example)
-The emulator is no supposed to work (at least we never tried) and it is very uncommon to develop using the emulator. It is however possible to install the fairphone apps on any phone and emulator

Greetings

2 Likes

Right. I forgot a “not”… Sorry for confusion.
I don’t know a pattern to check, but the last entries right after the fact might be having some hints, if there are any at all.
I compiled it sucessfully on VERSION=“14.04.3 LTS, Trusty Tahr”

Did anybody alread try this version?

What if I want to go back? Any way to reflash “Stock”? Did not see the option to make a backup in recovery

Thanks @keesj for your information. Let’s see:

-You need at least 8Gig of memory to compile webkit

I physically do have 8GB of mem. I have devoted up to 4GB to /tmp (fstab: tmpfs /tmp tmpfs nosuid 0 0), but have additional 8.5GB of swap. So if /tmp is full, together with swap the compile process can still allocate 10.5GB of memory (theoretically, as my OS will need some as well :slight_smile: ). so memory shouldn’t be a problem, is it?
Actually thinking about it, could the max. of 4GB of /tmp be too small? (Where does the compilation put their temp files and how much size does it need?)
Edit I just retried, failed again after 5 minutes (with the error mentioned before and df -h showed me a usage of only 1.9MB for /tmp

-First test that you uploaded the correct ssh key by doing an ssh to the servere.g. ssh -p 29418 code.fairphone.com

Solved, and newly fetched/syned. No changes were pulled.

-If you run into problems try building a pure AOSP (5.1) first (targeting the nexus for example)

Is there a step-by-step guide somewhere how to do it (I probably can use the same repo url but need other build targets and such, don’t I? I failed to locate the chosecombo to inspect it.) Sorry for the naive questions, I’m new to android and android development. :blush:
If it costs a lot additional disk space I’ll have to wait until I’m back home in two weeks or so and can offload some files.

-The emulator is no supposed to work (at least we never tried) and it is very uncommon to develop using the emulator. It is however possible to install the fairphone apps on any phone and emulator

I believe I didn’t try/use the emulator (at least not consciously).

Atm I have not enough disk space to create an additional VM with Ubuntu 14.04, I would need to wait two weeks or so until I have access to my infrastructure again to offload some files. Is there by chance someone who managed to compiled on Ubuntu 15.10? :slight_smile:

Looking at the error messages (and knowing that I didn’t edit the[se] files) I get the feeling that maybe someone (Google? Fairphone?) has committed changes that break the build. Or am I misinterpreting something here? I’m not sure where I shall look for git logs or what else to do.
Is anybody still able to build with the current source?

@sjjh yes, my compile went through. I have some strange effects, though. Compiling with make -j3 (2 cpus) , there seem to be some race confitions in Makefile dependencies. Sometimes make bails out with dependency messages, simply reissue the make command everything goes fine…

Klaus

Thanks for the info @lklaus. Could you maybe check the sha1 sum of the three files mentioned in the error message and compare them to my results posted below. I’m curious why it works for you and not for me. Maybe the error messages are misleading and something totally different is the root cause, no idea…

simon@laptop:~$ shasum src/fairphone_os/out/target/common/obj/PACKAGING/public_api.txt 
b88dbbd34859a0efeeab8208356bff8a702df698  src/fairphone_os/out/target/common/obj/PACKAGING/public_api.txt
simon@laptop:~$ shasum src/fairphone_os/out/target/common/obj/PACKAGING/checkpublicapi-last-timestamp 
da39a3ee5e6b4b0d3255bfef95601890afd80709  src/fairphone_os/out/target/common/obj/PACKAGING/checkpublicapi-last-timestampsimon@laptop:~$ shasum src/fairphone_os/frameworks/base/api/current.txt 
simon@laptop:~$ shasum src/fairphone_os/frameworks/base/api/current.txt 
7ec361085fda6ad6c572915034c2e6d6d5cfc594  src/fairphone_os/frameworks/base/api/current.txt

Happy new year :slight_smile: Simon

Edit: added third file to list. If I compare the public_api.txt and current.txt on my system there are differences.

the first file differs, the second is identical.
As compilation on my system takes some time, I might be a bit behind with synchronization. Will do a repo sync and recompile
edit as well: third file is identical

And Happy new Year, of course!

Klaus

Being in the out folder, I would imagine that the first file gets generated by me. So why/how did I end up with a difference there…? I might just remove the additional lines, but this seems more a hack than understanding the problem. Maybe I should do another sync and all following steps to set up the build process once again as well.

simon@laptop:~/src/fairphone_os$ diff out/target/common/obj/PACKAGING/public_api.txt frameworks/base/api/current.txt 
20d19
<     field public static final java.lang.String BACKUP = "android.permission.BACKUP";
82d80
<     field public static final java.lang.String INVOKE_CARRIER_SETUP = "android.permission.INVOKE_CARRIER_SETUP";
106d103
<     field public static final java.lang.String READ_PRIVILEGED_PHONE_STATE = "android.permission.READ_PRIVILEGED_PHONE_STATE";
116d112
<     field public static final java.lang.String RECEIVE_EMERGENCY_BROADCAST = "android.permission.RECEIVE_EMERGENCY_BROADCAST";

I don’t have those lines… repo sync is runing

I just felt lucky and tried another compilation run with -j2, running longer than 5 minutes now and no errors so far, keeping fingers crossed…

This time compiling with -j2 instead of -j1 (without changing anything on the source before) it failed after a much longer time and with a different error message:

Install: out/target/product/generic/system/etc/security/mac_permissions.xml
out/host/linux-x86/bin/checkpolicy:  loading policy configuration from out/target/product/generic/obj/ETC/sepolicy.recovery_intermediates/policy_recovery.conf
external/sepolicy/radio.te:33:ERROR 'unknown type persist_file' at token ';' on line 6909:
allow radio persist_file:dir search;
checkpolicy:  error(s) encountered while parsing configuration
external/sepolicy/Android.mk:149: recipe for target 'out/target/product/generic/obj/ETC/sepolicy.recovery_intermediates/sepolicy.recovery' failed
make: *** [out/target/product/generic/obj/ETC/sepolicy.recovery_intermediates/sepolicy.recovery] Error 1
make: *** Auf noch nicht beendete Prozesse wird gewartet …
simon@laptop:~/src/fairphone_os$ 

I’ve got no idea what’s going on, where my mistake is. Syncing the repo didn’t pull anything in, so it seems that my source is up to date.

Edit: I’ve got four blocks of following in dmesg:

    [ 9260.200763] CPU1: Core temperature above threshold, cpu clock throttled (total events = 237)
    [ 9260.200764] CPU0: Core temperature above threshold, cpu clock throttled (total events = 237)
    [ 9260.200767] CPU3: Package temperature above threshold, cpu clock throttled (total events = 239)
    [ 9260.200768] CPU2: Package temperature above threshold, cpu clock throttled (total events = 239)
    [ 9260.200769] CPU0: Package temperature above threshold, cpu clock throttled (total events = 239)
    [ 9260.200778] CPU1: Package temperature above threshold, cpu clock throttled (total events = 239)
    [ 9260.201779] CPU0: Core temperature/speed normal
    [ 9260.201780] CPU1: Core temperature/speed normal
    [ 9260.201782] CPU2: Package temperature/speed normal
    [ 9260.201783] CPU3: Package temperature/speed normal
    [ 9260.201783] CPU1: Package temperature/speed normal
    [ 9260.201790] CPU0: Package temperature/speed normal
    [ 9310.022764] mce: [Hardware Error]: Machine check events logged

:worried:

I’d be worried by the MCEs…

The SELinux compile error happens for me when I did not use chosecombo 1 FP2 2, but tried to compile for the emulator.

I got one mce error with each temperature notice in dmesg, so I assume they they belong together. I installed mcelog to be able to maybe find out more about them. Indeed this doesn’t look normal, I’d say.

I’m sure that I used the correct chosecombo invocation, as I copy & pasted it from source.fairphone.com, but I’ll give it another try (and hope to catch more information about that mce’s in case they occur again.

Next failed compilation:

In file included from /home/simon/src/fairphone_os/external/stlport/stlport/stl/_pair.h:36:0,
                 from /home/simon/src/fairphone_os/external/stlport/stlport/stl/_algobase.h:50,
                 from /home/simon/src/fairphone_os/external/stlport/stlport/stl/_tree.h:54,
                 from /home/simon/src/fairphone_os/external/stlport/stlport/stl/_map.h:34,
                 from /home/simon/src/fairphone_os/external/stlport/stlport/map:36,
                 from external/chromium_org/third_party/webrtc/modules/video_coding/main/source/codec_database.h:14,
                 from external/chromium_org/third_party/webrtc/modules/video_coding/main/source/codec_database.cc:11:
/home/simon/src/fairphone_os/external/stlport/stlport/stl/type_traits.h:244:28: internal compiler error: Segmentation fault
    typedef __false_type    has_trivial_copy_constructor;
                            ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://source.android.com/source/report-bugs.html> for instructions.
build/core/binary.mk:619: recipe for target 'out/target/product/FP2/obj/STATIC_LIBRARIES/third_party_webrtc_modules_webrtc_video_coding_gyp_intermediates/third_party/webrtc/modules/video_coding/main/source/codec_database.o' failed
make: *** [out/target/product/FP2/obj/STATIC_LIBRARIES/third_party_webrtc_modules_webrtc_video_coding_gyp_intermediates/third_party/webrtc/modules/video_coding/main/source/codec_database.o] Error 1
make: *** Auf noch nicht beendete Prozesse wird gewartet …

#### make failed to build some targets (23:36 (mm:ss)) ####

dmesg:

[ 2643.249055] CPU0: Core temperature above threshold, cpu clock throttled (total events = 1)
[ 2643.249056] CPU1: Core temperature above threshold, cpu clock throttled (total events = 1)
[ 2643.249059] CPU3: Package temperature above threshold, cpu clock throttled (total events = 1)
[ 2643.249060] CPU2: Package temperature above threshold, cpu clock throttled (total events = 1)
[ 2643.249061] CPU1: Package temperature above threshold, cpu clock throttled (total events = 1)
[ 2643.249070] CPU0: Package temperature above threshold, cpu clock throttled (total events = 1)
[ 2643.253064] CPU1: Core temperature/speed normal
[ 2643.253065] CPU0: Core temperature/speed normal
[ 2643.253067] CPU2: Package temperature/speed normal
[ 2643.253068] CPU3: Package temperature/speed normal
[ 2643.253069] CPU0: Package temperature/speed normal
[ 2643.253076] CPU1: Package temperature/speed normal
[ 2702.560290] mce: [Hardware Error]: Machine check events logged

Edit: There is a file /var/log/mcelog:

mcelog: failed to prefill DIMM database from DMI data
mcelog: Family 6 Model 3d CPU: only decoding architectural errors
Hardware event. This is not a software error.
MCE 0
CPU 1 THERMAL EVENT TSC 548f30962de
TIME 1451671461 Fri Jan  1 19:04:21 2016
Processor 1 heated above trip temperature. Throttling enabled.
Please check your system cooling. Performance will be impacted
Running trigger `unknown-error-trigger'
STATUS 88030803 MCGSTATUS 0
MCGCAP 1000c07 APICID 1 SOCKETID 0
CPUID Vendor Intel Family 6 Model 61
mcelog: Family 6 Model 3d CPU: only decoding architectural errors
Hardware event. This is not a software error.
MCE 1
CPU 0 THERMAL EVENT TSC 548f309c4d6
TIME 1451671461 Fri Jan  1 19:04:21 2016
Processor 0 heated above trip temperature. Throttling enabled.
Please check your system cooling. Performance will be impacted
Running trigger `unknown-error-trigger'
STATUS 88030803 MCGSTATUS 0
MCGCAP 1000c07 APICID 0 SOCKETID 0
CPUID Vendor Intel Family 6 Model 61
mcelog: Family 6 Model 3d CPU: only decoding architectural errors
Hardware event. This is not a software error.
MCE 2
CPU 0 THERMAL EVENT TSC 548f38f81e3
TIME 1451671461 Fri Jan  1 19:04:21 2016
Processor 0 below trip temperature. Throttling disabled
Running trigger `unknown-error-trigger'
mcelog: Too many trigger children running already
STATUS 88040802 MCGSTATUS 0
MCGCAP 1000c07 APICID 0 SOCKETID 0
CPUID Vendor Intel Family 6 Model 61
mcelog: Family 6 Model 3d CPU: only decoding architectural errors
Hardware event. This is not a software error.
MCE 3
CPU 1 THERMAL EVENT TSC 548f38fcc84
TIME 1451671461 Fri Jan  1 19:04:21 2016
Processor 1 below trip temperature. Throttling disabled
Running trigger `unknown-error-trigger'
mcelog: Too many trigger children running already
STATUS 88040802 MCGSTATUS 0
MCGCAP 1000c07 APICID 1 SOCKETID 0
CPUID Vendor Intel Family 6 Model 61

But this seems just a notice about a “normal” behaviour (as the result of modern and bad design of laptops). The fans are not strong enough for the thermal design of the CPUs, so they have to throttle to provide less heat. No reason to fail any gcc process, just need to wait a little longer. Or am I wrong?