[HOWTO] ✏ Compiling Fairphone Open OS / Rooting

fp1_wo_sw_updates · January 13, 2016, 5:28pm

I added it roughly to the Wiki. @lklaus/@vivia, please correct/beautify!

vivia · January 13, 2016, 5:32pm

I just removed the “untested” warning. I tested it myself on two devices (both with unofficial images) and it worked, and @lklaus tested it on a device with the official image.

Ideally I’d like to create an already rooted image, but I can’t figure out how on earth to do it

fp1_wo_sw_updates · January 13, 2016, 5:34pm

Great, I was hoping you would report something like that, thanks! I guess I will include the repo apply script in the wiki, it’s useful.

vivia · January 13, 2016, 5:38pm

I just had an idea, there should be no blobs involved in the recovery image, right? So in theory we could make a blob-free recovery.img and distribute it?

fp1_wo_sw_updates · January 13, 2016, 5:47pm

Good question/idea. I’m not an expert. If a clean build without the blobs is successful? I think so

The recovery is Android Open Source Project/Apache 2. I’m sure there are nicer recoveries out there, but this one is as good as any to start with. A few checksums here (if a build without blobs is successful) will help as well.

vivia · January 13, 2016, 5:49pm

I don’t think checksums are usable, the build contains information about your username, computer name, date/time, kernel on your computer etc. But I might try a clean build without blobs just for the sake of it. It’s just a hassle to have one more directory of a clean repo sync

fp1_wo_sw_updates · January 13, 2016, 5:53pm

Docker might help while experimenting? (make clean and clean out /vendor/qcom/proprietary) fp2-sibon-2.0.0-blobs.sh is just a zipped tar file cp’ing stuff to ./vendor/qcom/proprietary/. The rest of the magic (that should not be used) is in ./vendor/qcom/proprietary/vendorsetup.sh

(Update, my time to look into this is up, but I’m sure most of it can be found in build/core/Makefile)

jftr · January 13, 2016, 6:18pm

What’s the problem with that? Isn’t everything in the repo supposed to be in git repositories? It should be fairly easy to store your changes (in a different branch or you can git stash your changes). Or you checkout to a different working directory which uses almost no additional disk space because everything is stored in git anyway and only hard-links are checked out.

root · January 13, 2016, 6:18pm

I understood that you might publish the hash of your compiled open source recovery IMG here before sharing it. To verify the downloaded IMG is OK.

Arvil · January 13, 2016, 6:40pm

A build without the blobs probably won’t work (because compilation will fail on one or several make targets). It’s also unlikely the recovery system will be free from licensed stuff since there are several sh files with “qcom” in them in the recovery “root” dir.

fp1_wo_sw_updates · January 13, 2016, 6:59pm

I did a quick “make -j8” here, looked okay, just a few standard unix tools getting complied. But maybe I’m missing a lot here, it was just a small random test without really looking into it.

Recovery image (in build/core/Makefile, search for # Recovery imag) and (bootable/recovery/Android.mk) or is there more? But @lklaus is our recovery expert, so I just wait for his answer.

berta · January 13, 2016, 7:20pm

Heyho,

I’ve same problems compiling the source.
I’m using Debian Jessie (8.2). I installed gcc-4.8 and tried to use update-alternatives. But I get this error: “update-alternatives: Fehler: keine Alternativen für gcc”.
I don’t konw wether this is a problem or not.

Why I’m writing is, that make ends very soon with the following error:
> Export includes file: external/openssl/Android.mk – out/host/linux-x86/obj32/STATIC_LIBRARIES/libcrypto_static_intermediates/export_includes
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:30: error: package org.antlr.runtime.tree does not exist
> import org.antlr.runtime.tree.*;
> ^
> Export includes file: external/zlib/Android.mk – out/host/linux-x86/obj32/STATIC_LIBRARIES/libunz_intermediates/export_includes
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/MismatchedTreeNodeException.java:30: error: package org.antlr.runtime.tree does not exist
> import org.antlr.runtime.tree.TreeNodeStream;
> ^
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/MismatchedTreeNodeException.java:31: error: package org.antlr.runtime.tree does not exist
> import org.antlr.runtime.tree.Tree;
> ^
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/MismatchedTreeNodeException.java:41: error: cannot find symbol
> public MismatchedTreeNodeException(int expecting, TreeNodeStream input) {
> ^
> symbol: class TreeNodeStream
> location: class MismatchedTreeNodeException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/Parser.java:30: error: package org.antlr.runtime.debug does not exist
> import org.antlr.runtime.debug.DebugTokenStream;
> ^
> Import includes file: out/host/linux-x86/obj32/STATIC_LIBRARIES/libunz_intermediates/import_includes
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/ParserRuleReturnScope.java:30: error: package org.antlr.runtime.tree does not exist
> import org.antlr.runtime.tree.CommonTree;
> ^
> Import includes file: out/host/linux-x86/obj32/STATIC_LIBRARIES/libcrypto_static_intermediates/import_includes
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/UnbufferedTokenStream.java:30: error: package org.antlr.runtime.misc does not exist
> import org.antlr.runtime.misc.LookaheadStream;
> ^
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/UnbufferedTokenStream.java:50: error: cannot find symbol
> public class UnbufferedTokenStream extends LookaheadStream implements TokenStream {
> ^
> symbol: class LookaheadStream
> Export includes file: system/core/adb/Android.mk – out/host/linux-x86/obj32/EXECUTABLES/adb_intermediates/export_includes
> Notice file: system/core/adb/NOTICE – out/host/linux-x86/obj/NOTICE_FILES/src//bin/adb.txt
> Notice file: system/core/libzipfile/NOTICE – out/host/linux-x86/obj/NOTICE_FILES/src//lib64/libzipfile.a.txt
> Notice file: system/core/libzipfile/NOTICE – out/host/linux-x86/obj/NOTICE_FILES/src//lib/libzipfile.a.txt
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:110: error: cannot find symbol
> if ( input instanceof TreeNodeStream ) {
> ^
> symbol: class TreeNodeStream
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:124: error: cannot find symbol
> TreeNodeStream nodes = (TreeNodeStream)input;
> ^
> symbol: class TreeNodeStream
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:124: error: cannot find symbol
> Notice file: external/zlib/NOTICE – out/host/linux-x86/obj/NOTICE_FILES/src//lib64/libunz.a.txt
> TreeNodeStream nodes = (TreeNodeStream)input;
> ^
> symbol: class TreeNodeStream
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:126: error: cannot find symbol
> TreeAdaptor adaptor = nodes.getTreeAdaptor();
> ^
> symbol: class TreeAdaptor
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:152: error: cannot find symbol
> else if ( this.node instanceof Tree) {
> ^
> symbol: class Tree
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:153: error: cannot find symbol
> this.line = ((Tree)this.node).getLine();
> ^
> symbol: class Tree
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:154: error: cannot find symbol
> this.charPositionInLine = ((Tree)this.node).getCharPositionInLine();
> ^
> symbol: class Tree
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:155: error: cannot find symbol
> if ( this.node instanceof CommonTree) {
> ^
> symbol: class CommonTree
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:156: error: cannot find symbol
> this.token = ((CommonTree)this.node).token;
> ^
> symbol: class CommonTree
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:171: error: cannot find symbol
> else if ( input instanceof TreeNodeStream ) {
> ^
> symbol: class TreeNodeStream
> location: class RecognitionException
> Notice file: external/zlib/NOTICE – out/host/linux-x86/obj/NOTICE_FILES/src//lib/libunz.a.txt
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:172: error: cannot find symbol
> TreeNodeStream nodes = (TreeNodeStream)input;
> ^
> symbol: class TreeNodeStream
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:172: error: cannot find symbol
> TreeNodeStream nodes = (TreeNodeStream)input;
> ^
> symbol: class TreeNodeStream
> location: class RecognitionException
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/RecognitionException.java:173: error: cannot find symbol
> TreeAdaptor adaptor = nodes.getTreeAdaptor();
> ^
> symbol: class TreeAdaptor
> location: class RecognitionException
> Notice file: external/openssl/NOTICE – out/host/linux-x86/obj/NOTICE_FILES/src//lib/libcrypto_static.a.txt
> Export includes file: system/core/libbacktrace/Android.mk – out/host/linux-x86/obj32/SHARED_LIBRARIES/libbacktrace_intermediates/export_includes
> Export includes file: system/core/libbacktrace/Android.mk – out/host/linux-x86/obj32/SHARED_LIBRARIES/libbacktrace_test_intermediates/export_includes
> Export includes file: external/gtest/src/Android.mk – out/host/linux-x86/obj32/STATIC_LIBRARIES/libgtest_host_intermediates/export_includes
> Export includes file: external/gtest/src/Android.mk – out/host/linux-x86/obj32/STATIC_LIBRARIES/libgtest_main_host_intermediates/export_includes
> Export includes file: external/libunwind/Android.mk – out/host/linux-x86/obj32/SHARED_LIBRARIES/libunwind_intermediates/export_includes
> Export includes file: system/core/liblog/Android.mk – out/host/linux-x86/obj32/SHARED_LIBRARIES/liblog_intermediates/export_includes
> Export includes file: external/libunwind/Android.mk – out/host/linux-x86/obj32/SHARED_LIBRARIES/libunwind-ptrace_intermediates/export_includes
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/UnbufferedTokenStream.java:50: error: UnbufferedTokenStream is not abstract and does not override abstract method range() in TokenStream
> public class UnbufferedTokenStream extends LookaheadStream implements TokenStream {
> ^
> Import includes file: out/host/linux-x86/obj32/SHARED_LIBRARIES/libunwind_intermediates/import_includes
> external/antlr/antlr-3.4/runtime/Java/src/main/java/org/antlr/runtime/UnbufferedTokenStream.java:75: error: cannot find symbol
> public int LA(int i) { return LT(i).getType(); }
> ^
> symbol: method LT(int)
> location: class UnbufferedTokenStream
> Note: Some input files use unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> 23 errors
> build/core/host_java_library.mk:84: recipe for target ‘out/host/common/obj/JAVA_LIBRARIES/antlr-runtime_intermediates/javalib.jar’ failed
> make: *** [out/host/common/obj/JAVA_LIBRARIES/antlr-runtime_intermediates/javalib.jar] Error 41
> make: *** Warte auf noch nicht beendete Prozesse…
> Note: Some input files use unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.
> Note: Some input files use or override a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> Note: Some input files use unchecked or unsafe operations.
> Note: Recompile with -Xlint:unchecked for details.

> #### make failed to build some targets (01:45 (mm:ss)) ####

The whole output can be found here: http://pastebin.com/03QG3qvv.

Thank you and best wishes, berta

lklaus · January 13, 2016, 7:38pm

Hi,
I’m a bit at a loss here, I don’t know what the question is…

The concept behind my modifications to the stock recovery:

deactivate zip file verification (in install.cpp in recovery)
at least SuperSu in its install script mounts partition,so it needs an fstab, and does not look for recovery.fstab. I work around this by making a link from recovery.fstab to fstab (in the Makefile)
As the install skript is a shell skript, we need an sh interpreter in the recovery. I had a compiled busybox left over from some tests with TWRP. I saved the executables and libraries and made the rcbins.tar. This has to be untarred at some place (preferably $HOME) and will also be copied to the sbin from the recovery
as the busybox binaries from TWRP are dynamically compiled, the system needs to know where the libraries are. This is done by the modification to init.rc after “on init”

edit: just checked the wiki. As vivia specifies a location for rcbins.tar and delivers the Makefile, follow the wiki post and leave my remarks here as a general explanation to know whats happening

fp1_wo_sw_updates · January 13, 2016, 7:40pm

The question from @vivia was if it is okay to share the recovery.img, or if it does contain bin blobs code. The Makefile is so big … and stuff gets copied around so it’s hard to tell without a clean (non-bin-blob) build. So the question is: Does the recovery contain any bin blob code?

Jannis · January 13, 2016, 7:42pm

Looks like I managed to compile the source now (on Gentoo btw). The “API was modified” errors are gone, didn’t do anything different than last time, just started from scratch.
Before I flash the system.img here’s one question:
I downloaded the original OTA-images (“FP2-gms36-1.1.7-ota.zip”) has a “system.new.dat” instead of the “system.img”. Can I use fastboot to flash the “system.new.dat” to restore the original ROM?

Oh wait, let me guess: you boot recovery and use adb sideload with the “FP2-gms36-1.1.7-ota.zip” (without unzipping it on the PC). Right?

lklaus · January 13, 2016, 7:42pm

Ah, I see. I do thinks that there are at least fragments from the blobs, as @Arvil also stated, there are files with qcom in it. As I do have a very slow system for compiling at the moment (have to switch to docker…) it’s not that easy to set up everything without blobs and issue a make recoveryimage to see whether it compiles or not. But I doubt it…

fp1_wo_sw_updates · January 13, 2016, 7:50pm

Looks like java is not happy, but I’m not an expert. What versions are active on your Debian box?
$ java -version && gcc --version

Else, there is always dockerfile from @jftr

lklaus · January 13, 2016, 7:51pm

You might want to look in this thread: softbricked fp2. There’s a discussion about the suitability of the ota package

berta · January 13, 2016, 7:57pm

java -version && gcc --version:
java version "1.7.0_91"
OpenJDK Runtime Environment (IcedTea 2.6.3) (7u91-2.6.3-1~deb8u1)
OpenJDK 64-Bit Server VM (build 24.91-b01, mixed mode)
gcc (Debian 4.8.4-1) 4.8.4
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

And I took a serious look into fairphone2-build-env/Dockerfile at master · justfortherec/fairphone2-build-env · GitHub

berta

P.S. Maybe the CLASSPATH is wrong, but I don’t know why it should just in my case.

fp1_wo_sw_updates · January 13, 2016, 8:56pm

Good, that was the idea! But … as a stronger workaround you could also just use docker with the dockerfile, if everything else fails!

It works pretty well. You can just recycle your current working directory for that, so there would be no need to checkout all the code again, if you just want to build the image quickly.

Else, nothing comes to mind. Is javac -version the same version? Sometimes on Debian you can end up with different versions due to the alternatives management. Maybe someone else will come along and will be able to help you!

ls -al /etc/alternatives/javac should point to java-7-openjdk-amd64/bin/javac somehow.