State of the UBports installer, or command line

The ubuntu-device-flash appears to be an golang application, which can be compiled on many platforms. The compilation of an golang project should be as easy as install golang, use go get command to fetch-build-install. I tried only on ubuntu. Thus the compile process is a little tricky now.

Since they seem (to me) more concerned with making the installer look nice than getting it to actually work, I now added an automatic bug report to the collection with the same error message with a little further (Edit: and probably wrong) insight to GitHub … Fastboot: Unknown error: downloading 'boot.img'...OKAY [ 0.359s]booting...FAILED (remote: bootimage: incomplete or not signed)finished. total time: 0.361s · Issue #607 · ubports/ubports-installer · GitHub .

I think your approach is not corrected (tried it, too, BTW), because the installer works for me more than 50% of the time, using the exact same unsigned image. If not being signed was the root cause, there would be no FP2 running Ubuntu Touch, because they’ve never signed it, yet plenty of installations have succeeded.

My impression is that several people agree it is some kind of issue caused by running commands asynchronously to not block the UI. I’m not aware of having seen some detailed explanation, but by now I guess the last fastboot command (to reboot to recovery) is issued before the fastboot command to flash the boot image has finished. In that case the first part of “remote: bootimage: incomplete or not signed” would be correct.

1 Like

Yeah, I have seen that, too.
If that is indeed the cause I will instead blame them for not fixing the installer for months now, when a probable fix is known.
(Supposedly, this fork of the installer incorporates the fix, but I somehow couldn’t get it to run … GitHub - gronkern/ubports-installer: A simple tool to install Ubuntu Touch on UBports devices).

And I still think that we wouldn’t have all this trouble if they would prioritize function over eye candy, which should be a no-brainer.

You just have to compare the manual install scripts for Fairphone (Open) OS with the sources for the UBports installer … it would be funny, if it wouldn’t be so sad.

The 0.1.21-beta?
An older version of the installer was able to reboot to recovery on my phone, too … but this didn’t help to finish installing.

It seems nobody but that gronkern guy was able to get his changes to run.

To be honest, this “give $component some love” was a design only series. I tried to sneak functional fixes into it :wink:
In the telegram supergroup Florian mentioned that they might have someone to take over maintenance of the installer. I would contribute, but I find the code not-so-understandable.

I don’t think they are doing that with the installer. If you look at the last changes (Pull requests · ubports/ubports-installer · GitHub) most of them fixed some kind of errors in the installer or build setup or had functional enhancements like allowing to specify adb/fastboot when they are not in $PATH

Edit: the strange thing is, since 0.1.19 or so, the installer works for me most of the time. In contrast to other people’s experience I’m quite sure that 0.1.12 never worked for me (I always had to go back to 0.1.9).
Unfortunately there are subtle differences between linux distros. On opensuse the AppImage and deb versions for example can only work when adb/fastboot are in /usr/bin, having them in /usr/local/bin doesn’t work (that’s in $PATH for regular users but not for sudo; and it doesn’t help to run as root as the installer would still use sudo).

1 Like

I think they are doing exactly that by maintaining an Electron App monster for the nice GUI which doesn’t get the basic functionality right reliably on a “promoted” device … while not giving experienced users the manual install steps or providing a shell/ batch script which simply does the job and would be easy to troubleshoot.

1 Like

Ubuntu Touch is an community project. If anyone is to blame, than it is you for not providing an fix to the installer for months now, when a probable fix is known. The time you invest on automated bug reports and forum posts could be better invested to build and debug the installer, if it bugs you. Anyway, have a nice day. :slight_smile:

In essence you’re totally right, of course, but I’m pretty content with how I choose to spend my time.

To clarify, I didn’t create multiple bug reports, I just added one to the collection there with the same error. (I edited the wording in the post.)

I wouldn’t say much if the Fairphone 2 would not be called a “promoted device”. Either they can live up to their own promotion, or they could be more humble.
If you read a bit around here, you will find I can be very lenient and understanding when it comes to software shortcomings.
This “promoted device” thing just rubs me the wrong way when I look at the state of the installer.

Have a nice day, too :wink: .

I hope you don’t mind that I made a separate topic out of your posts. :wink:

IIRC there was (or is) some command line tool, but that’s just available on ubuntu and maybe also just for rather old versions, so that doesn’t help much.
So yeah, having some definitive steps how to do it manually would be very nice. And save lots of time for those who struggle with the installer.

But having some nice graphical user interface to install is not the worst idea, given that Ubuntu - at least in my recollection - started out to make linux easy (to install) for the average, not-so-tech-savvy user. So I can understand where the idea comes from to provide such an installer. But of course I agree that at the moment it’s by far not as helpful as intended.

2 Likes

Good idea, I brought some more posts along :wink: .

Hmmm … I tried it, and it looked more promising than the installer did … and seeing this again now it failed with “Can’t boot recovery image” in the end … perhaps this time a signed image could help :slight_smile: … but I didn’t find out where the tool put the images back then, I would have to look more into process monitoring (Process Monitor did the trick on Windows, afterall).
But I updated to Linux Mint 19.1 already, I would probably have to reinstall 18.1 for this.

Else … There’s a cli.js in the installer sources, but it is honest from the start …

“This tool is meant to be used for testing or for being wrapped by other scripts and programs. It is not meant to be a user-facing cli installation tool and does not try to be user-friendly.”

And … no kidding there :slight_smile: .

1 Like

I did today’s update not via updater app but with the installer.
Latest version 0.1.21-beta plus some additional log output (see my pull request at https://github.com/ubports/ubports-installer/pull/611).

Here are - most of - the steps I did:

  • use the AppImage version and run it as normal user on opensuse Leap 15.0[*]
  • the model of my phone wasn’t autodetected
  • selected the “16.04/devel” channel and kept everything else as default (I do QA on a test device, most regular users would want to go for “stable” when using as a daily driver)
  • I got “Fastboot: Unknown error: downloading ‘boot.img’…FAILED (data transfer failure (Protocol error))” while in fastboot mode
    • known issue: https://github.com/ubports/ubports-installer/issues/473
    • –> I clicked the “Try again” button and simply started over; didn’t happen again
    • (I think the error window also said I wasn’t using the latest version but that’s a message I ignore in general)
  • when instructed to boot into bootloader mode I did so with the indicated Power + Volume Down buttons
    • don’t use adb reboot bootloader from a different terminal, the UT installer uses port 5038 instead of the default 5037 so that a) the command would not work and b) a second running adb server will cause the installer to not see the phone in recovery mode
  • everything else went as expected

If someone wants to analyze what the installer does, here is the full log of the successful installation: https://paste.ubuntu.com/p/NvWw3Qf4Xs/

Executables for linux/windows/mac that include the additional (error) logging can be found at https://ci.ubports.com/blue/organizations/jenkins/ubports%2Fubports-installer/detail/PR-611/1/artifacts for now. I hope those changes will get merged to the official version soon.

[*]: opensuse doesn’t have /usr/local/bin in $PATH for commands run with sudo, so I created symlinks for adb/fastboot in /usr/bin (suse has not packages for those tools); if you don’t do that, the installer will fail to detect that the phone is in fastboot mode

1 Like

Looking at the log and what files are transferred with adb, these are the commands that should be possible to execute manually.

Boot to bootloader

fastboot flash recovery ~/.cache/ubports/images/FP2/recovery-FP2.img
fastboot flash splash ~/.cache/ubports/images/FP2/splash-FP2.img

Boot to recovery

adb push ~/.cache/ubports/pool/ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz /cache/recovery/
adb push ~/.cache/ubports/pool/ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/pool/device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz /cache/recovery/
adb push ~/.cache/ubports/pool/device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/pool/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz /cache/recovery/
adb push ~/.cache/ubports/pool/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/pool/version-697.tar.xz /cache/recovery/
adb push ~/.cache/ubports/pool/version-697.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/gpg/image-signing.tar.xz /cache/recovery/
adb push ~/.cache/ubports/gpg/image-signing.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/gpg/image-master.tar.xz /cache/recovery/
adb push ~/.cache/ubports/gpg/image-master.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/commandfile/ubuntu_commandFP29161 /cache/recovery/ubuntu_command

Got this by parsing the log I posted above via

grep "8 push" ~/.cache/ubports/ubports-installer.log | sed -e "s/.*exec cmd //; s/ -P 5038//; s/\\\\\"//g; s/\",\".*//"

But I’m not fully sure where to get these files from when not using the installer at all.
The fastboot images are specified at https://devices.ubports.com/api/installer/FP2.
Some of the actual install files come from http://system-image.ubports.com/ubports-touch/16.04/devel/FP2/index.json
Make sure to look at the entry with type: "full", otherwise the files starting with “ubports” are missing.
The gpg files come from http://system-image.ubports.com/gpg/
And the commands file is generated by the installer. For my last run it looks like this

format system
load_keyring image-master.tar.xz image-master.tar.xz.asc
load_keyring image-signing.tar.xz image-signing.tar.xz.asc
mount system
update ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz.asc
update device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz.asc
update keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz.asc
update version-697.tar.xz version-697.tar.xz.asc
unmount system

If you want to wipe the user data as well, add the line format data after format system.

So basically it should be possible to do all this without the installer now :wink:

3 Likes

I bundled all the files with a README and a linux shell script at https://www.dropbox.com/s/o9nu7xpvytpkguz/flash_ubports_fp2.zip?dl=0

Unzip the files, have a look at the README and the script. Worked for me :wink:

Edit: here is the full script so anyone could as well just copy the individual commands

#!/bin/bash

echo "Checking for connected Fairphone 2"

if !(fastboot devices | grep fastboot 2>&1 >/dev/null); then
    echo "Phone needs to be in bootloader mode (fastboot)"
    if !(adb devices 2>&1 >/dev/null); then
        echo "Reboot to bootloader and start again"
        exit 1
    else
         echo "Trying to reboot to bootloader (if that fails, reboot to bootloader manually)"
         echo "Start again when phone is in bootloader mode"
        adb reboot bootloader 2>&1 >/dev/null
        exit 1
    fi
fi

fastboot flash recovery ./images/recovery-FP2.img
fastboot flash splash ./images/splash-FP2.img
#fastboot flash splash ./images/splash-FP2_black.img

echo "Please reboot to recovery now"

adb wait-for-recovery

echo "Pushing 13 files to the phone to prepare installation"

adb push ./images/ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz /cache/recovery/
adb push ./images/ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz.asc /cache/recovery/
adb push ./images/device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz /cache/recovery/
adb push ./images/device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz.asc /cache/recovery/
adb push ./images/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz /cache/recovery/
adb push ./images/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz.asc /cache/recovery/
adb push ./images/version-697.tar.xz /cache/recovery/
adb push ./images/version-697.tar.xz.asc /cache/recovery/
adb push ./images/image-signing.tar.xz /cache/recovery/
adb push ./images/image-signing.tar.xz.asc /cache/recovery/
adb push ./images/image-master.tar.xz /cache/recovery/
adb push ./images/image-master.tar.xz.asc /cache/recovery/
adb push ./images/ubuntu_command /cache/recovery/
#adb push ./images/ubuntu_command_wipe /cache/recovery/ubuntu_command

echo "Rebooting to recovery now to start the actual installation"

adb reboot recovery
1 Like

Nice job @Ingo !
I’ll try to do the same exercise for stable release OT7 tomorrow :wink:

Why do you wait for a manual reboot to recovery?

adb wait-for-recovery

Is it possible to replace by fastboot reboot recovery or by fastboot reboot && adb reboot-recovery if the first doesn’t work?

“fastboot reboot recovery” doesn’t exist. fastboot can just reboot to fastboot again or to the full system. Or it can boot a recovery image, but this then would need to be signed (instructions how to do it float around the forum somewhere).

What could work is maybe

fastboot reboot
adb wait-for-device
adb reboot recovery
adb wait-for-recovery

But then, rebooting to the full system just to be able to reboot to recovery will likely restore the standard recovery.

P.S.: having this for OTA7 would be a nice idea. Then people could first install this and then use the updater to get the latest version. It would at least spare all the installer hassle :smiley:
I’m probably too lazy to do it again/keep the file updated :wink:

1 Like

With a OTA version each month at the quickest, it could be doable :smile:
EDIT: and it can be used as a workaround-solution only if installer fails!

EDIT 2: perhaps we can replace the (big) file folder by wget commands? So only the script has to be shared to people on this forum having trouble with the installer…

Sounds like we should set up a github project for that :wink:

Good idea, can you set-up that?

IMO the aim should not be to replace the installer, but give a backup solution.

I guess my time for such projects in January has run out :wink:
But if someone else wants to start, https://github.com/WeAreFairphone would be a good place to start.

I’m starting the installer-beta to get the log.
However I’m stuck at the “bootimage incomplete or not signed”. (bug https://github.com/ubports/ubports-installer/issues/184 )
I already clicked 2 times on “try again”…
How do you get trough that?

EDIT: is your commit implemented in the version PR 611 with additional log ?