State of the UBports installer, or command line

Good idea, I brought some more posts along :wink: .

Hmmm … I tried it, and it looked more promising than the installer did … and seeing this again now it failed with “Can’t boot recovery image” in the end … perhaps this time a signed image could help :slight_smile: … but I didn’t find out where the tool put the images back then, I would have to look more into process monitoring (Process Monitor did the trick on Windows, afterall).
But I updated to Linux Mint 19.1 already, I would probably have to reinstall 18.1 for this.

Else … There’s a cli.js in the installer sources, but it is honest from the start …

“This tool is meant to be used for testing or for being wrapped by other scripts and programs. It is not meant to be a user-facing cli installation tool and does not try to be user-friendly.”

And … no kidding there :slight_smile: .

1 Like

I did today’s update not via updater app but with the installer.
Latest version 0.1.21-beta plus some additional log output (see my pull request at https://github.com/ubports/ubports-installer/pull/611).

Here are - most of - the steps I did:

  • use the AppImage version and run it as normal user on opensuse Leap 15.0[*]
  • the model of my phone wasn’t autodetected
  • selected the “16.04/devel” channel and kept everything else as default (I do QA on a test device, most regular users would want to go for “stable” when using as a daily driver)
  • I got “Fastboot: Unknown error: downloading ‘boot.img’…FAILED (data transfer failure (Protocol error))” while in fastboot mode
    • known issue: https://github.com/ubports/ubports-installer/issues/473
    • –> I clicked the “Try again” button and simply started over; didn’t happen again
    • (I think the error window also said I wasn’t using the latest version but that’s a message I ignore in general)
  • when instructed to boot into bootloader mode I did so with the indicated Power + Volume Down buttons
    • don’t use adb reboot bootloader from a different terminal, the UT installer uses port 5038 instead of the default 5037 so that a) the command would not work and b) a second running adb server will cause the installer to not see the phone in recovery mode
  • everything else went as expected

If someone wants to analyze what the installer does, here is the full log of the successful installation: https://paste.ubuntu.com/p/NvWw3Qf4Xs/

Executables for linux/windows/mac that include the additional (error) logging can be found at https://ci.ubports.com/blue/organizations/jenkins/ubports%2Fubports-installer/detail/PR-611/1/artifacts for now. I hope those changes will get merged to the official version soon.

[*]: opensuse doesn’t have /usr/local/bin in $PATH for commands run with sudo, so I created symlinks for adb/fastboot in /usr/bin (suse has not packages for those tools); if you don’t do that, the installer will fail to detect that the phone is in fastboot mode

1 Like

Looking at the log and what files are transferred with adb, these are the commands that should be possible to execute manually.

Boot to bootloader

fastboot flash recovery ~/.cache/ubports/images/FP2/recovery-FP2.img
fastboot flash splash ~/.cache/ubports/images/FP2/splash-FP2.img

Boot to recovery

adb push ~/.cache/ubports/pool/ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz /cache/recovery/
adb push ~/.cache/ubports/pool/ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/pool/device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz /cache/recovery/
adb push ~/.cache/ubports/pool/device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/pool/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz /cache/recovery/
adb push ~/.cache/ubports/pool/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/pool/version-697.tar.xz /cache/recovery/
adb push ~/.cache/ubports/pool/version-697.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/gpg/image-signing.tar.xz /cache/recovery/
adb push ~/.cache/ubports/gpg/image-signing.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/gpg/image-master.tar.xz /cache/recovery/
adb push ~/.cache/ubports/gpg/image-master.tar.xz.asc /cache/recovery/
adb push ~/.cache/ubports/commandfile/ubuntu_commandFP29161 /cache/recovery/ubuntu_command

Got this by parsing the log I posted above via

grep "8 push" ~/.cache/ubports/ubports-installer.log | sed -e "s/.*exec cmd //; s/ -P 5038//; s/\\\\\"//g; s/\",\".*//"

But I’m not fully sure where to get these files from when not using the installer at all.
The fastboot images are specified at https://devices.ubports.com/api/installer/FP2.
Some of the actual install files come from http://system-image.ubports.com/ubports-touch/16.04/devel/FP2/index.json
Make sure to look at the entry with type: "full", otherwise the files starting with “ubports” are missing.
The gpg files come from http://system-image.ubports.com/gpg/
And the commands file is generated by the installer. For my last run it looks like this

format system
load_keyring image-master.tar.xz image-master.tar.xz.asc
load_keyring image-signing.tar.xz image-signing.tar.xz.asc
mount system
update ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz.asc
update device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz.asc
update keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz.asc
update version-697.tar.xz version-697.tar.xz.asc
unmount system

If you want to wipe the user data as well, add the line format data after format system.

So basically it should be possible to do all this without the installer now :wink:

3 Likes

I bundled all the files with a README and a linux shell script at https://www.dropbox.com/s/o9nu7xpvytpkguz/flash_ubports_fp2.zip?dl=0

Unzip the files, have a look at the README and the script. Worked for me :wink:

Edit: here is the full script so anyone could as well just copy the individual commands

#!/bin/bash

echo "Checking for connected Fairphone 2"

if !(fastboot devices | grep fastboot 2>&1 >/dev/null); then
    echo "Phone needs to be in bootloader mode (fastboot)"
    if !(adb devices 2>&1 >/dev/null); then
        echo "Reboot to bootloader and start again"
        exit 1
    else
         echo "Trying to reboot to bootloader (if that fails, reboot to bootloader manually)"
         echo "Start again when phone is in bootloader mode"
        adb reboot bootloader 2>&1 >/dev/null
        exit 1
    fi
fi

fastboot flash recovery ./images/recovery-FP2.img
fastboot flash splash ./images/splash-FP2.img
#fastboot flash splash ./images/splash-FP2_black.img

echo "Please reboot to recovery now"

adb wait-for-recovery

echo "Pushing 13 files to the phone to prepare installation"

adb push ./images/ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz /cache/recovery/
adb push ./images/ubports-b68401f63e2fac3c8c533a804183ee276ee834b0154ca3ec20c80504dd17415c.tar.xz.asc /cache/recovery/
adb push ./images/device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz /cache/recovery/
adb push ./images/device-10ac49deff49c650c78d8c8139c6099a7e1058649a318dfc6f668a08fba9382a.tar.xz.asc /cache/recovery/
adb push ./images/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz /cache/recovery/
adb push ./images/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz.asc /cache/recovery/
adb push ./images/version-697.tar.xz /cache/recovery/
adb push ./images/version-697.tar.xz.asc /cache/recovery/
adb push ./images/image-signing.tar.xz /cache/recovery/
adb push ./images/image-signing.tar.xz.asc /cache/recovery/
adb push ./images/image-master.tar.xz /cache/recovery/
adb push ./images/image-master.tar.xz.asc /cache/recovery/
adb push ./images/ubuntu_command /cache/recovery/
#adb push ./images/ubuntu_command_wipe /cache/recovery/ubuntu_command

echo "Rebooting to recovery now to start the actual installation"

adb reboot recovery
1 Like

Nice job @Ingo !
I’ll try to do the same exercise for stable release OT7 tomorrow :wink:

Why do you wait for a manual reboot to recovery?

adb wait-for-recovery

Is it possible to replace by fastboot reboot recovery or by fastboot reboot && adb reboot-recovery if the first doesn’t work?

“fastboot reboot recovery” doesn’t exist. fastboot can just reboot to fastboot again or to the full system. Or it can boot a recovery image, but this then would need to be signed (instructions how to do it float around the forum somewhere).

What could work is maybe

fastboot reboot
adb wait-for-device
adb reboot recovery
adb wait-for-recovery

But then, rebooting to the full system just to be able to reboot to recovery will likely restore the standard recovery.

P.S.: having this for OTA7 would be a nice idea. Then people could first install this and then use the updater to get the latest version. It would at least spare all the installer hassle :smiley:
I’m probably too lazy to do it again/keep the file updated :wink:

1 Like

With a OTA version each month at the quickest, it could be doable :smile:
EDIT: and it can be used as a workaround-solution only if installer fails!

EDIT 2: perhaps we can replace the (big) file folder by wget commands? So only the script has to be shared to people on this forum having trouble with the installer…

Sounds like we should set up a github project for that :wink:

Good idea, can you set-up that?

IMO the aim should not be to replace the installer, but give a backup solution.

I guess my time for such projects in January has run out :wink:
But if someone else wants to start, https://github.com/WeAreFairphone would be a good place to start.

I’m starting the installer-beta to get the log.
However I’m stuck at the “bootimage incomplete or not signed”. (bug https://github.com/ubports/ubports-installer/issues/184 )
I already clicked 2 times on “try again”…
How do you get trough that?

EDIT: is your commit implemented in the version PR 611 with additional log ?

Pull Request 611 only contains the additional logging.
Pull Request 517 contains the attempt to fix the “bootimage incomplete or not signed” error plus most of the logging extensions. But other people’s test showed that it’s not fully working and introduced other errors.

What I don’t understand: I usually don’t have that many problems as are reported at github. The “bootimage incomplete or not signed” I only had once. And I think over time I used the installer successfully two dozen times or so.

1 Like

at least with PR517 it start the download of the big image :slight_smile:

Here is the terminal output (process still ongoing, but we can already identify the files that change compared to devel) :smiley:

debug:  url=https://system-image.ubports.com//pool/ubports-c34db5ecd943e8c6d003bb96d8825dfb06ebabe510f9d55b4467c944019afdd6.tar.xz, path=/home/olivier/.cache/ubports/pool, checksum=40d70ab2cdf28af24d738d0d7e53dceab4aa29aaa23460f1224c886c6a133d86, url=https://system-image.ubports.com//pool/ubports-c34db5ecd943e8c6d003bb96d8825dfb06ebabe510f9d55b4467c944019afdd6.tar.xz.asc, path=/home/olivier/.cache/ubports/pool, url=https://system-image.ubports.com//pool/device-9bd5166032690abaefde7030ec186b79a1ad9a677d38382c6346422bd8875dc7.tar.xz, path=/home/olivier/.cache/ubports/pool, checksum=6e1fb613e4e152e9afbf0ceb76a1e764a53bfb1ea4a14ad7a2c5d92f3541be34, url=https://system-image.ubports.com//pool/device-9bd5166032690abaefde7030ec186b79a1ad9a677d38382c6346422bd8875dc7.tar.xz.asc, path=/home/olivier/.cache/ubports/pool, url=https://system-image.ubports.com//pool/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz, path=/home/olivier/.cache/ubports/pool, checksum=5b6e870acf8944330acb2a9dad6f5270bd06c377603a3115f805290e989ab421, url=https://system-image.ubports.com//pool/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz.asc, path=/home/olivier/.cache/ubports/pool, url=https://system-image.ubports.com//ubports-touch/16.04/stable/FP2/version-5.tar.xz, path=/home/olivier/.cache/ubports/pool, checksum=dc69d850252c8f6dd0d396260a3dca482442e9fbd47001e9d14a02c1509210fe, url=https://system-image.ubports.com//ubports-touch/16.04/stable/FP2/version-5.tar.xz.asc, path=/home/olivier/.cache/ubports/pool, url=https://system-image.ubports.com/gpg/image-signing.tar.xz, path=/home/olivier/.cache/ubports/gpg, url=https://system-image.ubports.com/gpg/image-signing.tar.xz.asc, path=/home/olivier/.cache/ubports/gpg, url=https://system-image.ubports.com/gpg/image-master.tar.xz, path=/home/olivier/.cache/ubports/gpg, url=https://system-image.ubports.com/gpg/image-master.tar.xz.asc, path=/home/olivier/.cache/ubports/gpg

error: Not existing /home/olivier/.cache/ubports/pool/ubports-c34db5ecd943e8c6d003bb96d8825dfb06ebabe510f9d55b4467c944019afdd6.tar.xz
error: Not existing /home/olivier/.cache/ubports/pool/ubports-c34db5ecd943e8c6d003bb96d8825dfb06ebabe510f9d55b4467c944019afdd6.tar.xz.asc
error: Not existing /home/olivier/.cache/ubports/pool/device-9bd5166032690abaefde7030ec186b79a1ad9a677d38382c6346422bd8875dc7.tar.xz
error: Not existing /home/olivier/.cache/ubports/pool/device-9bd5166032690abaefde7030ec186b79a1ad9a677d38382c6346422bd8875dc7.tar.xz.asc
info: /home/olivier/.cache/ubports/pool/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz already exists with the expected checksum, so download will be skipped
info: /home/olivier/.cache/ubports/pool/keyring-4c4e7ef380ebcfa2c31084efa199138e93bfed8fc58aa3eb06bdf75a78af9b57.tar.xz.asc already exists with the expected checksum, so download will be skipped
error: Not existing /home/olivier/.cache/ubports/pool/version-5.tar.xz
error: Not existing /home/olivier/.cache/ubports/pool/version-5.tar.xz.asc
info: /home/olivier/.cache/ubports/gpg/image-signing.tar.xz already exists with the expected checksum, so download will be skipped
info: /home/olivier/.cache/ubports/gpg/image-signing.tar.xz.asc already exists with the expected checksum, so download will be skipped
info: /home/olivier/.cache/ubports/gpg/image-master.tar.xz already exists with the expected checksum, so download will be skipped
info: /home/olivier/.cache/ubports/gpg/image-master.tar.xz.asc already exists with the expected checksum, so download will be skipped
info: Starting download of 6 files
info: Downloading file, 1% complete

To summarise, only 6 lines have to be changed in @Ingo script, to go from devel to OTA-7 :slight_smile:

1 Like

Quick and dirty … I did the fastboot and adb stuff from the script on Windows with manually rebooting the phone between steps … and it worked.
Now how easy was that?
(Now I only need time to play around with UBports.)

Thank you so much for taking the time to figure this out.
KISS principle at its best :slight_smile: .

1 Like

@AnotherElk are you trying devel or stable ? :slight_smile:

devel, with no SIM card at the moment (it runs on my “spare parts” :slight_smile: ).

Could you please write the list of commands you ran, so that the next can save up time?

  • Connect the phone to your computer via USB and boot it into Fastboot mode
  • Download @Ingo’s flash_ubports_fp2.zip …
  • Extract the contents to a directory where you can find them
  • Open a command line and change into this directory
  • Make sure fastboot devices works at this stage (i.e. doesn’t just return an empty line)
  • Open the file flash_ubports_fp2.sh (the Linux shell script) in a text editor
  • Copy or write every line in the script which begins with “fastboot” into the command line and execute them
    For Windows change every “/” in directory paths to “\
  • Reboot the phone into recovery mode (should be the UBports recovery then)
  • Copy or write every line in the script which begins with “adb push” into the command line and execute them
    For Windows change every “/” in directory paths to “\
  • Execute adb reboot recovery
  • The phone should reboot and install UBports then.

Done (if no errors come up).

4 Likes

Thank you. At the end I used the script from Ingo, that I did not see, and it worked fine.

2 Likes