Making a backup system image for recovery of FP 2 Open

Help Fairphone 2
, ,
1

Hi Forum,

What I want to do is create a backup system image of my fairphone 2 Open in current state.

I have just purchased a fairphone 2 and managed to install Open OS to get it google free and find alternatives for common google apps. It all worked out thanks to the topics in this forum!

Now I got stuck at the last but necessary step. To make a system image of the phone in this state, such that I can restore this image once the phone gets in unsolvable trouble or lost (and I have bought a new one). This image must be saveable on an external HD or computer.

I am sure it is possible with TWRP or adb and by rooting and booting. Many topics are around about it, but they are a bit to specialistic to me. I dont read in them that they do what I want, and just performing all the steps to see what happens seems to risky to me. My level of knowledge is: I used to know how image recovery works in windows environment.

The many and spread topics on the forum have confused me. Now I have been using the search function and still am looping into the same topics which I dont understand:

I started here: https://forum.fairphone.com/t/pencil2-backup-tools-for-fp-os/23749.

There the subject https://forum.fairphone.com/t/howto-use-twrp-without-flashing-it-e-g-for-backup/23747?u=paulakreuzer doesnt trigger me.

https://forum.fairphone.com/t/pencil2-recover-backup-user-data-fp-os-and-fp-open-os/20974 explains a bit more about booting and flashing and having software needed on an extra PC needed in the process.

https://forum.fairphone.com/t/complete-backup-tools-twrp-adb-and-others/23568 states exactly what i want: [quote=“Lomig, post:1, topic:23568”]
My wish would actually to have a “cloning” tool, so that I could easily make a backup (one or several files) that would be sufficient to recreate an exact replica of my phone and all its data and apps, in case the update would fail to the point that the phone gets hard bricked (which I experienced last July…).
[/quote] But the discussion does not clarify how to do this. I only understand it is about some detailed problem.

https://forum.fairphone.com/t/twrp-experiences-pros-cons/17215 also explains exactly what I want, but it is about some specific situations I dont follow, not really about how to use TWRP straighforward.

https://forum.fairphone.com/t/pencil2-using-twrp-on-the-fairphone-2/29452 claims to explain how to use TWRP, but I understand of it that it tells you how to install it, but not using it and how it works. The intalling process looks a bit risky to me. I am affraid I ruin the phone if I follow this.

Am I overlooking some thread ? Thanks In advance.

regards

1 Like
2

I am regularly using TWRP for making full backups (system image backups) of my FP2 running the FP Open OS.

First, you need to start TWRP by keeping volume up and power pressed for a couple of seconds during power on. If you boot it for the first time, it asks to enable write permission for the system partition. Please confirm this and afterwards, you just select “Backup”. Here you can change the name of the backup file, choose storage (internal or Micro SD) and so on.

Hope this helps. If you have any further questions, feel free to ask. :slight_smile:

2 Likes
3

AFAIK you already have TWRP installed when you’re on FP Open, as you stated. From within TWRP you can execute a system backup, which backups OS and other system related things, but not apps and data.

For the latter you need a backup app, like Helium or Titanium or similar.

When you don’t want to rely on an installed SD card, you could then execute a batch to transfer your backup archive(s) to your cloud server e.g. Nextcloud while being connected to Wifi. This can be done from a shell script from within Termux for instance, or from Tasker, to give some examples.

1 Like
4

Thanks @Juli_R, this works! Fairly simple after all.

However.

After that I tried to connect the FP2 to the computer via USB and see if I could trace back te backupfile. But the PC suddenly didnt see the FP2 anymore. After some searching in the forum I tried to switch on/off USB debugging mode in developer mode. Now USB works again. After that I even found the TWRP folder and started copying it: error copying file or folder.

I have had something like this before, with the whapsapp data folder, at some point the error disappeared. I think after shutting down whatsapp or something like that. I am going to hunt this down further tomorrow.

1 Like
5

Thanks @Amber , I’ll sort these tips out when I have progressed to that state of being.

1 Like
6

I don’t see why you should need USB debugging here.
Just slide down the notification area, select the notification “USB for charging” and change it to “File transfers”. Then you should be able to see your internal storage plus SD card on the PC and transfer the backup file.
EDIT: Regarding TWRP backup remember that it doesn’t backup /data/media directory (e.g. your photos a.s.o.). So you’ve to save this part also or, as a “workaround”, you’ve to rename this directory within TWRP before doing the backup (and rename it back after the backup!).

1 Like
7

Hi @Volker The problem was/is that there is no notification at all when I connect USB cable. And no charging. But it does charge when USB cable is connected to a power socket with USB interface. The same thing happens when I try another cable. Somewhere on the forum I found the following and it works for me:

Several times so far (today and yesterday) I have experienced that nothing happens when I connect my FP2 to my PC via USB. Doing the USB debugging trick then solves the problem.

1 Like
8

Regarding the question “what do I actually back up when I make a full system image backup with TWRP ?”, I found a helpfull article explaining the file/folder structure used by Android. And what partitions mean and which are used by android.

http://www.stevesandroidguide.com/android-files/

4 Likes
9

TWRP backups these things by default:

  • boot partition. This contains the Linux kernel executable (important) and some other data to boot the device.
  • system partition. The /system “folder” in Android contains the operating system without changes. That includes all the UI, system apps and how the system works, in general.
  • data partition. The /data “folder” in Android contains your system settings (wifi points, preferences, contacts, etc), those apps you have instaled, and the app data for your apps and system apps (chat conversations, notes, e-mails, app preferences, your favourite game’s records, etc).

TWRP saves each partition in a raw dump (a file you cannot open normally). The NANDroid backup (as we call the collection of raw dumps) is saved in the TWRP/BACKUPS/<device_code> folder in the FP2 internal storage (formerly called sdcard as in your linked article, but /storage/emulated/0/ for newer Android versions).

Because of that, TWRP doesn’t backup user data (internal storage, or /storage/emulated/0) because that will be recursive (saving internal storage in internal storage). You should transfer those contents to your computer with an USB cable via Android’s MTP (TWRP also has a built-in MTP, easier in my opinion).

7 Likes
10

All right. I have been working on it passed days. The last step for me was to transfer the by TWRP created image to my laptop via USB. This has succeeded, allthough I had this copy error at first hand:

But, in the end I was able to copy the files without error in two ways (as suggested by @Amber and @Roboe)

  1. long way: install Android studio and use the adb command in Command Prompt. Some remarks on this will follow below.
  2. short way:
    2.1 just reboot to TWRP
    2.2 connect USB cable, when TWRP has started
    2.3 windows recognizes the phone, and enables browsing the files (in the sdcard folder) via explorer, exactly as it does when Android is running on the phone.
    2.4 Select and copy the folder TWRP to your harddrive, now I do not get the copy error.

Copy error is probably omitted because Android is not running when TWRP is running. So Android can not occupy or block the files or something.

Remarks on long way: I was almost finished with method 1 (long way) when I discovered that method 2 (short way) worked. Method 1 was quite demanding. I did these steps:

  1. found this clear help: https://www.howtogeek.com/240655/how-to-copy-twrp-android-backups-to-your-pc-for-safe-keeping/, but bumped into some drawbacks.
  2. drawback 1: I couldnt install adb as explained in the tutorial since the mentioned installation files were not present in the zip download from the android developer website. So i then choose to install the Android Studio in stead, (which is over 1 gigabyte!) hoping that the adb tool would be included. Strangely enough I did not find adb.exe in the program folder (C:\Program Files\Android\Android Studio) after installation, but I did find it on another location: C:\Users\toon\AppData\Local\Android\Sdk\platform-tools. Were they there allready ?! I am not sure. But now at least it was there.
  3. drawback 2: Next step was to launch the Command Prompt window in the folder location were adb.exe was seated. But pushing shift + right mouse button in this folder gave me a thing called PowerShell which looked like command prompt but it did not accept the command “adb”. I learned that Command Prompt is replaced by PowerShell in the update to windows 10. But Command Prompt has not disappeared from windows 10, it is still there (type cmd in the start menu). But then I had to change the folder path to the location of adb.exe. That could be done by pasting “cd C:\Users\toon\AppData\Local\Android\Sdk\platform-tools” and hitting enter.
  4. Now all the commands worked as explained in the clear help. The TWRP folder was copied to my laptops harddrive.
3 Likes
11

Nothing weird with Android Studio and ADB. Android Studio requires and thus installs the Android SDK (in its own folder, that’s it). The Android SDK includes famous-between-tweakers adb and fastboot utilities.

Great you were successful and learned that much, :slight_smile:

12

I still a bit confused about the android filesystem.

On root level I see the partitions system, chache and data. The boot partition I think is equal to firmware(edit: not correct). But I also see many other folders (or partitions) and files on the same level:

type name
folder acct
folder cache
folder config
folder d
folder data
folder dev
folder etc
folder firmware
folder mnt
folder oem
folder persist
folder proc
folder res
folder root
folder sbin
folder sdcard
folder storage
folder sys
folder system
folder tombstones
folder vendor
file charger
file default.prop
file file_contexts
file fstab.qcom
file init
file init.class_main.sh
file init.environ.rc
file init.fp.camera_detect.rc
file init.mdm.sh
file init.qcom.class_core.sh
file init.qcom.early_boot.sh
file init.qcom.factory.sh
file init.qcom.rc
file init.qcom.sh
file init.qcom.ssr.sh
file init.qcom.syspart_fixup.sh
file init.qcom.usb.rc
file init.qcom.usb.sh
file init.rc
file init.recovery.usb.rc
file init.superuser.rc
file init.target.rc
file init.trace.rc
file init.usb.configfs.rc
file init.usb.rc
file init.zygote32.rc
file property_contexts
file seapp_contexts
file selinux_version
file sepolicy
file service_contexts
file ueventd.qcom.rc
file ueventd.rc

Now TWRP only backs up partitions called boot, data,system and cache. It looks like the other folders and files are omitted … How does this work ?

I would say that a complete backup of the phone would be a copy of everything underneath the root.

Also I would expect that the program TWRP itself and the bootloader software are somewhere amongst those folders. Is that correct ?

13

… as @Roboe explained before. Or do I misunderstand something here?

14

You are mixing up partitions with directories. A directory —specially in the root directory— could be a mount point for a partition, but it doesn’t need to be and could have another name.

This is the relation of relevant partitions and real folders in Android:

partition mount point
system /
data /data (installed applications and their internal data, system config, wifi keys, etc)
cache /cache
userdata /storage/emulated (internal storage: your photos, files, etc)
(don’t take it too literal: it’s /sdcard in TWRP and older Android versions)
boot not mounted when Android is up
recovery not mounted when Android is up

Note: Yes, a partition could be mounted in a directory (eg. /), and another partition could then be mounted on a directory inside the former directory (eg. /cache or /data).

So don’t worry at all: when TWRP backups your system partition, it is backing up all those files and folders, excluding data and cache —which are backed up on its own— or userdata —which you need to manually backup through MTP.

3 Likes
15

All right, thanks @AnotherElk and @Roboe, that has improved my understanding. Below I’ll add try to connect what I see on my phone to what I learned from you guys. I am sorry for the long text, but I try to rearrange all I understood so far. Maybe in the end I can summarize this all to make it usefull for other forum visitors.

THE STUFF I SEE AND WHERE IT RESIDES

So: the stuff I can see via windows explorer once my phone is connected over USB to my laptop is not part of the TWRP back-ups and must be copied manually, eg. using windows explorer (this is also called: copying files via MTP). The TWRP backup files themselves are also stored at this location, so they can also be copied manually. Lets call “the the stuff I can see” personal data and the place I see it personal data storage for now.

The personal data storage occurs at several places in the android file and folder tree. As far as I have experienced:

  1. //data/media/0
  2. //storage/self/primary (links to 6)
  3. //storage/emulated/0 (is sometimes empty)
  4. //sdcard (links to 2)
  5. //mnt/user/0/primary (links to 3)
  6. //mnt/user/0/primary/primary (links to 3)
  7. //mnt/sdcard (links to 4)
  8. …

I am affraid I can continue this list further if I look further. Anyway, this list shows that nr 2,4,5,6,7 act as shortcuts and in the end all point to 3. And none of them are pointing to 1. Also I saw that in some occasions nr 3 was empty or did not exist (depending on how you browse through the tree).

So now I am tempted to conclude that nr 1 is the original location for personal data storage, and 3 is a copy. The copy serves as an emulation of 1 for some apps for some reason.

Some supplementary fundamental problem on the side

Some supplementary fundamental problem on the side: especially the case that 2 points to 6, and 6 points to 3 is confusing. It conflicts with the definitions of a folder and a shortcut (as I know from windows). Since if you try to browse to 6 then you’ll get redirected to 3 before youve reached the final folder (exactly according to 5). So if you only walk the path of 5 you would say that 5 is a shortcut, and thus 6 can not exist. But then if you walk the path of 2 you end up in 6 meaning that 5 is a folder. I get some feelings of duality here.

THE PARTITIONS I FIND

To further understand the collection of partions I find on my fairphone I made a table of them aswell, using the app “Diskinfo” (Once I also read the great book “But how do it know” to have a better understanding of a computer anyway). Diskinfo tells me the following (sorted on type and then partition size):

nr partition name mounted to size type remark
1 Data /data 25,9 GB internal storage (MMC) contains apps and their data, system confi, wifi keys
2 System /system 2 GB internal storage (MMC) containing operating system
3 Cache /cache 640 MB internal storage (MMC) contains temporary data used by apps for faster loading
4 modem /firmware 64 MB internal storage (MMC)
5 persist /persist 32 MB internal storage (MMC)
6 boot not mounted 16 MB internal storage (MMC) containing linux kernel, only 16 MB?
7 recovery not mounted 16 MB internal storage (MMC) contains TWRP ?
8 splash not mounted 10 MB internal storage (MMC)
9 mmcblk0rpmb not mounted 4 MB internal storage (MMC)
10 modemst1 not mounted 1.5 MB internal storage (MMC)
11 modemst2 not mounted 1.5 MB internal storage (MMC)
12 aboot not mounted 1 MB internal storage (MMC)
13 pad not mounted 1 MB internal storage (MMC)
14 fsg not mounted 1.5 MB internal storage (MMC)
15 sbl1 not mounted 512 KB internal storage (MMC)
16 dbi not mounted 32 KB internal storage (MMC)
17 DDR not mounted 32 KB internal storage (MMC)
18 rpm not mounted 500 KB internal storage (MMC)
19 tz not mounted 500 KB internal storage (MMC)
20 fsc not mounted 1 KB internal storage (MMC)
21 ssd not mounted 8 KB internal storage (MMC)
22 /dev /dev 923 MB TMPFS mount points
23 /mnt /mnt 923 MB TMPFS mount points
24 /storage /storage 923 MB TMPFS mount points
25 /storage/self /storage/self 923 MB TMPFS mount points
26 RAM 1.8 GB Memory

There should be at least two types of memory. One type directly accessible for manipulation by the processor (the 1.8 GB RAM type) and one to store programs and their data (the MMC type). There are also 4 TMPFS mount point partitions in the list. I think they also reside on the MMC type of memory.

I assume for simplicity that this list of partitions occupies all the memory available in the phone. So there is no unpartitioned space left. This can be checked by adding all partition sizes: in my case they add up to 32 GB and 273 MB and 663 KB and some bytes. That matches not exact the 32 GB specified by fairphone, but closely. So this assumption seems correct.

So this also means that all other software than the Android operating system must be here. Also the software that runs before the operating system is started: this must be the boot loader software. Also I expect TWRP and Fastboot to be there. And maybe more. Each of them has probably their own partition, allthough their names are not reflected in the partion names.

The table also shows the mount points. So these must be the locations in the Android tree where the files and folders in the partition can be accessed once Android has started to run as operating system. Many of the partitions are not mounted. I see that most of the TMPFS partitions are mounted to the paths which I encountered before as being shortcuts to the location for personal data storage.

Conclusion 1
In the partition list I do not recognize explicitly the personal data storage. So I think this must then be part of the Data partition (nr 1). TWRP also includes the Data partition while creating the backup. But since the created backup files are stored in the personal data storage TWRP cannot include the personal data storage iself. So it includes everything in Data except for all data in personal data storage.

Conclusion 2
TWRP is not including all partions in the memory. Only a few (data, boot, system and cache). If the not included partitions get damaged or lost somehow then I dont have a backup for it.

Conclusion 3
I cant see any partition in the table mounted to the root level: “/”. Still, once I am in Android I see files and folders (which are not mount points for other partitions) on root level. So I dont know in which partition these files and folders are, but if I have to guess: it would be the system partion (nr 2).

16

I have thought of a more simple question to ask, for which I did not find an answer yet via duckduckgo or google:

On what partition is the root “/” mounted ?

The diskinfo app doesnt tell me, and the adb commandline command “df -F” (according to this website) is not recognized by adb on my FP2.

17

No partitions get mounted in / (note the reverse-wise formulation of the response). I guess because of the kind of files that it is kind of a “virtual filesystem” mounted from the initramfs that resides in boot.

Some clarifications (possible edits to your post):

Call it whatever, but its name is “internal storage” in the Android UI, since its not app’s data and doesn’t reside in the external SD.

All those double-slashes // are plain wrong. Should be single /.

Technical name is symlinks (symbolic links). Yes, those are folders that simulate another one in other location.

This is technically impossible (redundant symlinking).

Forget about that, Android is Linux-based. Filesystems are completely different. A shortcut in Windows is a file which gets interpreted specially when Windows is running (you can open a Windows shortcut with a text editor). A link in Linux/Unix (either symbolic or hard link) is part of the filesystem, whichever OS you happen to run over it.

Also please, take into considerations that the 0 in each of those paths is the ID of the user (principal user). If you happen to have multiple users, the ID changes, obviously.

Weird. Which file manager or terminal emulator did you use? That folder should be non-empty always.

As in any computer: storage (in this specific case in an eMMC basis, instead of a hard disk) and memory (random-access memory, RAM, volatile and quick).

Nop. TMPFS is a volatile filesystem that resides in the RAM.

All things system are in system, of course. The bootloader resides in /boot, and is a ad-hoc version of LittleKernel (LK) by Qualcomm for their Snapdragon SoCs. TWRP resides in the recovery partition. Fastboot is a low-level API for flashing or managing low-level things that I though resided in the SoC itself or some ROM, but seems to reside in the bootloader? This doesn’t have much sense because I wiped boot in some Android phones in multiple ocasions.

True. As TWRP puts it, all data except /data/media, :wink:

Those partitions are not meant to be edited and are not used directly in Android (= firmware for the radio and SoC). Anyway, if you need to restore them, each Fairphone update ZIP includes most of them.

1 Like
18

Meanwhile I have made below schema. I am quite comfortable about what and how to backup, allthough some questionmarks remain. See illustration below.

overview1
overview11473Ă—1701 221 KB

1 Like
20

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.