On my phone, I have a couple of directories with many files (~/DCIM/…, ~/Photos/…, ~/Music/… and such). Occasionally, I would like to synchronize them with my Linux Laptop.
For smaller chunks of data (pictures taken in one day), uploading them to a cloud service and then downloading to the computer works reasonably well. However, it comes with some caveats:
Hiccups can occur, upload over WiFi is not always robust. This can lead to directories being incomplete (which I occasionally only notice after weeks/months) or files broken. Fixing that can be complicated, because it needs manual review. Also, most cloud services are somewhat bad at keeping the creation dates and such meta data; this is bad if I “fix” errors later. Most cloud services allow pretty simple auto-uploading, but deleting/moving/renaming/replacing files will often cause even more hiccups. For non-negligible numbers of files and total file size, stability/reliability/performance are a concern.
For that, I can move all files on the internal SD card, then remove it, plug it in my laptop, and then synchronize all directories with a tool like rsync. That works, but is complicated and requires a lot of space on my SD card. I would like a similar solution, but using the internal storage (and the SD-card as well, ideally), via USB. Is there any written documentation about that anywhere that I am missing?
For the record: What I would like to do is
Make sure my computer has a backup, just in case
Plug the phone to the laptop via USB
Do some magic tricks to get it mounted as a drive
Run rsync --archive --update --delete /…/??? ~/Pictures/Phone or something, or use borg in a similar way
What I’d want from it is
skip files that already exist, especially when they are newer
make sure the creation date, access date and other relevant metadata is preserved (i.e. do not create the files on my laptop with today’s creation date, keep the original)
delete files locally that do not exist on my phone (ideally, give me options to exclude file patterns)
handle interruptions safely, allow me to re-start if connection is lost
Is there a good way to do that with Android? I am on a Fairphone 5 with Stock Android, not rooted, and on openSUSE Leap/Gnome, but I think this is more of a general question.
Well, with USB you have to use MTP, which just does not work reliably.
I’d suggest using SSH over Wi-Fi. For that try KDEconnect (as the app on the phone). Since you use GNOME, just ask Google, there is some GTK app for your Linux.
I do exactly that on my FP4 from time to time. I have installed termux (available both in the Play Store or FDroid) on the phone which gives a regular linux terminal. Inside termux I installed rsync, so I can run the following on my phone:
rsync -a --exclude=thumbnails /data/data/com.termux/files/home/storage/dcim/Camera/ ingo@192.168.178.11:/home/ingo/PhonePhotoBackup/
That copies over all photos to PC that I haven’t synced yet and doesn’t sync back photos from the PC that are not on the phone. If you want different behavior, I guess rsync does support it with the right combination of parameters.
For me it’s working fine over WiFi (and an ssh server on my linux PC).
IIRC I had to allow storage access for termux once and needed to find the correct path to the internal storage that termux can read. Over adb it would be /storage/emulated/0/DCIM/Camera but that wasn’t accessible from termux.
Haven’t looked into networking via USB cable (beyond “adb push/pull”) so far because I never had to. Might be possible, too, if WiFi isn’t stable enough.
When you plug in your phone through USB, there should be a notification telling you it’s charging through USB. Tap it, and you’ll be able to select file transfer. Now you can mount your internal storage on your Linux machine.
I use an Android App called “FTP Server” (by Banana Studio) to run a secure FTP session, and a custom Python backup script on the far side to recurse certain listed directories and copy everything not seen before. Additionally, it also supports FTP extensions such as checksumming the file remotely (i.e. not pulling the file down, then performing the checksum locally) for checking if a file has delta’d out from under you. I haven’t used that feature yet in automation, but have tested it manually, and it works. The app runs unrooted so can only see a subset of the filesystem, but it mostly grabs the things I don’t want going to the hive mind.
I’ve a separate utility to do iCal stuff, and that uses WebDav. I’m still trying to find an SMS backup solution though.
Yes, thank you. That’s a valid solution I use for copying a couple of files.
However, I was seeking for a solution that keeps creation data intact, only updates newer files, and deletes files deleted/moved. I think just drag-and-drop from Nautilus/Dolphin won’t solve this.
Just one note, in case you didn’t notice: Anything USB has one problem, above all stability issues of MTP and the like: Even the FP6 has only USB 2 like it’s predecessor. Yes, the chipset of the FP6 would be capable of delivering USB 3, but Fairphone did not implement that.
And, to make things worse, the standard Files app of Android to this date does not have a progress indicator and immediately goes into background execution, so when you copy big data, you never know how much has finished and what is remaining.
I’ve found that every time I’ve thought about using MTP, to the point where I don’t bother any more (not just with Fairphone, this seems to be a baked in Android issue across a variety of phones).
I’ve seen that there’s an rsync server for Android, but haven’t sunk much investigative time into it (yet).
And once you accessed the mtp in Nautilus, can you follow this suggestion for rsync? I haven’t tried it myself (although it sounds neat) and it reads like there’s a few gotcha’s, but it seems simple enough that it’s probably work with a bit of fiddling.
So far MTP is working for me on a Macbook Air running Mint Mate 22.3 with an FP6, but maybe because I am running /e/OS. I find that I have more issues with the standard Android connections.
I suspect /e/OS + Linux might be the deciding factor in reliability here. I’ve only used standard Android (I should consider switching, but finite time and all that), plus either Windows or Linux, and Android has been the common factor. I strongly suspect that Google have an incentive for you to use their tools, so it’s pretty much a baked on afterthought with very little support love given to it. But that’s just like, my opinion, man.
Mine is not a USB solution but it seems to do what you want. It works for me on on Linux Mint 22.3 and an FP3. The two devices talk to my Vodafone router and so are on the same network.
I use Cx File Explorer on the FP3 - free to download and use. The ‘Access from network’ option allows the Linux Mint Nemo ‘Connect to Server’ option to mount the FP3 as ‘pc on 192.168.1.16:2041’ with the usual copy, paste, delete, etc. options.
I sometimes use Warpinator - also free to download and use - to transfer small numbers of files between the two devices (and my Windows 11 pc) if I don’t need to delete, rename or whatever..
I use FreeFileSync - yet more software which is free to download and use - to sync the two devices. I only sync …/device and …/sdcard as that’s where my user data is. It takes less than a minute to transfer over 260MB.
FreeFileSync comes up with loads of ‘Not a directory’ errors for non-user files - e.g. …/sdcard/Android/data/org.mozilla.firefox but I have set FFS to ignore them. FFS comes with loads of options/settings which should handle what you want to do.
For me it meant creating a job called ‘backup_FP3’ to sync ‘/run/user/1000/gvfs/ftp:host=192.168.1.16,port=2041,user=pc/device’ (and …user=pc/sdcard) with ‘/home/john/live_files/backups/Fairphone/FP3_non_system/device’ (and ……/sdcard) in FreeFileSync. Once I’ve run Cx File Explorer and Nemo to mount the FP3 I just open FFS and double click on backup_FP3. No hassle. Get back to me if you want any more info.
I too use Syncthing, since years on phones and Linux (binary download, using web browser interface). Works great. The function of keeping old files versions in staggered version sis good and also helps like easy backup for inadvertent change or deletion on any system.
Until short time ago I used USB connect with my FP4 on e/OS/ with LMDE6 Linux. Since some time, I cannot verify when this effect came up, I after some time had to first reboot the handset to again get the USB connection setting offered in the pull down list after connecting the cable no matter to which of my Linux machines. Nothing too annoying though.
Anyway since you’re also using Linux, have you ever tried KDEconnect? It names “KDE”, but trust me, it runs well on Mint/LMDE/Android…
I found it by random, installed it and fell in love with it instantly. You won’t even thing again about plugin in any cable into your phone/Linux edge devices to get your personal data transferred between all registered devices within your (wireless) home network.
And this brilliant piece of FOSS code can even do a lot more. I believe once you tried it you won’t miss any other option for data transmission across different devices within your private network domain.
KDEConnect is really nice. Not for syncing though. But very nice for occasional file copy, and valuable for convenient clipboard sharing. Also you can from computer make your phone sound so you find where you put it (if in LAN wifi range…) And as already said, other functions.
Desktop agnostic. I have read it also works under Microsoft systems.