File acces to SD card from computer

[RobertCailliau]

I have a Fairphone 4 running Android 12.
I would like to update the files on the FP4’s SD card from my laptop which is an Apple M2 machine running Ventura (OS13.3).

But I have had no success over several days spent on this, hence my request for help. There are a lot of details following, I hope they make it clear what is needed, and avoid standard answers.

Note: it is a 1TB card, carrying most of my documents, over 300GB of data, so it is out of the question to keep that up-to-date manually, it needs a program working on a whole list. Also, I don’t need to sync: only copy from the laptop to the SD card. And not all file types, e.g. files in a proprietary format for which no application on the FP4 exist, should not be copied. I have therefore a program that copies only desired files in a batch operation (those of desired types and modified since last intervention). That works fine if the program has access to the SD card either through the FTP protocol or because it is mounted as an external disc.

I have tried numerous ways to copy files to the SD card:

1. taking the SD card out and sticking it into a card reader; this works but is obviously unacceptable because I have to open the FP4, which even implies taking the battery out.

2. using Android File Transfer: fine for a few files and manual manipulations, but can’t be used for bulk transfer and is not scriptable. Not acceptable.

3. using one of the numerous sync applications, but they do not allow specifying types to sync. Some don’t even show the SD card. Not an acceptable solution.

4. running an FTP server on the phone. That would be workable, but none of the installations I tried fuctioned, probably because of restrictions in Android 13. Many recipes available are also old, and therefore do not take recent changes into account. The “termux” recipe I found refers to a command not available. “Primitive FTP” did not work (did I forget to switch some mode/permission on Android?)

5. mounting the SD card as an external disc, accessible directly from the Apple Finder. That would be the best solution, but none of the recipes I tried work. There is “MTP for Mac”, which needs switching off certain protections. I don’t mind, and did, but it crashed my machine with kernel panics. The one time it did work it was exactly what I expected and worked OK. But it conflicts with the OS and crashed my machine consistently, probably also because (I suspect) it is Intel code, not M2.

For reasons of speed I would obviously prefer a connection via USB-C cable, but WiFi would be acceptable.

Let me repeat:

• transfer is to the SD card, not the built-in memory
• sync is not necessary, only copy one way
• only files modified after a given date should be transferred
• only files of a specified set of types should be transferred
• FTP would work
• mounting the SD card in the Finder is preferred

Hence the solution is either mounting the card as external disc, or running an FTP server on Android 13.

Thanks for any help.

[yvmuell]

@RobertCailliau

on Android 13.

Which OS actually is installed? Cant be FPOS as its on Android 12 so far.

[hirnsushi]

@RobertCailliau

using one of the numerous sync applications, but they do not allow specifying types to sync.

Syncthing lets you set very fine grained rules which files shouldn’t be synced.

I’ve been using it for years and it generally just works™, no experience with the MacOS version though, YMMV.

[RobertCailliau]

You are right: my mistake (corrected now) it is 12, not 13. Typo. Sorry.
R.

[RobertCailliau]

Thnks for the hint, but this is what it does on mac 13.3:

no window, and it does not even quit, does not show up in the emergency-quit either. I had to use the Activity Monitor and force it to quit from there (plus its associated demon).
Sigh.

[Vinamis]

Suggestion :
Do all the process on your Mac.
Simulate your SD Repository content.
Update this.
Replace your SD content with this new Repository content.

[hirnsushi]

There are other users with similar issues, could be a M1/2 problem. Some people seem to have solved it with a direct download instead of homebrew, if you haven’t tried that already that might be an option.
The GUI is just at http://localhost:8384/ so all that indicator would do is tell you the current state and let you open it in a browser.

Normally I’d suggest just using adb push to get files to the phone, compared to the awfully unstable MTP that at least works reliably, but AFAIK Google hasn’t updated their tools for ARM Macs yet.
(Maybe it’s only fastboot that’s missing though, worth a check)

[RobertCailliau]

Vinamis:
but does that not imply taking the sd card out of the phone?
My repository is my entire computer’s user home directory.

Hirnsushi:
I have in the meantime found that all settings on the FP4 are correct: connecting it to a friend’s Windows/Intel machine isntantly mounted the internal and SD in the Explorer. So it is only on MacOS it goes wrong.

As you say, it is probably an M2 problem.
So, for the time being, it looks like running an FTP server on the FP4 would do it. So far no success with that, but I’ll try again.

[Vinamis]

Robert,

What I mean :

You copy using your selective program, from the computer to a “SD LOCAL” directory also on the computer (this directory already contains the equivalent of the content of the FP4’s SD card; the physical SD card remains in the FP4).

After updating “SD LOCAL” on the computer, you just have to transfer “SD LOCAL” to the physical SD card.

If I understood the original request correctly.

[RobertCailliau]

You write: “this directory already contains the equivalent of the content of the FP4’s SD card”
but that is 380GB.
Then, even if I did that, how would one transfer it to the SD card? With Android File Transfer, that could in principle be done in one single manipultion, as a folder, but it would take ages! I checked: the entire folder gets replaced by Android File Transfer, not just changed items.
Plus, of course, the copy would itself be 380GB big, taking a quarter of the space on my laptop.
So, unless we find something that runs reliably on MacOS Ventura on an M2 machine, it seems the solution is either mounting the FP4 as an external disc, or running an FTP server on the FP4.

Android does not support a number of protocols natively, and it’s not just file access. I had similar trouble finding an app that runs CalDAV and CardDAV. DAVx5 does the trick for that.
But this seems to be a refusal from the part of MacOS somehow.

[RobertCailliau]

The app “FTP Server” does what I want, but it seems to get access only to the main storage, not the SD card for some reason.
Some other ones might work (e.g. File Explorer) but they are not compatible with Android 12.

[Vinamis]

Completely agree with you.
I don’t know the Mac features and of course a large capacity SD will take time to transfer. The end justifies the means.
In principle the Mac is faster for processing. The rest is just transfer speed.

[RobertCailliau]

Hm. It seems I have a few ftp server apps that would work. But somehow I can’t give them access to the SD card. When I try, I get this:

(this looks like it is the internal storage, but it is not, and exactly the same comes up for the sd card as for the internal storage.)

I have tried to format the card in other formats, but (of course?) never get any indication what anything really means. There must be a simple permission somewhere, because apps like File Manager do have read/write access to the SD card.
So? Anyone?

[hirnsushi]

The app has to ask for general file access, otherwise it will only be able to use folders you specifically select / create for it. That’s a security limitation in newer Android versions.

If the app hasn’t been updated in a while, or doesn’t have this permission enabled (reserved for file managers and similar), your only other option is root.

Which app are you using and have you had a look at their bugtracker for open issues?

[RobertCailliau]

Thanks. Let’s see if anything is understandable at all:

(1)The app has to ask for general file access,
Quite, indeed, that is understandable.

(2) Which apps:
(a) FTP Server (free) : this seems not to be maintained atg all since 2020, so let’s not talk about that one.
(b)primitive ftpd: seems OK, but it gives this:

Now, to me this ins NOT understandable. In the screenshot it is set to “plain old file system, the next option is “Super User” which I don’t want to do because it means to “root” my device; the other three options are Android jargon that is obscure at least. If I choose what looks to me the obvious one, i.e. Android Storafe Access Framework, then I immediately get this:
(as in previous message):

I’ll spare you my comments, because it is not at all obvious that that page is a system message, and does not come from the app itself. In addition there is no indication whatever about what I can do to lift that restriction. There is a menu on the top left, and if I click that, I can change to the SD card, but I get the same message: “Can’t use this folder”.

Note also that on the storage type choice list, there is a link “See our github page”. But that page then says “… In order to access all files… you may have to grant “all files access” permission in Android settings.
The page is here:

GitHub - wolpi/prim-ftpd: FTP server app for android

and at the bottom it says explicitly it is not compliant, but also that I should install from f-droid. Which I did.
Yes, well, so I followed those instructions, and now have:

Great. Back to the app, set “plain” and using Cyberduck I can access the internal storage but not the sd card, and when I set the app to “Android storage access” I still get the message that the folder can’t be used…

That github page is as unintelligible as all the other stuff: is there now a way to do it or not?
”experimental computing” is not a good way to make progress.

I’ll keep trying to find either an ftp server or a way to mount the FP4 as an exgternal disc.

[hirnsushi]

In the folder selection dialogue, open the sidebar and select one of the subfolders.
Android won’t let you choose the root of internal storage / SD card, even with full access (mainly because that access still doesn’t include restricted folders like Android), but you can just create your desired folder structure one level deeper.

To quote the GitHub Page:

Optional support for Android Storage Access Framework to access external sd-card the official way (NOTE requires selecting a directory, not the root of the sd-card).

[RobertCailliau]

Hirnsushi,

Figure that I had already tried that, and indeed several times! But without any success.
In the meantime I also tried ”WiFi FTP” and even bought the pro version. That also failed miserably.

BUT, then your message came, and in desperation I tried again to choose a folder instead of the root. For totally unknown reasons, this time, choosing the folder inside the SD card (and, as before, NOT using the root of the SD card), Primitive FTP asked to allow access!!
But it never did so before, neither on the SD card nor on the built-in storage, (when choosing a folder instead of the root).
Then, in Cyberduck, strange things showed up (things that should have been folders but were binary files and such). But they went away too.

However, the connection then succeeded, and, for the time being, it seems to work!

I tried LiveCode statements like:

put url("binfile://Volumes/Ventura/Users/robertcailliau/Desktop/zonk.jpeg") into url("ftp://192.168.123.3:2123/A/C/zonk.jpeg")

and that worked as expected. It even created the intermediate directories A and C on the SD card.

So, it seems that thanks to you the problems are solved. Especially thanks for spotting the requirement to use a folder and not the root, even though I had tried that.
I will keep you posted about real results. It all seems not very reliable, but we’ll see.

Robert.

PS: my program does this:

• it knows the last date/time when I put changed files onto the SD card.
• it examines all desirable files (pdf, html, jpeg, …) among the over 350’000 that are on my laptop’s home directory, to see which have changed since that date/time.
• it then issues for each such file a command like the one above to put the changed file onto the SD card.

It takes a little more than 30 seconds to look at those 350’000 files, and it comes up with between 20 and 50 files that need to be copied, if I do it every day.
(note: as stated before, those 350’000 files take up about 380GB)