Magisk module for (system-)camera apps

Smojo · August 17, 2022, 10:33pm

So basically I can create a folder for the system app I want to install via Magisk module (here OpenCamera).

Correct me if I’m wrong @hirnsushi. I just need to create a module like this (this time there is no overlay/replace needed, as it is a complete new folder so I will let Magisk “merge” the files and folders):

adb shell su -c mkdir /data/adb/modules/OCSysApp`
adb shell su -c mkdir -p /data/adb/modules/OCSysApp/system/app/OpenCamera
adb push '/<myPCdir>/OpenCamera/base.apk' '/data/adb/modules/OCSysApp/system/app/OpenCamera'

'+ reboot?

I was also wondering what is the difference between the /system/app folder and the /system/priv-app folder. (also app and priv-app exist under /product [linked under /system/product]) again. )

EDIT: Still not sure were I will put it, but this explains the different places quite good: source.android.com - bootloader - partitions

→ I think <my-module-path>/system/product/app/<my-system-app> is a good place for it.

hirnsushi · August 18, 2022, 4:55pm

I’ve moved your post to a separate topic to not derail the Calyx thread. Feel free to modify the title.

At this point you might want to have a look at the Magisk documentation, this is getting too involved to create on the phone. Especially since you can’t adb push to /data/adb/modules without adb root, you would have to take the /data/local/tmp detour.

For a simple module all you need is the folder structure outlined in the docs. Make sure you have everything mentioned here in place as well.
Check the file / folder permissions, or, for good measure, add set_perm_recursive $MODPATH 0 0 0755 0644 to your customize.sh. I like to include a message in there, like ui_print "Here we go", so I know it’s actually been executed.
Then just zip everything up from the root of your module zip -r0q foo.zip ./*, push it to your phone and install it through Magisk.

Smojo · August 19, 2022, 1:33pm

Thx, yes that make sense.

I first thought it is the better place, as a lot of Calyx adopted/modified stuff, they just modified, is in there (like Etar). When I read the other thread(s) the probably better place is <my-module-path>/system/priv-app/<my-system-app> (and I don’t know what is bound to priv-app folder and potentially needed here, so maybe lets stick to that one for now)

You are probably right preparing a zip and add it in Magisk makes more sense, if the structure is a bit more complex (even this might still be a “hallo world”-like-module).

Regarding module structure it would look like this (afaics right now - maybe other stuff has to be added):

/data/adb/modules/CamAsSystemApp/ <--- my $MODID)
├ META-INF/com/google/android
| ├ update-binary      <--- The module_installer.sh you downloaded
| └ updater-script     <--- Should only contain the string "#MAGISK"
├ customize.sh  <--- will contain the recursive permission setting
├ module.prop <--- some words what it is, to identify better if I create more of these modules some day ^^
├ system/priv-app/
| └ OpenCamera/ <--- lets start with this, as I the details about the stock camera confuse me a bit right now, also I can easily get the OC apk)
|   └ base.apk
├ /system/etc/permissions/
| └ permissions.xml <--- not sue if this has to have a specific naming - the example of xlbax was `privapp-permissions-com.fp.camera.xml` so mayby `privapp-permissions-<packagename>.xml` or it doesn't matter, as the package name will be mentioned in the xml again as well

In general, what happens if my package is not valid (syntac error or whatever)? Will Magisk tell me during loading the module, or is there a risk that I will screw up my device?
Just asking, as I only have one FP4 right now and it is my daily driver as well (with the FP2 at one point I bought a second one so it was more safe to play around).

hirnsushi · August 19, 2022, 2:53pm

That’s the perfect place, the camera app needs to be in the priv(ileged)-app folder to get access to the system cameras, since we aren’t building the ROM and don’t have the option to sign it with the proper certificate.

That’s only the first setup, it’s way easier to test changes that way. Just run zip again, push it to the phone and install it.

There doesn’t seem to be a strict naming convention for this (other than that it starts with privapp-permissions-), but if you have a look at /system/etc/permissions/, single apps just use their package name, works fine for me as a suffix.

I would actually suggest starting with FPCamera for your first module, the benefit is you get to look at how it’s implemented in stock, and you don’t have to patch SYSTEM_CAMERA support into the .apk, one less potential point of failure.
You’ll need some tools, both simg2img and lpunpack (part of android-tools in my distribution) or you can use imjtool.

Get the latest factory images and extract them
Use simg2img super.img super.raw.img to create a proper super partition
Extract the system partition lpunpack -p system_a super.raw.img
[Optional] if you are using imjtool, just use imjtool name_of.img extract twice, once for super.img and then for the extracted image.img
At the end mount your system_a.img and copy everything you need over to your module directory

Magisk will tell you if there are problems during installation, but it’s not checking the files in your /system folder for errors. You can screw up your device if you overlay / add the wrong thing, usually those problems can be fixed by booting into safe mode (disables all modules) or following the troubleshooting guide, worst case scenario factory reset / reinstall.
I haven’t had to resort to any of that with the FP4 so far (well I did have to reflash it after disabling microG, as expected), but it’s always a possibility. Now would be a good time to check your backup strategy

Smojo · August 21, 2022, 7:36am

To flash Calyx I needed to replace the platform-tools (normally I only use adb and fastbook anyway), which were part of my distribution
→ PureOS → a based on Debian 11 (Bullseye) based OS
When I just install the version was too outated:

/usr/lib/android-sdk/platform-tools/adb --version
Android Debug Bridge version 1.0.41
Version 28.0.2-debian
Installed as /usr/lib/android-sdk/platform-tools/adb

So I manually downloaded the Android SDK which I use as default now:

adb --version
Android Debug Bridge version 1.0.41
Version 33.0.2-8557947
Installed as /AndroidSDK/platform-tools/adb

Anyway platform-tools seem to be not the same as android-tools as I don’t see simg2img or lpunpack in there. My distribution only offers a package called android-tools-mkbootimg

> apt-cache search android-tools
adb - Android Debug Bridge
android-tools-mkbootimg - transitional package
fastboot - Android fastboot tool

Not sure if this is the same but I can also try the imjtool.

Apart from that I saw that it is (was) also possible to clone this git repo to get the basic structure of a module: magisk-module-template/README.md at master · oasisfeng/magisk-module-template · GitHub
But it seems to be outdated so let’s stick to the guide and create the folders and files manually.

hirnsushi · August 21, 2022, 5:12pm

You can build android-tools yourself if you want. I’ve just tried the nix package (always wanted to play with nix anyway), that works great as well.
Or use imjtool, just a simple download and extract, not as many features and not open source for some reason, but it gets the job done…

There is an extended Magisk module template available, that I’ve seen used in some modules, but only adds unnecessary complexity to this project.
The simple folder structure outlined in the docs and the included functions are absolutely sufficient to get this going.

Smojo · August 22, 2022, 8:34am

This was really simple, thanks for the hint. I probably read about nix one day (not sure), but never used it until now.

sh <(curl -L https://nixos.org/nix/install) --daemon
nix-env --install android-tools

At least way simpler than building the android-tools from scratch. There were so many dependencies which eventually needed building additional tools. I was a bit overwhelmed and it felt like that this exceeds my knowledge/skills.

Only thing I would like to solve is that nix is setting itself at the beginning of the PATH variable so now it is also the primary adb and fastboot included in the android tools:

which adb
~/.nix-profile/bin/adb

adb --version
Android Debug Bridge version 1.0.41
Version 33.0.2-8557947
Installed as ~//Software/AndroidSDK/platform-tools/adb
## (still uses the other one, as I set an alias in ~/.bash_aliases in addition)

~/.nix-profile/bin/adb --version
Android Debug Bridge version 1.0.41
Version 31.0.3p1-android-tools
Installed as /nix/store/kashma2vqhr321xdw7smpx65dzj0ki9w-android-tools-31.0.3p1/bin/adb
## (slightly older adb so I would prefer my manual installation as default)

echo $PATH
<myhome>/.nix-profile/bin:/nix/var/nix/profiles/default/bin:<myhome>/Software/AndroidSDK/platform-tools:<myhome>//.local/bin: ....

We run a bit offtopic but there is probably an easy way to
a) change the order and set nix to the second position (it is not in ~/.profile)
b) exclude adb and fastboot from PATH
c) install only adb and fastboot from nix (nix android-tools)

xblax · August 22, 2022, 9:28am

I’m also using Debian. Never had problems with ADB and Fastboot versions included in the distribution for anything with the Fairphone 4.

Btw. I’m quite sure that FPCamera.apk will not work out-of the box on Calyx even with System Camera permission. I tried it on LineageOS 19.1 yesterday with my changes to make it work on LOS 18.1. There are further changes required to make it work.

Installing OpenCamera might get you better results for now. It’s very easy to add the required permission. Download source code from Open Camera - Browse /v_1_50_1 at SourceForge.net, edit AndroidManifest.xml and build APK with gradlew assembleDebug - Gradle downloads all build-dependencies automatically.

hirnsushi · August 22, 2022, 11:36am

It doesn’t have to be fully usable, it’s only meant as a stepping stone to check if there aren’t any other issues with the Magisk module. Wide-angle should work when started from a shortcut, and it’s possible to check if it properly got the SYSTEM_CAMERA permission.
It’s the first proper Magisk module for @Smojo (I guess), so I think it makes sense to start with something that has all the necessary parts already in place and can more or less just be copied over from stock, as to not introduce additional potential points of failure in the beginning.

Consider this thread as hacking towards the goal at a nice relaxing pace, the next step would be OpenCamera, yes

Having your tools properly set up is an important part of any project, and I don’t think a lot of people care anyway if this thread stays on topic

I know android-tools are a (little) bit behind upstream, those adb and fastboot versions work fine for me though.
If you still want to change the order in your $PATH, check /etc/profile.d/ for nix related files, the installer could also have modified either ~/.bash_profile, ~/.bash_login, or (apparently not in your case) ~/.profile. I’m on a different distro and installed nix through my package manager, so even with the install script before me, I’m still not sure where it might be in your case

Smojo · August 23, 2022, 6:27am

Ok, let’s close the nix and profile discussion.

I tried to figure out how nix is dealing with it but the daemon somehow puts nix installed binaries at the front of the PATH at the very end of a shell/terminal start.
At bit more research about how nix works and maybe limit the build tools to a specific nix-shell or so might help here as well ^^, but I feel like we really lose the focus of what is planned to be done in this topic → building the “camera as system app” magisk module.

So here what I did for now:

# the file
/etc/profile.d/nix.sh
# points to: 
/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh
# which is read-only and cannot be adjusted (without modifying it what I would not do/recommend as I don't want to "hack" nix that way) 
# and it consists of several lines but also the one I was looking for:
--> export PATH="$HOME/.nix-profile/bin:/nix/var/nix/profiles/default/bin:$PATH"

As it feels quite special what I’m complaining about ("I want a specific adb+fastboot to be used, while still using the build tools I got via nix?), I just renamed the two binaries I was complaining about. This will at least prevent for now, that I accidentally use them:

ls -l /nix/store/ | grep android-tools
-r--r--r-- 1 root   root       2933  1. Jan 1970  16xcqfjlbc48f8j7bg0lr4bbw314dvw4-android-tools-31.0.3p1.tar.xz.drv
-r--r--r-- 1 root   root       3505  1. Jan 1970  c182syxdfjwrx1apm5s5am40g4qfwyvz-android-tools-31.0.3p1.drv
dr-xr-xr-x 5 root   root       4096  1. Jan 1970  kashma2vqhr321xdw7smpx65dzj0ki9w-android-tools-31.0.3p1

cd /nix/store/kashma2vqhr321xdw7smpx65dzj0ki9w-android-tools-31.0.3p1/bin
mv adb adb.disabled
mv fastboot fastboot.disabled

(maybe not very sustainable in case the nix android tools will be updated or so, but anyway it’s fine for now)

@xblax In general to answer “Why do you need a newer adb or fastboot and don’t just use the debian default one?” There was a requirement by Calyx to update this (at least fastboot) otherwise the flash script will not go on? I also saw some people reported issues and had to update it. I double checked the flash-all.sh of FP4 Calyx factory image and found this:

if ! [ $($(which fastboot) --version | grep "version" | cut -c18-23 | sed 's/\.//g' ) -ge 3103 ]; then
  echo "fastboot too old; please download the latest version at https://developer.android.com/studio/releases/platform-tools.html"
  exit 1
fi

→ has to be -ge (“greater than or equal”) 3103

Debian/PureOS: Version 28.0.2-debian
nix: Version 31.0.3p1-android-tools
manual downloaded: Version 33.0.2-8557947

→ conclusion: nix is still ok, debian too old (at least for Calyx, Lineage might not be that picky - at least I always used the debian version with my FP2), so it was ok/needed for me to update it before flashing CalyxOS (for the FP4).

Smojo · August 23, 2022, 7:18am

and of course I understood that this AndroidManifest.xml code change needs the sources of the Camera App, which has to be patched, and recompile it to an APK. I currently don’t have the tools (like gradle etc.) in place to do this. I’m more the user kind of person, who flashes a CustomRom from time to time. My Linux and Android flashing knowledge is not zero, but this whole Magisk module creation is quite challenging for me already.

So let’s focus on the module and maybe the APK patching will follow later as well.

Smojo · August 23, 2022, 8:36pm

I think I finally have all the stuff I need. Didn’t had much time to work on this.

Extracting the APK from a ROM zip

download and extract the factory ROM zip, the Camera app you need, is included

cd <extracted-factory-zip-folder>/images

simg2img super.img super.raw.img
lpunpack system_a super.raw.img

# at this point: as the extracted factory ROM zip is quite big, 
# I moved the system_a.img to another directory and deleted the extracted folder

mkdir /tmp/fp4fpos-system_a
sudo mount -t ext4 -o loop,ro system_a.img /tmp/fp4fpos-system_a

After that the APK can be found in:
→ /tmp/fp4fpos-system_a/system/priv-app/FPCamera

Side topic/question:
There is a folder oat/arm64 (also within /system/priv-app/FPCamera)
→ with these two files:

FPCamera.odex
FPCamera.vdex

What are they used for?

I also set up apktool (instruction here: Apktool - How to Install), as I was interested to get the package name from an APK. I only have openjdk version “11.0.16” 2022-07-19 and I didn’t check the versioning mapping to the offical Java, but it worked so seems to be ok (as Java 1.8 is required).
EDIT: There are a bunch of articles about this out there. Java versioning schema changed a bit over the time (also because of Oracle bought Sun etc.) 1.8 is basically Java 8. The Open jdk, my debian based OS provides, is Version 11.x (so new enough).

apktool decode --output /tmp/FPCamera --force-all FPCamera.apk

# checked the AndroidManifest.xml in the output folder
less /tmp/FPCamera/AndroidManifest.xml `

# you will see package name within the manifest tag <manifest ...> 
... package="com.fp.camera" ...

Will create the module next as soon as I have some spare time.

hirnsushi · August 24, 2022, 2:28pm

From the docs:

.vdex: contains the uncompressed DEX code of the APK, with some additional metadata to speed up verification.

.odex: contains AOT compiled code for methods in the APK.

In short, preoptimized code, but that’s as far as my understanding goes as well, not an Android developer either
Just copy the whole /system/priv-app/FPCamera folder over and use it in your module.

If all you need is the package name, aapt2 dump packagename foo.apk will get you there quicker.
aapt2 is part of android-sdk-build-tools (there is a Debian package ), which you might need in the future anyway.

But since you have the AndroidManifest.xml already in front of you, you can also have a look where / how the SYSTEM_CAMERA permission is declared in there

BTW, there’s an apktool nix package available (apparently there’s a nix package for everything ), manually having to copy binaries to /usr/local/bin that aren’t being tracked by any package manager, that hurts a little

Smojo · August 24, 2022, 3:40pm

Ok, let’s go that way.

This is the whole permission part:

    <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION"/>
    <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION"/>
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
    <uses-permission android:name="android.permission.ACCESS_WIFI_STATE"/>
    <uses-permission android:name="android.permission.CAMERA"/>
    <uses-permission android:name="android.permission.SYSTEM_CAMERA"/>
    <uses-permission android:name="android.permission.CHANGE_WIFI_STATE"/>
    <uses-permission android:name="android.permission.INTERNET"/>
    <uses-permission android:name="android.permission.READ_SYNC_SETTINGS"/>
    <uses-permission android:name="android.permission.RECORD_AUDIO"/>
    <uses-permission android:name="android.permission.VIBRATE"/>
    <uses-permission android:name="android.permission.WAKE_LOCK"/>
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
    <uses-permission android:name="android.permission.WRITE_SETTINGS"/>
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE"/>
    <uses-permission android:name="android.permission.WRITE_SYNC_SETTINGS"/>
    <uses-permission android:name="android.permission.ACCESS_NOTIFICATION_POLICY"/>
    <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/>

I might need? For? Compiling code to an APK?

You are right and as I have nix in place it is probably the better way to set in up.

hirnsushi · August 24, 2022, 3:54pm

We don’t actually need to do something with that information at the moment, but it will become important later, when you want to add that permission to OpenCamera for example.

The more important file you are missing is /system/etc/permissions/privapp-permissions-platform.xml, take a note of the structure of that file and the part that’s related to com.fp.camera.
You need to create a similar privapp-permissions-com.fp.camera.xml for your module, containing only the part that’s needed for FP Camera:

<?xml version="1.0" encoding="utf-8"?>
<permissions>
    <privapp-permissions package="com.fp.camera">
        <permission name="android.permission.SYSTEM_CAMERA"/>
        <permission name="android.permission.WRITE_MEDIA_STORAGE"/>
    </privapp-permissions>
</permissions>

For example, or just to have at your disposal when playing around. I’ve used aapt2 in several scripts, it’s quite useful

Smojo · August 24, 2022, 4:15pm

Yes, understood that already. I was just a bit confused about all the privs listed in the FPcamera APK’s manifest file (but it is part of the APK anyway so no additional “to-do” for the module - at least not for the FPcamera).

Btw, I searched for ‘camera’ in the /system/etc/permissions/privapp-permissions-platform.xml (in the mounted system_a image) and found an additional occurance of

    <privapp-permissions package="com.android.shell">
    ...
        <!-- Permission required to test system only camera devices. -->
        <permission name="android.permission.SYSTEM_CAMERA" />
    ...

hirnsushi · August 24, 2022, 5:36pm

Yeah, I noticed that as well.

There’s probably a reason com.android.shell has access to all the things, including SYSTEM_CAMERA.
I would guess debugging

Smojo · August 24, 2022, 8:23pm

Output (saved log) of the Magisk Module installation:

- Copying zip to temp directory
- Installing CamAsSystemApp.zip
- Current boot slot: _a
- Device is system-as-root
Archive:  /data/user/0/com.topjohnwu.magisk/cache/flash/install.zip
  inflating: module.prop
*************************
 Cam as system app (FP4) 
 by Smojo 
*************************
Archive:  /data/user/0/com.topjohnwu.magisk/cache/flash/install.zip
  inflating: customize.sh
*******************
 Powered by Magisk 
*******************
- Extracting module files
Archive:  /data/user/0/com.topjohnwu.magisk/cache/flash/install.zip
   creating: system/
   creating: system/priv-app/
   creating: system/priv-app/FPCamera/
  inflating: system/priv-app/FPCamera/FPCamera.apk
   creating: system/priv-app/FPCamera/oat/
   creating: system/priv-app/FPCamera/oat/arm64/
  inflating: system/priv-app/FPCamera/oat/arm64/FPCamera.odex
  inflating: system/priv-app/FPCamera/oat/arm64/FPCamera.vdex
   creating: system/etc/
   creating: system/etc/permissions/
  inflating: system/etc/permissions/privapp-permissions-com.fp.camera.xml
  inflating: customize.sh
  inflating: module.prop
Setting permissions of installed file(s)
Done
- Done

Looks okayish. I tend to reboot now. Any objections? ^^

hirnsushi · August 24, 2022, 8:33pm

Nope, looks good

Let’s see if everything worked
First thing you should check is dumpsys package com.fp.camera.

The main activity will crash, but if you open the app via long press shortcut and choose Pro for example, you should be able to switch lenses.

Smojo · August 25, 2022, 6:02am

Yes it worked.

Overview of a quick “Test Camera Functionality - FPCamera (as system app)”:

Normal Mode
(scoop crash log attached at the end of this post - basically: “unable to retrieve camera characteristics”)
Pro Mode
Switch lenses (to wide angle lense)
Panorama
HiRes - 48MP Mode
(accessible + possible to take pictures, but really 48MP? My test shot had 2.8MP ^^)
Video
Time Lapse (speed?)
SloMo (speed?; crashes after some sec)

What is the issue with the normal mode? (how did you even discover that the Pro mode works? At that point I would have stopped and put it aside saying:“Hm doesn’t work.” )

Scoop App - Crash capture of FPCamera (normal mode)

click to show ...

FATAL EXCEPTION: ControlHandler
Process: com.fp.camera, PID: 23315
java.lang.IllegalArgumentException: getCameraCharacteristics:754: Unable to retrieve camera characteristics for unknown device : No such file or directory (-2)
at android.hardware.camera2.CameraManager.throwAsPublicException(CameraManager.java:1263)
at android.hardware.camera2.CameraManager.getCameraCharacteristics(CameraManager.java:591)
at android.hardware.camera2.CameraManager.getPhysicalIdToCharsMap(CameraManager.java:630)
at android.hardware.camera2.CameraManager.openCameraDeviceUserAsync(CameraManager.java:665)
at android.hardware.camera2.CameraManager.openCameraForUid(CameraManager.java:958)
at android.hardware.camera2.CameraManager.openCameraForUid(CameraManager.java:979)
at android.hardware.camera2.CameraManager.openCamera(CameraManager.java:817)
at com.android.camera.device.OneCameraImpl.doOpenCamera(OneCameraImpl.java:357)
at com.android.camera.device.OneCameraImpl$CameraHandler.handleMessage(OneCameraImpl.java:214)
at android.os.Handler.dispatchMessage(Handler.java:106)
at android.os.Looper.loopOnce(Looper.java:201)
at android.os.Looper.loop(Looper.java:288)
at android.os.HandlerThread.run(HandlerThread.java:67)
Caused by: android.os.ServiceSpecificException: getCameraCharacteristics:754: Unable to retrieve camera characteristics for unknown device : No such file or directory (-2) (code 3)
at android.os.Parcel.createExceptionOrNull(Parcel.java:2440)
at android.os.Parcel.createException(Parcel.java:2410)
at android.os.Parcel.readException(Parcel.java:2393)
at android.os.Parcel.readException(Parcel.java:2335)
at android.hardware.ICameraService$Stub$Proxy.getCameraCharacteristics(ICameraService.java:812)
at android.hardware.camera2.CameraManager.getCameraCharacteristics(CameraManager.java:570)
... 11 more

Anything special you look for in dumpsys package com.fp.camera?