Hi
I donloaded the blobs and checked if the checksum stored in the file compared to the archive matches but it seams like its different. Maybe im wron but shouldn’t if match with the .tgz file?
It appears to me that you compared a shell script (.sh) to a gzipped tarball (.tgz). Of course these files are different. Maybe you should uncompress the tgz file first and check its insides afterwards.
That MD5 sum is stored as string variable rather than included as a comment. The script at some point uses that string variable to verify that what it extracts matches that sum. So it’s not the file as a whole that’s being tested, but by the looks of it everything but the first n lines, which contain the script.
fp2-sibon-2.0.0-blobs.sh --check
Will check whether the archive is intact.
Update:
Long version:
It first calculates the offset at which the archive is contained in the file:
head -n 576 fp2-sibon-2.0.0-blobs.sh | wc -c | tr -d " "
This returns 15662, which is then passed to dd as the read size, with the argument to skip the first read. It then passes through dd again to check whether the information is in the expected number of blocks and bytes. The value for the expected number of 1024 byte blocks is calculated from the filesizes variable in the script (202174062 for the 2.0.0 blobs) by dividing it by 1024 (and using the integer) and calculating the modulo. For me, the below matches the md5sum for the 2.0.0 blobs.
dd if=fp2-sibon-2.0.0-blobs.sh ibs=15662 skip=1 obs=1024 conv=sync 2> /dev/null | { dd ibs=1024 obs=1024 count=197435; dd ibs=1 obs=1024 count=622; } 2> /dev/null | md5sum
TLDR: the md5sum refers to a derivative of the install script, rather than the file itself or any files produced by it. Hope this helps.
Not using the shell script at all for packing the blobs would also help 
Update: The best way to understand what @Johannes is talking about is to use sh -x with the script that unpacks the tar ball if I recall correctly. I haven’t done this for a while.
Sidenote: Comparing the single bin blobs is complicated due to all the java (yes, java) that is also included in the “bin” blobs. You need to unpack them as well if you want to do a good comparison (if you want to do compare more than just the overall checksum of the tar ball) to see what really has changed between the releases.
Thanks for that verry exact explaination!
I was able to reproduce that failure i got by building.
someware between yesterday and 11.5.2016 that checksum changed.
If i calculate that with the terminalcodes you gave me it spits out the exact same checksum as my build is complaining to be not the one stored in the file…
the build:
For each version the offset and filesize will be different. I’ve manually entered the filesize variable below (find it by e.g head -n 567 fp2-sibon-2.0.1-blobs.sh | grep "^filesizes=\""; automating it is possible, but at some point you’d have re-written the script that’s in the blob files, hence the comment that it may be easier to let the blob file self-validate).
~/Downloads/FP2> dd if=fp2-sibon-2.0.0-blobs.sh ibs=$(head -n 576 fp2-sibon-2.0.0-blobs.sh | wc -c | tr -d " ") skip=1 obs=1024 conv=sync 2> /dev/null | { dd ibs=1024 obs=1024 count=$(expr 202174062 / 1024); dd ibs=1 obs=1024 count=$(expr 202174062 % 1024); } 2> /dev/null | md5sum 14351800dc7c3896b060f1033c65aebc -
~/Downloads/FP2> dd if=fp2-sibon-2.0.1-blobs.sh ibs=$(head -n 576 fp2-sibon-2.0.1-blobs.sh | wc -c | tr -d " ") skip=1 obs=1024 conv=sync 2> /dev/null | { dd ibs=1024 obs=1024 count=$(expr 208350522 / 1024); dd ibs=1 obs=1024 count=$(expr 208350522 % 1024); } 2> /dev/null | md5sum 2769f5a7cf4b725fb1b3bdb96a3f9f06 -
If you just want to verify whether the archive is OK, I’d still go for ./fp2-sibon-2.0.1-blobs.sh --check, but I guess that you mean this isn’t working for you either. I notice you have a slightly different md5sum command, are you running (free)BSD by any chance?
And if it helps, here are the MD5 sums I get for the files (that successfully self-validate):
~/Downloads/FP2> md5sum fp2-sibon-2.0.* 67ed2a970003f084a0487ca5d4a028fe fp2-sibon-2.0.0-blobs.sh 27174409600f3a29a5345c43789fae83 fp2-sibon-2.0.0-blobs.tgz 3579d0cc6ff9a4ca96f0c106017acc87 fp2-sibon-2.0.1-blobs.sh 6ced270ce58073b3c875c9e1d7250d12 fp2-sibon-2.0.1-blobs.tgz
P.S. That value of 567 for the length of the script part of the file came from the file too - so it may change in later versions…
Hi Johannes and many thanks for that fast reply…
I can not follow because im not a prof. coder. I just try and error some interesting stuff… 
so i tried to execute this;
and i got this out of my OSX 10.11.4 Terminal window:
A self-extracting shell script to test something on different unix systems … there are lot of things that could go wrong.
A plain tar ball compressed with LZMA2/xz and a published and signed checksum would be more useful. But you cannot trust the bin blobs anyway.
47M fp2-sibon-2.0.3-blobs.sh
34M fp2-sibon-2.0.3-blobs.tar.xz
Difficult to say, but one of the tools may be returning something slightly different on OSX than what the linux tools do. As I do not have access to OSX, I can’t really help. The md5 command itself does seem to give the same output, as your checksums for the tgz seem to match what I get…
So, - would that mean to check the container (.tgz) instead of the file (.sh) would solve the problem?
Or how can i understand that?
What I meant is that because the output of the md5 command you posted in your first post matches the one that I get for the same file using the md5sum command on Linux, we can assume that:
Which in turn means the reason it isn’t self validating correctly is probably due to a difference in one of the other parts of the series of commands that the self verification steps run. The link that @fp1_wo_sw_updates posted gives an example that on some Linuxes the dd command doesn’t support some of the options used. This would directly lead to a different output from dd, which gives a different md5sum (despite the file being intact in both cases). Now if the script uses the same method to extract the binaries (sorry, I don’t have time to check), the md5sum mismatching in those cases is actually a good thing - as it means that even if you’d ignore the checksum mismatch, you’d still get the wrong output from an intact file.
Makeself packages (such as this one) are listed as working with MacOSX, but I’m not familiar enough with Mac, Makeself and probably a whole other series of things to make an informed judgement on what could be the cause of the issue.
If unpacking on the Ubuntu VM works, is copying them over from there an option for you?
I’m against using Makeself at all. It makes it all more complicated. Those installers are mainly made to install proprietary gaming software onto Unix systems quickly the “Windows” way (“Setup.exe/Setup.bat”).
But this is about compiling your own rom/phone software. The easiest way to do this would be a plain compressed tar ball for the bin blobs. For this tar ball Fairphone just publishes the checksum. Done.
If one wants to be extra fancy, Fairphone can also sign the bin blob tar ball and the code that matches each official release code release (that fits the bin blob) with a PGP/GPG key.
Let me go a steb back:
This MD5SUM is from your post:
My MD5 one post further up was this (in a Printscreen: “Bildschirmfoto 16-05-23 um 20.22.21”):
27174409600f3a29a5345c43789fae83 fp2-sibon-2.0.0-blobs.tgz
So to me the MD5SUM and the osx MD5 of the fp2-sibon-2.0.0-blobs.tgz files is exactly the same.
Am i missing someting?
Because that wold maybe mean, that changeing the check from .sh file to .tgz compressed-file wold solve the issue already… doesn’t it?
Regards, Novski
Correct, the file you downloaded is the same as the one that I have.[quote=“novski, post:13, topic:18107”]
Because that wold maybe mean, that changeing the check from .sh file to .tgz compressed-file wold solve the issue already… doesn’t it?
[/quote]
The md5sum stored in the file is the md5sum of only a part of the file. The check that is performed as part of the unpack procedure is therefore not against the entire file, but rather against a piece of it that is extracted in a particular way. What’s going wrong is that extraction process.
So the extracted blobs may be injured anyway and a build wold not work because of broken blobs?
If you bypass the md5sum verification that is likely to happen, yes. Though more likely unpacking the blobs will just fail, rather than producing broken blobs. The ones unpacked in your Ubuntu virtual machine probably will work in your build environment.
This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.