Hi @wolf,
this is not related to Fairphone OS, but a feature in Android itself. I think it was implemented in Android 4.4 (KitKat).
There is plenty of information about this notification and how to get rid of it on the web. One way to circumvent this is to move the certificate to the system truststore (requires root). You could also build Fairphone Open Source OS by yourself and add the certificates before compiling.
jnsp