One option would be to keep the contacts in two different address books or accounts (e.g. when synching them with your own Nextcloud server) and switch the account to be displayed on and off as required in the contacts app.
If you are on LineageOS you could also use groups to distinguish your contacts.
However none of these solutions will switch automatically based on the active SIM, so you might consider those more as workarounds.