Fail to completely restore Google backup

I did a factory reset of my phone since it was getting increasingly slower and apps kept crashing. I had made a backup on Google One, but restoring the backup on my phone has been a huge hassle. I finally managed to get the apps reinstalled, but nothing else. Though when I check the backup online at Google 1 it says it includes sms, photos, backgrounds etc etc. Why are these not automatically restored? Very grateful if someone could give me any tips on how to proceed.

When I restarted the phone, I declined to restore from another device and instead chose to restore from my Google-account. I got the message that it would restore everything, but that it could take a while and that I could use the phone meanwhile. But nothing more happened.