I found this old topic about problems with screenshots and it suggests that for some people it’s not easy to press both buttons simultaneously. But with the right timing it should work. Also you only have to hold the buttons for a second so there is no chance to do a hard reset.
PS: If you don’t get it to work at all you can also use apps like AndroSS or ChronoSnap (I haven’t tried them) or take a pic with some other camera.